這一節我們用TensorFlow定義簡單的分類器。首先考慮分類器的方程式是什么是值得的。數學習的技巧是使用sigmoid函數。sigmoid函數繪制如圖3-40, 通常標記為σ, 是實數域里的函數取值(0, 1)。這個特征很便利，因為我們可以將sigmoid的輸出解釋為事件發現的概率。? (轉換離散事件到連續值是機器學習里反復出現的主題)

圖3-40. 繪制sigmoid 函數.

預測離散事件的概率的方程式如下。這些方程式定義了簡單的邏輯回歸模型:

y0 = σ（ wx + b）

y1 = 1 ? σ （wx + b）

TensorFlow提供了工具函數來計算sigmoidal值的交叉熵損失。最簡單的函數是tf.nn.sigmoid_cross_entropy_with_logits. ( logit是sigmoid的逆。實際上，這意味著傳遞參數到 sigmoid, wx + b, 而不是sigmoidal value σ wx + b 本身)。我們推薦使用 TensorFlow的實現而不是手工定義交叉熵,因為計算交叉熵損失有許多復雜的數值問題。

#List3-44

import numpy as np

np.random.seed(456)

import tensorflow as tf

#tf.set_random_seed(456)

import matplotlib.pyplot as plt

from sklearn.metrics import accuracy_score

from scipy.special import logit

# Generate synthetic data

N = 100

# Zeros form a Gaussian centered at (-1, -1)

x_zeros = np.random.multivariate_normal(mean=np.array((-1, -1)), cov=.1*np.eye(2), size=(N//2,))

y_zeros = np.zeros((N//2,))

# Ones form a Gaussian centered at (1, 1)

x_ones = np.random.multivariate_normal(mean=np.array((1, 1)), cov=.1*np.eye(2), size=(N//2,))

y_ones = np.ones((N//2,))

x_np = np.vstack([x_zeros, x_ones])

y_np = np.concatenate([y_zeros, y_ones])

# Save image of the data distribution

plt.xlabel(r"$x_1$")

plt.ylabel(r"$x_2$")

plt.title("Toy Logistic Regression Data")

# Plot Zeros

plt.scatter(x_zeros[:, 0], x_zeros[:, 1], color="blue")

plt.scatter(x_ones[:, 0], x_ones[:, 1], color="red")

plt.savefig("logistic_data.png")

x_np,y_np

模型的訓練代碼見List3-45 ，與線性回歸模型的代碼相同。

#List3-45

W = tf.Variable(tf.random.normal((2, 1)))

b = tf.Variable(tf.random.normal((1,)))

W,b

x=tf.cast(x_np,tf.float32)

y=tf.cast(y_np,tf.float32)

learning_r=0.01

optimizer = tf.optimizers.SGD(learning_r)

n_steps = 100

# Train model

for i in range(n_steps):???

??? with tf.GradientTape() as tape:

??????? #_, summary, loss = sess.run([train_op, merged, l], feed_dict=feed_dict)

??????? y_logit = tf.squeeze(tf.matmul(x, W) + b)

??????? # the sigmoid gives the class probability of 1

??????? y_one_prob = tf.sigmoid(y_logit)

??????? # Rounding P(y=1) will give the correct prediction.

??????? y_pred = tf.round(y_one_prob)

??????? entropy = tf.nn.sigmoid_cross_entropy_with_logits(logits=y_logit, labels=y)

??????? # Sum all contributions

??????? l = tf.reduce_sum(entropy)???????

??????? gradients=tape.gradient(l,[W,b])

??? optimizer.apply_gradients(zip(gradients, [W, b]))

??? #W=W-tf.Variable(learning_r,tf.float32)*W

??? #b=b-tf.Variable(learning_r,tf.float32)*b

??? print("loss: %f" % l)

??? #train_writer.add_summary(summary, i)

??? # Get weights

??? w_final=W

??? b_final=b

# Make Predictions

??? #y_pred_np = sess.run(y_pred, feed_dict={x: x_np})

#score = accuracy_score(y_np, y_pred_np)

#print("Classification Accuracy: %f" % score)

plt.clf()

# Save image of the data distribution

plt.xlabel(r"$x_1$")

plt.ylabel(r"$x_2$")

plt.title("Learned Model (Classification Accuracy: 1.00)")

plt.xlim(-2, 2)

plt.ylim(-2, 2)

# Plot Zeros

plt.scatter(x_zeros[:, 0], x_zeros[:, 1], color="blue")

plt.scatter(x_ones[:, 0], x_ones[:, 1], color="red")

x_left = -2

y_left = (1./w_final[1]) * (-b_final + logit(.5) - w_final[0]*x_left)

x_right = 2

y_right = (1./w_final[1]) * (-b_final + logit(.5) - w_final[0]*x_right)

plt.plot([x_left, x_right], [y_left, y_right], color='k')

plt.savefig("logistic_pred.png")

圖3-41