樸素貝葉斯分類器

樸素貝葉斯是一種基于密度估計的分類算法，它利用貝葉斯定理進行預測。該算法的核心假設是在給定類別的情況下，各個特征之間是條件獨立的，盡管這一假設在現實中通常不成立，但樸素貝葉斯分類器依然能夠生成對有偏類密度估計具有較強魯棒性的后驗分布，尤其是在后驗概率接近決策邊界（0.5）時。

樸素貝葉斯分類器通過最大后驗概率決策規則將觀測值分配到最有可能的類別。

具體步驟如下：

1. 密度估計：計算每個類別中各特征的密度分布。
2. 后驗概率建模：根據貝葉斯公式計算后驗概率。對于所有類別$k=1,\dots ,K$，

$$\hat{P}(Y=k|X_1,\dots ,X_d)=\frac{P(Y=k)\prod _{j=1}^{d}P(X_j|Y=k)}{{\sum }_{k=1}^{K}P(Y=k)\prod _{j=1}^{d}P(X_j|Y=k)},$$

其中：

• $Y$表示觀測值所屬類別的隨機變量。
• $X_1,\dots ,X_d$是觀測值的特征變量。
• $P(Y=k)$是類別$k$的先驗概率。

3. 分類決策：通過比較不同類別的后驗概率，將觀測值歸類到后驗概率最大的類別中。

兩類密度估計方法

Normal (Gaussian) Distribution

The ‘normal’ distribution (specify using ‘normal’) is appropriate for predictors that have normal distributions in each class. For each predictor you model with a normal distribution, the naive Bayes classifier estimates a separate normal distribution for each class by computing the mean and standard deviation of the training data in that class.

Kernel Distribution

The ‘kernel’ distribution (specify using ‘kernel’) is appropriate for predictors that have a continuous distribution. It does not require a strong assumption such as a normal distribution and you can use it in cases where the distribution of a predictor may be skewed or have multiple peaks or modes. It requires more computing time and more memory than the normal distribution. For each predictor you model with a kernel distribution, the naive Bayes classifier computes a separate kernel density estimate for each class based on the training data for that class. By default the kernel is the normal kernel, and the classifier selects a width automatically for each class and predictor. The software supports specifying different kernels for each predictor, and different widths for each predictor or class.

定理 設$(X_1,X_2,?,X_n)$是$n$維連續型隨機變量，$f(x_1,x_2,?,x_n)$是其聯合概率密度函數，$f_{X_i}(x_i)$是關于$X_i(i=1,2,?,n)$的邊緣概率密度函數，則隨機變量$X_1,X_2,?,X_n$相互獨立等價于

$$f(x_1,x_2,?,x_n)=\prod _{i=1}^n{f}_{X_i}(x_i),$$

其中$(x_1,x_2,?,x_n)$為任意的實數組。