朴素贝叶斯分类算法预测流程详解：从数学原理到工程实现

1次阅读

没有评论

共计 2634 个字符，预计需要花费 7 分钟才能阅读完成。

在文本分类和垃圾邮件过滤等场景中，朴素贝叶斯算法因其高效和简单而广受欢迎。它的核心优势在于计算速度快、对少量训练数据表现良好，并且容易实现。特别是在处理高维稀疏数据时，朴素贝叶斯能够快速给出预测结果，这使得它成为许多实际应用中的首选算法。

朴素贝叶斯基于贝叶斯定理，假设特征之间条件独立。后验概率计算公式如下：

$$ P(y|x_1, x_2, …, x_n) = \frac{P(y)\prod_{i=1}^{n}P(x_i|y)}{P(x_1, x_2, …, x_n)} $$

其中，$P(y)$ 是类别的先验概率，$P(x_i|y)$ 是特征 $x_i$ 在类别 $y$ 下的条件概率。条件独立性假设虽然简化了计算，但也可能影响模型的表现，特别是在特征间存在强相关性的情况下。

下面是一个 Python 实现的示例，包含了平滑处理和对数空间计算：

from typing import Dict, List
import numpy as np
from scipy.sparse import csr_matrix

class NaiveBayesClassifier:
    def __init__(self, alpha: float = 1.0):
        self.alpha = alpha  # Laplace smoothing parameter
        self.class_priors: Dict[int, float] = {}
        self.feature_probs: Dict[int, Dict[int, float]] = {}

    def fit(self, X: csr_matrix, y: List[int]) -> None:
        """Fit the model using sparse matrix X and labels y."""
        n_samples, n_features = X.shape
        classes = np.unique(y)

        # Calculate class priors with Laplace smoothing
        for c in classes:
            self.class_priors[c] = (np.sum(y == c) + self.alpha) / (n_samples + len(classes) * self.alpha)

        # Calculate feature probabilities in log space
        for c in classes:
            class_mask = (y == c)
            class_count = np.sum(class_mask)

            # Sum features for current class
            if isinstance(X, csr_matrix):
                feature_counts = X[class_mask].sum(axis=0).A1
            else:
                feature_counts = X[class_mask].sum(axis=0)

            # Apply smoothing and convert to log probabilities
            total_count = class_count + 2 * self.alpha
            self.feature_probs[c] = {f: np.log((feature_counts[f] + self.alpha) / total_count)
                for f in range(n_features)
            }

    def predict_proba(self, X: csr_matrix) -> np.ndarray:
        """Predict class probabilities for samples in X."""
        n_samples = X.shape[0]
        classes = sorted(self.class_priors.keys())
        probs = np.zeros((n_samples, len(classes)))

        for i, c in enumerate(classes):
            # Start with log of class prior
            log_prob = np.log(self.class_priors[c])

            # Add log probabilities of features
            if isinstance(X, csr_matrix):
                for row in range(n_samples):
                    _, cols = X[row].nonzero()
                    log_prob_row = log_prob + sum(self.feature_probs[c].get(f, np.log(self.alpha / (np.exp(self.class_priors[c]) + 2 * self.alpha)))
                        for f in cols
                    )
                    probs[row, i] = np.exp(log_prob_row)
            else:
                for row in range(n_samples):
                    log_prob_row = log_prob + sum(self.feature_probs[c].get(f, np.log(self.alpha / (np.exp(self.class_priors[c]) + 2 * self.alpha))) * val
                        for f, val in enumerate(X[row]) if val != 0
                    )
                    probs[row, i] = np.exp(log_prob_row)

        # Normalize probabilities
        probs = probs / probs.sum(axis=1, keepdims=True)
        return probs