深入解析BP神经网络：从数学原理到Python实战

1次阅读

共计 2416 个字符，预计需要花费 7 分钟才能阅读完成。

BP 神经网络（Backpropagation Neural Network）虽然结构简单，但实际训练中常遇到以下典型问题：

梯度消失（Vanishing Gradient）：深层网络中，梯度在反向传播时呈指数级衰减，导致底层参数几乎不更新
收敛速度慢（Slow Convergence）：传统梯度下降（Gradient Descent）采用固定学习率时，容易在平坦区域停滞
过拟合（Overfitting）：当网络容量过大时，会在训练集上表现很好但泛化能力差

这些痛点直接影响了模型的实用效果，接下来我们将通过数学原理和代码实现来逐个破解。

设网络有 $L$ 层，第 $l$ 层的输出为：
$$\mathbf{a}^l = \sigma(\mathbf{z}^l) = \sigma(\mathbf{W}^l\mathbf{a}^{l-1} + \mathbf{b}^l)$$
其中 $\sigma$ 是激活函数（如 Sigmoid），$\mathbf{W}^l$ 为权重矩阵。

采用均方误差（MSE）：
$$J(\mathbf{W},\mathbf{b}) = \frac{1}{2m}\sum_{i=1}^m ||\mathbf{y}_i – \mathbf{a}^L_i||^2$$

输出层误差：
$$\delta^L = \nabla_{\mathbf{a}}J \odot \sigma'(\mathbf{z}^L)$$
隐藏层误差传播：
$$\delta^l = ((\mathbf{W}^{l+1})^T\delta^{l+1}) \odot \sigma'(\mathbf{z}^l)$$
参数梯度计算：
$$\frac{\partial J}{\partial \mathbf{W}^l} = \delta^l (\mathbf{a}^{l-1})^T$$
$$\frac{\partial J}{\partial \mathbf{b}^l} = \delta^l$$

import numpy as np

class NeuralNetwork:
    def __init__(self, layers):
        """
        网络初始化
        :param layers: 各层神经元数量，如 [784, 256, 10]
        """
        self.weights = [np.random.randn(y, x)*0.01 
                        for x, y in zip(layers[:-1], layers[1:])]
        self.biases = [np.zeros((y, 1)) for y in layers[1:]]

    def sigmoid(self, z):
        return 1/(1+np.exp(-z))

    def forward(self, x):
        """前向传播"""
        a = x
        for w, b in zip(self.weights, self.biases):
            z = np.dot(w, a) + b
            a = self.sigmoid(z)
        return a

    def train(self, X, y, lr=0.1, epochs=100):
        """训练过程"""
        for epoch in range(epochs):
            # 动态学习率衰减
            current_lr = lr * (0.95 ** epoch)

            # 正向传播
            activations = [X]
            zs = []
            for w, b in zip(self.weights, self.biases):
                z = np.dot(w, activations[-1]) + b
                zs.append(z)
                activations.append(self.sigmoid(z))

            # 反向传播
            delta = (activations[-1] - y) * activations[-1] * (1-activations[-1])
            nabla_w = [np.zeros_like(w) for w in self.weights]
            nabla_b = [np.zeros_like(b) for b in self.biases]

            nabla_w[-1] = np.dot(delta, activations[-2].T)
            nabla_b[-1] = delta

            for l in range(2, len(self.weights)+1):
                delta = np.dot(self.weights[-l+1].T, delta) * \
                        activations[-l] * (1-activations[-l])
                nabla_w[-l] = np.dot(delta, activations[-l-1].T)
                nabla_b[-l] = delta

            # 参数更新
            self.weights = [w-current_lr*nw 
                           for w, nw in zip(self.weights, nabla_w)]
            self.biases = [b-current_lr*nb 
                          for b, nb in zip(self.biases, nabla_b)]