深入解析bp反向传播算法过程：从数学原理到Python实现

1次阅读

共计 2281 个字符，预计需要花费 6 分钟才能阅读完成。

反向传播 (BP) 算法的核心是通过链式法则计算损失函数对网络参数的梯度。我们先从最简单的单层感知机入手：

给定输入向量 $x$，权重 $w$，偏置 $b$，激活函数 $\sigma$，输出为：
$$y = \sigma(w^Tx + b)$$
定义损失函数 $L$（如均方误差）：
$$L = \frac{1}{2}(y – t)^2$$
其中 $t$ 是真实标签
计算损失对输出的梯度：
$$\frac{\partial L}{\partial y} = y – t$$
计算输出对激活输入的梯度（以 Sigmoid 为例）：
$$\frac{\partial y}{\partial z} = \sigma(z)(1-\sigma(z))$$
$$z = w^Tx + b$$
最终通过链式法则得到权重梯度：
$$\frac{\partial L}{\partial w} = \frac{\partial L}{\partial y} \frac{\partial y}{\partial z} \frac{\partial z}{\partial w} = (y-t) \cdot \sigma'(z) \cdot x$$

对于多层网络，误差需要从输出层逐层反向传播：

前向传播阶段记录所有中间结果（激活值、加权输入等）
输出层计算初始误差 $\delta^L$：
$$\delta^L = \nabla_a L \odot \sigma'(z^L)$$
其中 $\odot$ 表示逐元素乘
反向传播误差到前一层：
$$\delta^{l} = ((w^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l)$$
计算各层参数梯度：
$$\frac{\partial L}{\partial w^l} = \delta^l (a^{l-1})^T$$
$$\frac{\partial L}{\partial b^l} = \delta^l$$

import numpy as np

class NeuralNetwork:
    def __init__(self, layers):
        self.weights = [np.random.randn(y, x) for x, y in zip(layers[:-1], layers[1:])]
        self.biases = [np.random.randn(y, 1) for y in layers[1:]]

    def sigmoid(self, z):
        return 1/(1+np.exp(-z))

    def sigmoid_prime(self, z):
        return self.sigmoid(z)*(1-self.sigmoid(z))

    def forward(self, x):
        """前向传播"""
        for w, b in zip(self.weights, self.biases):
            z = np.dot(w, x) + b
            x = self.sigmoid(z)
        return x

    def train(self, X, y, epochs, lr):
        """训练过程"""
        for _ in range(epochs):
            # 随机选取一个样本
            i = np.random.randint(len(X))
            x, t = X[i], y[i]

            # 前向传播并保存中间结果
            activations = [x]
            zs = []
            for w, b in zip(self.weights, self.biases):
                z = np.dot(w, activations[-1]) + b
                zs.append(z)
                activations.append(self.sigmoid(z))

            # 反向传播
            delta = (activations[-1] - t) * self.sigmoid_prime(zs[-1])
            nabla_w = [np.zeros(w.shape) for w in self.weights]
            nabla_b = [np.zeros(b.shape) for b in self.biases]
            nabla_w[-1] = np.dot(delta, activations[-2].T)
            nabla_b[-1] = delta

            for l in range(2, len(self.weights)+1):
                z = zs[-l]
                delta = np.dot(self.weights[-l+1].T, delta) * self.sigmoid_prime(z)
                nabla_w[-l] = np.dot(delta, activations[-l-1].T)
                nabla_b[-l] = delta

            # 参数更新
            self.weights = [w - lr*nw for w, nw in zip(self.weights, nabla_w)]
            self.biases = [b - lr*nb for b, nb in zip(self.biases, nabla_b)]