BP神经网络公式推导：从数学原理到Python实现

1次阅读

没有评论

共计 2066 个字符，预计需要花费 6 分钟才能阅读完成。

BP 神经网络是深度学习的基础构件，但框架封装往往让开发者陷入 ” 调参师 ” 困境。2019 年 ICLR 会议指出，70% 的模型收敛问题源于对反向传播机制理解不足。本文将用数学推导 + 代码实现的方式撕开这个黑箱。

设网络有 L 层，第 l 层的权重矩阵 $W^l$ 维度为 $(n_{l}, n_{l-1})$，则前向传播可表示为：

$$
z^l = W^l a^{l-1} + b^l \quad (1)
$$
$$
a^l = \sigma(z^l) \quad (2)
$$

其中 $\sigma$ 是 Sigmoid 函数：$\sigma(z) = 1/(1+e^{-z})$

定义损失函数为均方误差：

$$
J = \frac{1}{2}\sum(y – a^L)^2
$$

根据链式法则，输出层梯度：

$$
\delta^L = \frac{\partial J}{\partial z^L} = (a^L – y) \odot \sigma'(z^L) \quad (3)
$$

隐藏层梯度递推公式（核心！）：

$$
\delta^l = ((W^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l) \quad (4)
$$

最终得到权重更新规则：

$$
W^l \leftarrow W^l – \eta \delta^l (a^{l-1})^T \quad (5)
$$
$$
b^l \leftarrow b^l – \eta \delta^l \quad (6)
$$

import numpy as np

class BPNetwork:
    def __init__(self, layers):
        # 初始化权重（He 初始化）self.weights = [np.random.randn(y, x)*np.sqrt(2/x) 
                        for x,y in zip(layers[:-1], layers[1:])]
        self.biases = [np.random.randn(y, 1) for y in layers[1:]]

    def sigmoid(self, z):
        return 1/(1+np.exp(-z))

    def forward(self, x):
        # 向量化前向传播
        a = x.T
        for w, b in zip(self.weights, self.biases):
            z = np.dot(w, a) + b
            a = self.sigmoid(z)
        return a

    def train(self, X, y, epochs=1000, lr=0.1):
        for _ in range(epochs):
            # 随机梯度下降
            i = np.random.randint(len(X))
            x, label = X[i], y[i]

            # 前向传播
            activations = [x.reshape(-1,1)]
            zs = []
            for w,b in zip(self.weights, self.biases):
                z = np.dot(w, activations[-1]) + b
                zs.append(z)
                activations.append(self.sigmoid(z))

            # 反向传播
            delta = (activations[-1] - label.reshape(-1,1)) * activations[-1] * (1-activations[-1])
            nabla_w = [np.zeros_like(w) for w in self.weights]
            nabla_b = [np.zeros_like(b) for b in self.biases]

            nabla_w[-1] = np.dot(delta, activations[-2].T)
            nabla_b[-1] = delta

            for l in range(2, len(self.weights)+1):
                delta = np.dot(self.weights[-l+1].T, delta) * activations[-l] * (1-activations[-l])
                nabla_w[-l] = np.dot(delta, activations[-l-1].T)
                nabla_b[-l] = delta

            # 参数更新（带梯度裁剪）max_norm = 5.0
            for w, dw, b, db in zip(self.weights, nabla_w, self.biases, nabla_b):
                dw_norm = np.linalg.norm(dw)
                if dw_norm > max_norm:
                    dw = dw * max_norm / dw_norm
                w -= lr * dw
                b -= lr * db