深入解析BP神经网络：反向传播算法的数学推导与实现细节

1次阅读

共计 1720 个字符，预计需要花费 5 分钟才能阅读完成。

BP 神经网络（Backpropagation Neural Network）是多层感知机的典型代表，其核心是通过反向传播算法调整网络参数。我们先回顾正向传播过程：

输入层 ：接收原始特征向量 $X=(x_1,x_2,…,x_n)$
隐藏层计算 ：对第 $l$ 层，$z^l=W^l a^{l-1}+b^l$，$a^l=\sigma(z^l)$，其中 $\sigma$ 为激活函数
输出层 ：最后一层输出 $\hat{y}=a^L$

以 Sigmoid 激活函数为例：

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

采用均方误差损失：
$$
J(W,b) = \frac{1}{2m}\sum_{i=1}^m (\hat{y}^{(i)} – y^{(i)})^2
$$

输出层误差 ：
$$
\delta^L = \frac{\partial J}{\partial z^L} = (\hat{y} – y) \odot \sigma'(z^L)
$$
隐藏层误差传播 ：
$$
\delta^l = ((W^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l)
$$
参数梯度 ：
$$
\frac{\partial J}{\partial W^l} = \delta^l (a^{l-1})^T
$$
$$
\frac{\partial J}{\partial b^l} = \delta^l
$$

import numpy as np

class BPNetwork:
    def __init__(self, layers):
        self.weights = [np.random.randn(y, x)*0.1 
                       for x, y in zip(layers[:-1], layers[1:])]
        self.biases = [np.zeros((y, 1)) for y in layers[1:]]

    def backprop(self, x, y):
        # 前向传播
        activation = x
        activations = [x]
        zs = []

        for w, b in zip(self.weights, self.biases):
            z = np.dot(w, activation) + b
            zs.append(z)
            activation = sigmoid(z)
            activations.append(activation)

        # 反向传播
        delta = (activations[-1] - y) * sigmoid_prime(zs[-1])
        nabla_w = [np.zeros(w.shape) for w in self.weights]
        nabla_b = [np.zeros(b.shape) for b in self.biases]

        nabla_w[-1] = np.dot(delta, activations[-2].T)
        nabla_b[-1] = delta

        for l in range(2, len(self.weights)+1):
            delta = np.dot(self.weights[-l+1].T, delta) * sigmoid_prime(zs[-l])
            nabla_w[-l] = np.dot(delta, activations[-l-1].T)
            nabla_b[-l] = delta

        return nabla_w, nabla_b