BP神经网络反向传播算法推导过程详解：从数学基础到代码实现

1次阅读

没有评论

共计 2708 个字符，预计需要花费 7 分钟才能阅读完成。

BP 神经网络是一种典型的前馈神经网络，由输入层、隐藏层和输出层组成。每层神经元通过带权重的连接与下一层相连，通过激活函数引入非线性特性。

前向传播过程可描述为：

输入层接收特征向量 $x$
隐藏层第 $j$ 个神经元的净输入：$z_j = \sum_{i}w_{ji}x_i + b_j$
经过激活函数（如 Sigmoid）变换：$a_j = \sigma(z_j)$
输出层计算同理，最终得到预测输出 $\hat{y}$

数学表达式为：
$$\hat{y} = \sigma(W^{(2)}\sigma(W^{(1)}x + b^{(1)}) + b^{(2)})$$

采用均方误差 (MSE) 作为损失函数：
$$E = \frac{1}{2}\sum_{k}(y_k – \hat{y}_k)^2$$

应用链式法则计算梯度：

误差对输出层权重 $w_{kj}$ 的偏导：
$$\frac{\partial E}{\partial w_{kj}} = \frac{\partial E}{\partial \hat{y}k}\frac{\partial \hat{y}_k}{\partial z_k}\frac{\partial z_k}{\partial w$$}
展开计算各项：
$\frac{\partial E}{\partial \hat{y}_k} = -(y_k – \hat{y}_k)$
$\frac{\partial \hat{y}_k}{\partial z_k} = \sigma'(z_k)$
$\frac{\partial z_k}{\partial w_{kj}} = a_j$
合并得到：
$$\delta_k = (y_k – \hat{y}k)\sigma'(z_k)$$
$$\frac{\partial E}{\partial w = \delta_k a_j$$}

误差对隐藏层权重 $w_{ji}$ 的偏导：
$$\frac{\partial E}{\partial w_{ji}} = \sum_{k}\frac{\partial E}{\partial \hat{y}k}\frac{\partial \hat{y}_k}{\partial z_k}\frac{\partial z_k}{\partial a_j}\frac{\partial a_j}{\partial z_j}\frac{\partial z_j}{\partial w$$}
定义隐藏层误差项：
$$\delta_j = \sigma'(z_j)\sum_{k}w_{kj}\delta_k$$
最终梯度表达式：
$$\frac{\partial E}{\partial w_{ji}} = \delta_j x_i$$

import numpy as np

class BPNeuralNetwork:
    def __init__(self, input_size, hidden_size, output_size):
        # 初始化权重
        self.W1 = np.random.randn(input_size, hidden_size)
        self.b1 = np.zeros(hidden_size)
        self.W2 = np.random.randn(hidden_size, output_size)
        self.b2 = np.zeros(output_size)

    def sigmoid(self, x):
        return 1 / (1 + np.exp(-x))

    def sigmoid_derivative(self, x):
        return x * (1 - x)

    def forward(self, X):
        self.z1 = np.dot(X, self.W1) + self.b1
        self.a1 = self.sigmoid(self.z1)
        self.z2 = np.dot(self.a1, self.W2) + self.b2
        self.a2 = self.sigmoid(self.z2)
        return self.a2

    def backward(self, X, y, learning_rate):
        # 输出层误差
        output_error = y - self.a2
        output_delta = output_error * self.sigmoid_derivative(self.a2)

        # 隐藏层误差
        hidden_error = np.dot(output_delta, self.W2.T)
        hidden_delta = hidden_error * self.sigmoid_derivative(self.a1)

        # 更新权重
        self.W2 += learning_rate * np.dot(self.a1.T, output_delta)
        self.b2 += learning_rate * np.sum(output_delta, axis=0)
        self.W1 += learning_rate * np.dot(X.T, hidden_delta)
        self.b1 += learning_rate * np.sum(hidden_delta, axis=0)

    def train(self, X, y, epochs, learning_rate):
        for epoch in range(epochs):
            output = self.forward(X)
            self.backward(X, y, learning_rate)