BP神经网络基本原理：从数学推导到Python实现

1次阅读

没有评论

共计 2764 个字符，预计需要花费 7 分钟才能阅读完成。

BP 神经网络是深度学习的基础模型，在图像识别、语音处理等领域展现出强大的特征学习能力。它通过多层非线性变换实现复杂函数逼近，而反向传播算法则高效解决了参数优化问题。理解其数学本质是掌握现代深度学习的关键第一步。

神经网络的前向传播本质是矩阵运算与激活函数的复合。以单隐层网络为例：

输入层到隐层计算 ：
$$\mathbf{z}_1 = \mathbf{W}_1 \mathbf{x} + \mathbf{b}_1$$
$$\mathbf{a}_1 = \sigma(\mathbf{z}_1)$$
其中 $\sigma$ 为 sigmoid 函数：$\sigma(z) = \frac{1}{1+e^{-z}}$
隐层到输出层计算 ：
$$\mathbf{z}_2 = \mathbf{W}_2 \mathbf{a}_1 + \mathbf{b}_2$$
$$\hat{\mathbf{y}} = \sigma(\mathbf{z}_2)$$
sigmoid 导数特性 ：
$$\sigma'(z) = \sigma(z)(1-\sigma(z))$$
这一性质将在反向传播中大幅简化计算

反向传播通过误差反向传递更新参数，核心是链式求导：

输出层误差计算 ：
$$\delta_2 = (\hat{\mathbf{y}} – \mathbf{y}) \odot \sigma'(\mathbf{z}_2)$$
隐层误差计算 ：
$$\delta_1 = (\mathbf{W}_2^T \delta_2) \odot \sigma'(\mathbf{z}_1)$$
参数梯度计算 ：
$$\frac{\partial L}{\partial \mathbf{W}_2} = \delta_2 \mathbf{a}_1^T$$
$$\frac{\partial L}{\partial \mathbf{b}_2} = \delta_2$$
$$\frac{\partial L}{\partial \mathbf{W}_1} = \delta_1 \mathbf{x}^T$$
$$\frac{\partial L}{\partial \mathbf{b}_1} = \delta_1$$

import numpy as np

class BPNetwork:
    def __init__(self, input_size, hidden_size):
        # Xavier 初始化
        self.W1 = np.random.randn(hidden_size, input_size) * np.sqrt(1/input_size)
        self.b1 = np.zeros((hidden_size, 1))
        self.W2 = np.random.randn(1, hidden_size) * np.sqrt(1/hidden_size)
        self.b2 = np.zeros((1, 1))

    def sigmoid(self, z):
        return 1 / (1 + np.exp(-z))

    def forward(self, x):
        self.z1 = np.dot(self.W1, x) + self.b1
        self.a1 = self.sigmoid(self.z1)
        self.z2 = np.dot(self.W2, self.a1) + self.b2
        return self.sigmoid(self.z2)

    def train(self, X, y, epochs=1000, lr=0.1, l2_lambda=0.01):
        for epoch in range(epochs):
            # 学习率衰减
            current_lr = lr * (0.95 ** (epoch//100)) 

            # 前向传播
            y_pred = self.forward(X)

            # 反向传播
            delta2 = (y_pred - y) * y_pred * (1 - y_pred)
            dW2 = np.dot(delta2, self.a1.T) + l2_lambda * self.W2
            db2 = np.sum(delta2, axis=1, keepdims=True)

            delta1 = np.dot(self.W2.T, delta2) * self.a1 * (1 - self.a1)
            dW1 = np.dot(delta1, X.T) + l2_lambda * self.W1
            db1 = np.sum(delta1, axis=1, keepdims=True)

            # 参数更新
            self.W2 -= current_lr * dW2
            self.b2 -= current_lr * db2
            self.W1 -= current_lr * dW1
            self.b1 -= current_lr * db1

梯度检查实现 ：

def gradient_check(network, x, y, epsilon=1e-7):
    # 计算解析梯度
    network.forward(x)
    network.backward(x, y)

    # 检查 W1 的梯度
    grad_analytic = network.dW1
    grad_numerical = np.zeros_like(network.W1)

    for i in range(network.W1.shape[0]):
        for j in range(network.W1.shape[1]):
            W1_plus = network.W1.copy()
            W1_plus[i,j] += epsilon
            network.W1 = W1_plus
            loss_plus = network.compute_loss(x, y)

            W1_minus = network.W1.copy()
            W1_minus[i,j] -= epsilon
            network.W1 = W1_minus
            loss_minus = network.compute_loss(x, y)

            grad_numerical[i,j] = (loss_plus - loss_minus) / (2*epsilon)
            network.W1[i,j] = W1_plus[i,j]  # 恢复原值

    # 计算相对误差
    diff = np.linalg.norm(grad_analytic - grad_numerical) / \
          (np.linalg.norm(grad_analytic) + np.linalg.norm(grad_numerical))

    return diff < 1e-7