BP神经网络实战：从链式求导到反向传播的完整实现与优化

1次阅读

共计 1828 个字符，预计需要花费 5 分钟才能阅读完成。

BP 神经网络（Backpropagation Neural Network）是一种多层前馈网络，通过误差反向传播算法训练权重。它由输入层、隐藏层和输出层组成，每层包含若干神经元。神经元之间的连接权重决定了信号的传递强度。

前向传播是指输入信号从输入层经隐藏层到输出层的传递过程。具体步骤如下：

输入层接收输入向量 x
对于每个隐藏层神经元，计算加权输入 z = w*x + b
通过激活函数（如 sigmoid）计算输出 a = σ(z)
重复上述过程直到输出层

数学表达式为：

a^{(l)} = σ(W^{(l)}a^{(l-1)} + b^{(l)})

反向传播是 BP 网络的核心，通过计算损失函数对权重的梯度来更新参数。关键在于链式求导法则的应用。

以均方误差为例：

L = 1/2 * (y_pred - y_true)^2

计算输出层误差 δ^L = ∂L/∂a^L ⊙ σ'(z^L)
反向传播误差：δ^l = (W^{l+1}^T δ^{l+1}) ⊙ σ'(z^l)
计算梯度：∂L/∂W^l = δ^l a^{l-1}^T
更新权重：W^l = W^l – η * ∂L/∂W^l

import numpy as np

class BPNetwork:
    def __init__(self, layers):
        self.weights = [np.random.randn(y, x) for x, y in zip(layers[:-1], layers[1:])]
        self.biases = [np.random.randn(y, 1) for y in layers[1:]]

    def sigmoid(self, z):
        return 1/(1+np.exp(-z))

    def forward(self, x):
        for w, b in zip(self.weights, self.biases):
            x = self.sigmoid(np.dot(w, x) + b)
        return x

    def train(self, X, y, epochs, lr):
        for _ in range(epochs):
            for x, y_true in zip(X, y):
                # Forward pass
                activations = [x]
                zs = []
                for w, b in zip(self.weights, self.biases):
                    z = np.dot(w, activations[-1]) + b
                    zs.append(z)
                    activations.append(self.sigmoid(z))

                # Backward pass
                delta = (activations[-1] - y_true) * activations[-1] * (1 - activations[-1])
                nabla_w = [np.zeros(w.shape) for w in self.weights]
                nabla_b = [np.zeros(b.shape) for b in self.biases]
                nabla_w[-1] = np.dot(delta, activations[-2].T)
                nabla_b[-1] = delta

                for l in range(2, len(self.weights)+1):
                    z = zs[-l]
                    sp = self.sigmoid(z) * (1 - self.sigmoid(z))
                    delta = np.dot(self.weights[-l+1].T, delta) * sp
                    nabla_w[-l] = np.dot(delta, activations[-l-1].T)
                    nabla_b[-l] = delta

                # Update weights
                self.weights = [w - lr*nw for w, nw in zip(self.weights, nabla_w)]
                self.biases = [b - lr*nb for b, nb in zip(self.biases, nabla_b)]

深层网络中，梯度可能在反向传播时指数级减小，导致底层权重更新缓慢。解决方案：