反向传播神经网络(bp-net)入门指南：从数学原理到Python实现

1次阅读

共计 4124 个字符，预计需要花费 11 分钟才能阅读完成。

反向传播神经网络（Backpropagation Neural Network, bp-net）是深度学习中最基础的模型之一。它通过模拟人脑神经元的工作方式，实现对复杂函数的逼近。一个典型的 bp-net 由输入层、隐藏层和输出层组成，每层包含若干个神经元。

前向传播是指数据从输入层经过隐藏层最终到达输出层的过程。对于第 $l$ 层的第 $j$ 个神经元，其输出可以表示为：

$$a_j^{(l)} = f\left(\sum_{i} w_{ji}^{(l)}a_i^{(l-1)} + b_j^{(l)}\right)$$

其中：
– $w_{ji}^{(l)}$ 是第 $l-1$ 层第 $i$ 个神经元到第 $l$ 层第 $j$ 个神经元的连接权重
– $b_j^{(l)}$ 是第 $l$ 层第 $j$ 个神经元的偏置
– $f(\cdot)$ 是激活函数

反向传播是通过计算损失函数对权重的梯度，从输出层向输入层逐层更新参数的过程。使用链式法则，我们可以得到权重和偏置的梯度：

$$\frac{\partial L}{\partial w_{ji}^{(l)}} = \delta_j^{(l)}a_i^{(l-1)}$$

$$\frac{\partial L}{\partial b_j^{(l)}} = \delta_j^{(l)}$$

其中 $\delta_j^{(l)}$ 是第 $l$ 层第 $j$ 个神经元的误差项，对于输出层有：

$$\delta_j^{(L)} = \frac{\partial L}{\partial a_j^{(L)}}f'(z_j^{(L)})$$

对于隐藏层：

$$\delta_j^{(l)} = \left(\sum_{k}w_{kj}^{(l+1)}\delta_k^{(l+1)}\right)f'(z_j^{(l)})$$

与 CNN 比较：bp-net 是全连接网络，不擅长处理图像等具有空间结构的数据
与 RNN 比较：bp-net 没有记忆功能，无法处理序列数据
优势：结构简单，易于理解和实现，适合初学者入门
劣势：参数量大，容易过拟合，训练速度慢

下面是使用 NumPy 实现的完整 bp-net 代码：

import numpy as np
from sklearn.datasets import fetch_openml
from sklearn.preprocessing import OneHotEncoder

class BPNN:
    def __init__(self, layer_sizes):
        """
        初始化网络
        :param layer_sizes: 各层神经元数量，如[784, 128, 64, 10]
        """
        self.layer_sizes = layer_sizes
        self.weights = [np.random.randn(y, x)/np.sqrt(x) 
                        for x, y in zip(layer_sizes[:-1], layer_sizes[1:])]
        self.biases = [np.random.randn(y, 1) for y in layer_sizes[1:]]

    def sigmoid(self, z):
        return 1/(1+np.exp(-z))

    def sigmoid_prime(self, z):
        return self.sigmoid(z)*(1-self.sigmoid(z))

    def forward(self, x):
        """前向传播"""
        a = x
        for w, b in zip(self.weights, self.biases):
            z = np.dot(w, a) + b
            a = self.sigmoid(z)
        return a

    def train(self, X_train, y_train, epochs=10, batch_size=32, lr=0.1):
        """训练网络"""
        n = len(X_train)
        for epoch in range(epochs):
            indices = np.random.permutation(n)
            X_shuffled = X_train[indices]
            y_shuffled = y_train[indices]

            for i in range(0, n, batch_size):
                X_batch = X_shuffled[i:i+batch_size]
                y_batch = y_shuffled[i:i+batch_size]

                # 初始化梯度
                grad_w = [np.zeros(w.shape) for w in self.weights]
                grad_b = [np.zeros(b.shape) for b in self.biases]

                # 计算 batch 梯度
                for x, y in zip(X_batch, y_batch):
                    delta_grad_w, delta_grad_b = self.backprop(x, y)
                    grad_w = [gw+dgw for gw, dgw in zip(grad_w, delta_grad_w)]
                    grad_b = [gb+dgb for gb, dgb in zip(grad_b, delta_grad_b)]

                # 更新参数
                self.weights = [w-(lr/batch_size)*gw 
                               for w, gw in zip(self.weights, grad_w)]
                self.biases = [b-(lr/batch_size)*gb 
                              for b, gb in zip(self.biases, grad_b)]

            # 打印训练进度
            accuracy = self.evaluate(X_train, y_train)
            print(f"Epoch {epoch+1}/{epochs}, Accuracy: {accuracy:.2f}")

    def backprop(self, x, y):
        """反向传播"""
        grad_w = [np.zeros(w.shape) for w in self.weights]
        grad_b = [np.zeros(b.shape) for b in self.biases]

        # 前向传播
        activation = x
        activations = [x]
        zs = []

        for w, b in zip(self.weights, self.biases):
            z = np.dot(w, activation) + b
            zs.append(z)
            activation = self.sigmoid(z)
            activations.append(activation)

        # 输出层误差
        delta = (activations[-1] - y) * self.sigmoid_prime(zs[-1])
        grad_b[-1] = delta
        grad_w[-1] = np.dot(delta, activations[-2].T)

        # 隐藏层误差
        for l in range(2, len(self.layer_sizes)):
            z = zs[-l]
            delta = np.dot(self.weights[-l+1].T, delta) * self.sigmoid_prime(z)
            grad_b[-l] = delta
            grad_w[-l] = np.dot(delta, activations[-l-1].T)

        return grad_w, grad_b

    def evaluate(self, X, y):
        """评估准确率"""
        predictions = np.argmax([self.forward(x) for x in X], axis=1)
        labels = np.argmax(y, axis=1)
        return np.mean(predictions == labels)

# 加载 MNIST 数据集
mnist = fetch_openml('mnist_784', version=1)
X = mnist.data.values.astype('float32') / 255.0
y = mnist.target.astype('int64')

# 数据预处理
encoder = OneHotEncoder(sparse=False)
y_onehot = encoder.fit_transform(y.reshape(-1, 1))

# 划分训练测试集
X_train, y_train = X[:60000], y_onehot[:60000]
X_test, y_test = X[60000:], y_onehot[60000:]

# 创建并训练网络
network = BPNN([784, 128, 64, 10])
network.train(X_train, y_train, epochs=20, batch_size=64, lr=0.1)

# 测试准确率
test_accuracy = network.evaluate(X_test, y_test)
print(f"Test accuracy: {test_accuracy:.4f}")