BP神经网络原理详解与CSDN实战：从数学推导到Python实现

1次阅读

共计 4138 个字符，预计需要花费 11 分钟才能阅读完成。

BP 神经网络（Back Propagation Neural Network）是一种通过误差反向传播算法训练的多层前馈网络。它在分类、回归、模式识别等领域有广泛应用，尤其是在图像识别、自然语言处理等场景中表现出色。相比传统机器学习算法，BP 神经网络的优势在于能够自动学习特征表示，无需人工设计特征，且对非线性问题有较好的拟合能力。

前向传播是 BP 神经网络的基础，其核心思想是通过输入层、隐藏层和输出层的权重和偏置计算最终输出。具体步骤如下：

输入层接收输入数据
隐藏层每个神经元计算加权和，并通过激活函数处理
输出层同样计算加权和并通过激活函数处理

数学表达式如下：

$$ z^l = W^l a^{l-1} + b^l $$
$$ a^l = \sigma(z^l) $$

其中，$z^l$ 表示第 $l$ 层的加权和，$W^l$ 是权重矩阵，$b^l$ 是偏置向量，$a^l$ 是激活值，$\sigma$ 是激活函数。

反向传播是 BP 神经网络训练的核心，通过计算损失函数对参数的梯度来更新权重和偏置。关键步骤包括：

计算输出层误差
反向传播误差到各层
计算各层参数的梯度
更新参数

数学推导如下：

输出层误差：
$$ \delta^L = \nabla_a J \odot \sigma'(z^L) $$

隐藏层误差：
$$ \delta^l = ((W^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l) $$

参数梯度：
$$ \frac{\partial J}{\partial W^l} = \delta^l (a^{l-1})^T $$
$$ \frac{\partial J}{\partial b^l} = \delta^l $$

常用的激活函数包括：

Sigmoid：$\sigma(x) = \frac{1}{1 + e^{-x}}$
Tanh：$\tanh(x) = \frac{e^x – e^{-x}}{e^x + e^{-x}}$
ReLU：$f(x) = max(0, x)$
Leaky ReLU：$f(x) = max(\alpha x, x)$

每种激活函数都有其特点和适用场景，需要根据具体问题选择。

以下是使用 NumPy 实现的 BP 神经网络类：

import numpy as np

class BPNeuralNetwork:
    def __init__(self, layers, activation='sigmoid', learning_rate=0.01):
        self.layers = layers
        self.activation = activation
        self.learning_rate = learning_rate

        # 初始化权重和偏置
        self.weights = []
        self.biases = []
        for i in range(1, len(layers)):
            self.weights.append(np.random.randn(layers[i], layers[i-1]) * 0.1)
            self.biases.append(np.random.randn(layers[i], 1) * 0.1)

    def _sigmoid(self, z):
        return 1 / (1 + np.exp(-z))

    def _sigmoid_prime(self, z):
        s = self._sigmoid(z)
        return s * (1 - s)

    def _relu(self, z):
        return np.maximum(0, z)

    def _relu_prime(self, z):
        return (z > 0).astype(float)

    def _forward_prop(self, x):
        activations = [x]
        zs = []

        for i in range(len(self.weights)):
            z = np.dot(self.weights[i], activations[-1]) + self.biases[i]
            zs.append(z)

            if self.activation == 'sigmoid':
                a = self._sigmoid(z)
            elif self.activation == 'relu':
                a = self._relu(z)

            activations.append(a)

        return activations, zs

    def _back_prop(self, x, y, activations, zs):
        nabla_w = [np.zeros(w.shape) for w in self.weights]
        nabla_b = [np.zeros(b.shape) for b in self.biases]

        # 输出层误差
        delta = (activations[-1] - y)
        if self.activation == 'sigmoid':
            delta *= self._sigmoid_prime(zs[-1])
        elif self.activation == 'relu':
            delta *= self._relu_prime(zs[-1])

        nabla_b[-1] = delta
        nabla_w[-1] = np.dot(delta, activations[-2].T)

        # 隐藏层误差
        for l in range(2, len(self.layers)):
            z = zs[-l]
            if self.activation == 'sigmoid':
                sp = self._sigmoid_prime(z)
            elif self.activation == 'relu':
                sp = self._relu_prime(z)

            delta = np.dot(self.weights[-l+1].T, delta) * sp
            nabla_b[-l] = delta
            nabla_w[-l] = np.dot(delta, activations[-l-1].T)

        return nabla_w, nabla_b

    def train(self, x, y, epochs=1000, batch_size=32):
        n = len(x)

        for epoch in range(epochs):
            # Mini-batch 训练
            indices = np.random.permutation(n)
            for i in range(0, n, batch_size):
                batch_idx = indices[i:i+batch_size]
                x_batch = x[batch_idx]
                y_batch = y[batch_idx]

                nabla_w = [np.zeros(w.shape) for w in self.weights]
                nabla_b = [np.zeros(b.shape) for b in self.biases]

                # 计算梯度
                for xi, yi in zip(x_batch, y_batch):
                    activations, zs = self._forward_prop(xi.reshape(-1, 1))
                    delta_nabla_w, delta_nabla_b = self._back_prop(xi.reshape(-1, 1), yi.reshape(-1, 1), activations, zs)

                    nabla_w = [nw + dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
                    nabla_b = [nb + dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]

                # 更新参数
                self.weights = [w - (self.learning_rate / batch_size) * nw for w, nw in zip(self.weights, nabla_w)]
                self.biases = [b - (self.learning_rate / batch_size) * nb for b, nb in zip(self.biases, nabla_b)]

            # 打印训练进度
            if epoch % 100 == 0:
                loss = self._compute_loss(x, y)
                print(f"Epoch {epoch}, Loss: {loss}")

    def _compute_loss(self, x, y):
        loss = 0
        for xi, yi in zip(x, y):
            activations, _ = self._forward_prop(xi.reshape(-1, 1))
            loss += np.sum((activations[-1] - yi.reshape(-1, 1))**2)
        return loss / len(x)

    def predict(self, x):
        activations, _ = self._forward_prop(x.reshape(-1, 1))
        return activations[-1]

学习率是神经网络训练中最重要的超参数之一。常见的学习率调整策略包括：