BP神经网络算法流程图详解：从数学原理到Python实现

1次阅读

共计 2693 个字符，预计需要花费 7 分钟才能阅读完成。

BP 神经网络（Back Propagation Neural Network）是一种多层前馈神经网络，通过误差反向传播算法训练权重。它由输入层、隐藏层和输出层组成，每层包含多个神经元。

神经元结构 ：每个神经元接收前一层神经元的输出，经过加权求和后通过激活函数产生输出
前向传播 ：数据从输入层流向输出层
反向传播 ：误差从输出层反向传播到输入层，用于调整权重

 输入层 -> 隐藏层 -> 输出层
    |         |         |
    W_ih     W_ho     y_pred

输入数据通过输入层传递
计算隐藏层输入：$z_h = W_{ih}^T x + b_h$
应用激活函数：$a_h = \sigma(z_h)$
计算输出层输入：$z_o = W_{ho}^T a_h + b_o$
应用输出层激活函数：$y_{pred} = \sigma(z_o)$

常用损失函数（以二分类为例）：
$$ L = -\frac{1}{N}\sum_{i=1}^N [y_i\log(y_{pred}) + (1-y_i)\log(1-y_{pred})] $$

 输出层 <- 隐藏层 <- 输入层
    |         |         |
    δ_o      δ_h      W_update

计算输出层误差：$\delta_o = y_{pred} – y$
计算隐藏层误差：$\delta_h = (W_{ho} \delta_o) \odot \sigma'(z_h)$
更新权重：
$$ W_{ho} = W_{ho} – \eta \delta_o a_h^T $$
$$ W_{ih} = W_{ih} – \eta \delta_h x^T $$

import numpy as np
from typing import Tuple

class BPNeuralNetwork:
    def __init__(self, input_size: int, hidden_size: int, output_size: int):
        # 初始化权重
        self.W_ih = np.random.randn(input_size, hidden_size) * 0.01
        self.W_ho = np.random.randn(hidden_size, output_size) * 0.01
        self.b_h = np.zeros((1, hidden_size))
        self.b_o = np.zeros((1, output_size))

    def sigmoid(self, z: np.ndarray) -> np.ndarray:
        return 1 / (1 + np.exp(-z))

    def sigmoid_derivative(self, z: np.ndarray) -> np.ndarray:
        s = self.sigmoid(z)
        return s * (1 - s)

    def forward(self, x: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
        # 前向传播
        self.z_h = np.dot(x, self.W_ih) + self.b_h  # (batch, hidden)
        self.a_h = self.sigmoid(self.z_h)  # (batch, hidden)
        self.z_o = np.dot(self.a_h, self.W_ho) + self.b_o  # (batch, output)
        y_pred = self.sigmoid(self.z_o)  # (batch, output)
        return y_pred, self.z_o, self.a_h, self.z_h

    def backward(self, x: np.ndarray, y: np.ndarray, y_pred: np.ndarray, 
                 lr: float = 0.1) -> None:
        # 反向传播
        batch_size = x.shape[0]

        # 输出层误差
        delta_o = (y_pred - y) * self.sigmoid_derivative(self.z_o)  # (batch, output)

        # 隐藏层误差
        delta_h = np.dot(delta_o, self.W_ho.T) * self.sigmoid_derivative(self.z_h)  # (batch, hidden)

        # 更新权重和偏置
        self.W_ho -= lr * np.dot(self.a_h.T, delta_o) / batch_size  # (hidden, output)
        self.W_ih -= lr * np.dot(x.T, delta_h) / batch_size  # (input, hidden)
        self.b_o -= lr * np.mean(delta_o, axis=0, keepdims=True)
        self.b_h -= lr * np.mean(delta_h, axis=0, keepdims=True)

初始学习率通常设置为 0.01 到 0.1
使用学习率衰减策略：$\eta_t = \frac{\eta_0}{1 + decay\times t}$

梯度裁剪防止梯度爆炸：

max_grad_norm = 5.0
total_norm = np.sqrt(sum(np.sum(np.square(grad)) for grad in grads))
clip_coef = max_grad_norm / (total_norm + 1e-6)
if clip_coef < 1:
    for grad in grads:
        grad *= clip_coef