BP神经网络流程图详解：从数学原理到Python实现

1次阅读

没有评论

共计 3079 个字符，预计需要花费 8 分钟才能阅读完成。

BP 神经网络是一种多层前馈神经网络，通过误差反向传播算法进行训练。我们先来看一下神经网络的基本结构：

输入层：接收原始数据
隐藏层：进行特征提取和转换
输出层：产生最终预测结果

前向传播过程可以用以下公式表示：

输入层到第一隐藏层：

$$Z^{[1]} = W^{[1]}X + b^{[1]}$$
$$A^{[1]} = g^{[1]}(Z^{[1]})$$

隐藏层到输出层：

$$Z^{[2]} = W^{[2]}A^{[1]} + b^{[2]}$$
$$A^{[2]} = g^{[2]}(Z^{[2]})$$

其中，$W$ 是权重矩阵，$b$ 是偏置向量，$g$ 是激活函数。

常用的损失函数是均方误差(MSE)：

$$J(W,b) = \frac{1}{2m}\sum_{i=1}^m (y_i – a_i^{[L]})^2$$

flowchart TD
    A[前向传播计算输出] --> B[计算输出层误差]
    B --> C[反向传播误差]
    C --> D[计算各层梯度]
    D --> E[更新权重和偏置]

详细反向传播过程：

输出层误差：

$$\delta^{[L]} = (a^{[L]} – y) \odot g’^{[L]}(z^{[L]})$$

隐藏层误差：

$$\delta^{[l]} = (W^{[l+1]T}\delta^{[l+1]}) \odot g’^{[l]}(z^{[l]})$$

参数梯度：

$$\frac{\partial J}{\partial W^{[l]}} = \frac{1}{m}\delta^{[l]}a^{[l-1]T}$$
$$\frac{\partial J}{\partial b^{[l]}} = \frac{1}{m}\sum\delta^{[l]}$$

import numpy as np

class BPNN:
    def __init__(self, layer_dims, learning_rate=0.01):
        self.params = {}
        self.layer_dims = layer_dims
        self.learning_rate = learning_rate

        # 初始化参数
        for l in range(1, len(layer_dims)):
            self.params['W'+str(l)] = np.random.randn(layer_dims[l], layer_dims[l-1]) * 0.01
            self.params['b'+str(l)] = np.zeros((layer_dims[l], 1))

    def sigmoid(self, Z):
        return 1/(1+np.exp(-Z))

    def sigmoid_derivative(self, Z):
        s = self.sigmoid(Z)
        return s * (1-s)

    def forward_propagation(self, X):
        caches = []
        A = X
        L = len(self.params) // 2

        for l in range(1, L+1):
            A_prev = A
            W = self.params['W'+str(l)]
            b = self.params['b'+str(l)]
            Z = np.dot(W, A_prev) + b
            A = self.sigmoid(Z)
            caches.append((A_prev, W, b, Z))

        return A, caches

    def compute_cost(self, AL, Y):
        m = Y.shape[1]
        cost = (1./m) * (-np.dot(Y, np.log(AL).T) - np.dot(1-Y, np.log(1-AL).T))
        return np.squeeze(cost)

    def backward_propagation(self, AL, Y, caches):
        grads = {}
        L = len(caches)
        m = AL.shape[1]
        Y = Y.reshape(AL.shape)

        # 初始化反向传播
        dAL = - (np.divide(Y, AL) - np.divide(1-Y, 1-AL))

        # 第 L 层梯度
        current_cache = caches[L-1]
        A_prev, W, b, Z = current_cache
        dZ = dAL * self.sigmoid_derivative(Z)
        dW = (1./m) * np.dot(dZ, A_prev.T)
        db = (1./m) * np.sum(dZ, axis=1, keepdims=True)
        grads['dW'+str(L)] = dW
        grads['db'+str(L)] = db

        for l in reversed(range(L-1)):
            current_cache = caches[l]
            A_prev, W, b, Z = current_cache
            dA_prev = np.dot(W.T, dZ)
            dZ = dA_prev * self.sigmoid_derivative(Z)
            dW = (1./m) * np.dot(dZ, A_prev.T)
            db = (1./m) * np.sum(dZ, axis=1, keepdims=True)
            grads['dW'+str(l+1)] = dW
            grads['db'+str(l+1)] = db

        return grads

    def update_parameters(self, grads):
        L = len(self.params) // 2

        for l in range(1, L+1):
            self.params['W'+str(l)] -= self.learning_rate * grads['dW'+str(l)]
            self.params['b'+str(l)] -= self.learning_rate * grads['db'+str(l)]

    def train(self, X, Y, iterations, print_cost=False):
        costs = []

        for i in range(iterations):
            # 前向传播
            AL, caches = self.forward_propagation(X)

            # 计算损失
            cost = self.compute_cost(AL, Y)

            # 反向传播
            grads = self.backward_propagation(AL, Y, caches)

            # 更新参数
            self.update_parameters(grads)

            # 打印损失
            if print_cost and i % 100 == 0:
                print(f"Cost after iteration {i}: {cost}")
                costs.append(cost)

        return costs