多层神经网络传导与梯度下降：从数学原理到实现细节

1次阅读

没有评论

共计 1889 个字符，预计需要花费 5 分钟才能阅读完成。

神经网络的核心计算依赖于矩阵运算。前向传播本质上是权重矩阵与输入数据的连续乘积与非线性变换的叠加。假设网络有 L 层，第 l 层的权重矩阵为 $W^l$，偏置向量为 $b^l$，则前向传播可表示为：

输入层：$a^0 = x$
隐藏层计算：$z^l = W^l a^{l-1} + b^l$
激活函数：$a^l = \sigma(z^l)$

反向传播则是通过链式法则计算梯度的过程。损失函数 $L$ 对第 l 层参数的梯度为：

权重梯度：$\frac{\partial L}{\partial W^l} = \delta^l (a^{l-1})^T$
偏置梯度：$\frac{\partial L}{\partial b^l} = \delta^l$
其中 $\delta^l$ 表示误差项：

$\delta^l = (W^{l+1})^T \delta^{l+1} \odot \sigma'(z^l)$

import numpy as np

class NeuralNetwork:
    def __init__(self, layer_dims):
        """
        初始化网络参数
        layer_dims: 包含各层神经元数量的列表，如 [784, 30, 10]
        """
        self.params = {}
        for l in range(1, len(layer_dims)):
            # Xavier 初始化
            self.params[f'W{l}'] = np.random.randn(layer_dims[l], layer_dims[l-1]) / np.sqrt(layer_dims[l-1])
            self.params[f'b{l}'] = np.zeros((layer_dims[l], 1))

    def forward(self, X):
        """前向传播"""
        cache = {'A0': X}
        L = len(self.params) // 2

        for l in range(1, L+1):
            Z = np.dot(self.params[f'W{l}'], cache[f'A{l-1}']) + self.params[f'b{l}']
            cache[f'Z{l}'] = Z
            cache[f'A{l}'] = self.relu(Z) if l < L else self.softmax(Z)

        return cache

    def backward(self, cache, Y):
        """反向传播"""
        grads = {}
        L = len(self.params) // 2
        m = Y.shape[1]

        # 输出层梯度
        dZ = cache[f'A{L}'] - Y
        grads[f'dW{L}'] = np.dot(dZ, cache[f'A{L-1}'].T) / m
        grads[f'db{L}'] = np.sum(dZ, axis=1, keepdims=True) / m

        # 隐藏层梯度
        for l in reversed(range(1, L)):
            dA = np.dot(self.params[f'W{l+1}'].T, dZ)
            dZ = dA * self.relu_derivative(cache[f'Z{l}'])
            grads[f'dW{l}'] = np.dot(dZ, cache[f'A{l-1}'].T) / m
            grads[f'db{l}'] = np.sum(dZ, axis=1, keepdims=True) / m

        return grads

    def relu(self, x):
        return np.maximum(0, x)

    def relu_derivative(self, x):
        return (x > 0).astype(float)

    def softmax(self, x):
        exps = np.exp(x - np.max(x))
        return exps / np.sum(exps, axis=0)