神经网络模型实战：从基础分类到三大优化策略（Batch Size、优化器、激活函数）

1次阅读

共计 3404 个字符，预计需要花费 9 分钟才能阅读完成。

对于深度学习初学者来说，训练神经网络时常常会遇到以下问题：

模型准确率始终上不去，在测试集上表现不佳
训练过程收敛缓慢，需要花费大量时间
不知道如何选择合适的超参数，导致模型性能受限

这些问题往往源于对关键超参数的理解不足。本文将重点介绍三个最影响模型性能的超参数：Batch Size、优化器和激活函数，并通过具体实验展示它们的影响。

我们使用 PyTorch 框架实现经典的 LeNet- 5 模型在 MNIST 手写数字识别任务上的应用。以下是完整的代码实现：

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# 数据预处理
transform = transforms.Compose([transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

# 加载 MNIST 数据集
train_dataset = datasets.MNIST('./data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST('./data', train=False, transform=transform)

# 定义 LeNet- 5 模型
class LeNet5(nn.Module):
    def __init__(self):
        super(LeNet5, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5, padding=2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16*5*5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = torch.relu(self.conv1(x))
        x = torch.max_pool2d(x, 2)
        x = torch.relu(self.conv2(x))
        x = torch.max_pool2d(x, 2)
        x = x.view(x.size(0), -1)
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x

# 训练函数
def train(model, device, train_loader, optimizer, criterion, epoch):
    model.train()
    for batch_idx, (data, target) in enumerate(train_loader):
        data, target = data.to(device), target.to(device)
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()

# 测试函数
def test(model, device, test_loader, criterion):
    model.eval()
    test_loss = 0
    correct = 0
    with torch.no_grad():
        for data, target in test_loader:
            data, target = data.to(device), target.to(device)
            output = model(data)
            test_loss += criterion(output, target).item()
            pred = output.argmax(dim=1, keepdim=True)
            correct += pred.eq(target.view_as(pred)).sum().item()

    test_loss /= len(test_loader.dataset)
    accuracy = 100. * correct / len(test_loader.dataset)
    return test_loss, accuracy

# 主程序
def main():
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    # 基础参数
    batch_size = 64
    learning_rate = 0.01
    epochs = 10

    # 初始化模型
    model = LeNet5().to(device)
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=learning_rate)

    # 数据加载
    train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
    test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

    # 训练循环
    for epoch in range(1, epochs + 1):
        train(model, device, train_loader, optimizer, criterion, epoch)
        test_loss, accuracy = test(model, device, test_loader, criterion)
        print(f'Epoch {epoch}: Test Loss: {test_loss:.4f}, Accuracy: {accuracy:.2f}%')

if __name__ == '__main__':
    main()

Batch Size 是指在每次参数更新时使用的样本数量。不同的 Batch Size 会带来不同的效果：