机器学习入门：过拟合与欠拟合的核心定义与实战解决方案

1次阅读

共计 2490 个字符，预计需要花费 7 分钟才能阅读完成。

欠拟合（高偏差）：模型在训练集和验证集上均表现不佳，表现为 $J_{train}(\theta) \approx J_{cv}(\theta)$ 且误差值高
过拟合（高方差）：模型在训练集上误差 $J_{train}(\theta)$ 极低，但验证集误差 $J_{cv}(\theta)$ 显著偏高

import matplotlib.pyplot as plt

# 模拟不同复杂度模型的拟合效果
plt.figure(figsize=(12,4))
plt.subplot(121)
plt.title("Underfitting (High Bias)")
plt.scatter(X_train, y_train, s=20)
plt.plot(X_test, simple_model.predict(X_test), c='red')  # 简单线性模型

plt.subplot(122)
plt.title("Overfitting (High Variance)")
plt.scatter(X_train, y_train, s=20)
plt.plot(X_test, complex_model.predict(X_test), c='green')  # 高阶多项式模型
plt.show()

from sklearn.model_selection import learning_curve

def plot_learning_curve(estimator, title, X, y, cv=5):
    train_sizes, train_scores, val_scores = learning_curve(estimator, X, y, cv=cv, scoring='neg_mean_squared_error')

    plt.plot(train_sizes, -train_scores.mean(1), label='Training error')
    plt.plot(train_sizes, -val_scores.mean(1), label='Validation error')
    plt.xlabel("Training examples")
    plt.ylabel("MSE")
    plt.legend()

# 示例调用
plot_learning_curve(LinearRegression(), "Learning Curve", X, y)

特征工程增强

from sklearn.preprocessing import PolynomialFeatures

poly = PolynomialFeatures(degree=3)
X_poly = poly.fit_transform(X_train)  # 原始特征扩展到 3 次多项式

切换复杂模型

from sklearn.ensemble import RandomForestRegressor

rf = RandomForestRegressor(n_estimators=100)
rf.fit(X_train, y_train)

L2 正则化实现

from sklearn.linear_model import Ridge

ridge = Ridge(alpha=1.0)  # alpha 为正则化强度
ridge.fit(X_train, y_train)

PyTorch Dropout 示例

import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(20, 64)
        self.dropout = nn.Dropout(p=0.5)  # 50% 丢弃率
        self.fc2 = nn.Linear(64, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.dropout(x)
        return self.fc2(x)

图像数据：旋转 / 翻转需保持标签有效性
文本数据：同义词替换要避免改变语义
数值数据：添加噪声需控制幅度

from sklearn.linear_model import SGDRegressor
from sklearn.metrics import mean_squared_error

best_loss = float('inf')
patience = 3
counter = 0

for epoch in range(100):
    model.partial_fit(X_batch, y_batch)
    val_loss = mean_squared_error(y_val, model.predict(X_val))

    if val_loss < best_loss:
        best_loss = val_loss
        counter = 0
    else:
        counter += 1
        if counter >= patience:
            break  # 提前停止

在特征工程阶段使用全量数据统计量（如标准化参数）
在模型选择时多次使用测试集验证
数据预处理时未分开处理训练 / 测试集

# 错误示范：随机拆分导致类别分布不均
from sklearn.model_selection import KFold
kf = KFold(n_splits=5, shuffle=True)  # 可能破坏类别平衡

# 正确做法：分层采样
from sklearn.model_selection import StratifiedKFold
skf = StratifiedKFold(n_splits=5)  # 保持各类别比例