一、在 CIFAR10 上對比如下其他的預訓練模型

可以選擇不同的預訓練模型，如 VGG16、Inception V3 等，對比它們在 CIFAR10 數據集上的訓練時間、準確率等指標。以下是使用 VGG16 的示例代碼：

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
from torchvision.models import vgg16# 數據預處理
transform = transforms.Compose([transforms.Resize((224, 224)),  # Inception 和 VGG 要求輸入圖像大小為 224x224transforms.ToTensor(),transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])# 加載 CIFAR10 數據集
trainset = torchvision.datasets.CIFAR10(root='./data', train=True,download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,shuffle=True, num_workers=2)testset = torchvision.datasets.CIFAR10(root='./data', train=False,download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,shuffle=False, num_workers=2)# 加載預訓練的 VGG16 模型
model = vgg16(pretrained=True)
num_ftrs = model.classifier[6].in_features
model.classifier[6] = nn.Linear(num_ftrs, 10)  # 修改最后一層全連接層以適應 CIFAR10 的 10 個類別# 定義損失函數和優化器
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)# 訓練模型
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model.to(device)for epoch in range(2):  # 訓練 2 個 epochrunning_loss = 0.0for i, data in enumerate(trainloader, 0):inputs, labels = data[0].to(device), data[1].to(device)optimizer.zero_grad()outputs = model(inputs)loss = criterion(outputs, labels)loss.backward()optimizer.step()running_loss += loss.item()if i % 2000 == 1999:    # 每 2000 個 mini-batches 打印一次print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')running_loss = 0.0print('Finished Training')

?二、嘗試通過 ctrl 進入 ResNet 的內部，觀察殘差究竟是什么

在 PyTorch 中，如果你使用的是 PyCharm 等 IDE，可以按住 Ctrl 鍵并點擊 resnet18 函數，進入 torchvision.models.resnet 模塊。在該模塊中，可以找到 BasicBlock 類，它實現了 ResNet 的殘差塊。

class BasicBlock(nn.Module):expansion = 1def __init__(self, inplanes, planes, stride=1, downsample=None, groups=1,base_width=64, dilation=1, norm_layer=None):super(BasicBlock, self).__init__()if norm_layer is None:norm_layer = nn.BatchNorm2dif groups != 1 or base_width != 64:raise ValueError('BasicBlock only supports groups=1 and base_width=64')if dilation > 1:raise NotImplementedError("Dilation > 1 not supported in BasicBlock")# Both self.conv1 and self.downsample layers downsample the input when stride != 1self.conv1 = conv3x3(inplanes, planes, stride)self.bn1 = norm_layer(planes)self.relu = nn.ReLU(inplace=True)self.conv2 = conv3x3(planes, planes)self.bn2 = norm_layer(planes)self.downsample = downsampleself.stride = stridedef forward(self, x):identity = xout = self.conv1(x)out = self.bn1(out)out = self.relu(out)out = self.conv2(out)out = self.bn2(out)if self.downsample is not None:identity = self.downsample(x)out += identity  # 這一行實現了殘差連接out = self.relu(out)return out

在 forward 方法中， out += identity 這一行實現了殘差連接。 identity 是輸入的原始特征圖， out 是經過兩層卷積和批量歸一化處理后的特征圖，將它們相加后再通過 ReLU 激活函數，使得模型可以學習到輸入和輸出之間的殘差信息。