【🍊易編橙：一個幫助編程小伙伴少走彎路的終身成長社群🍊】
大家好，我是小森( ﹡?o?﹡ ) ！ 易編橙·終身成長社群創始團隊嘉賓，橙似錦計劃領銜成員、阿里云專家博主、騰訊云內容共創官、CSDN人工智能領域優質創作者 。?
今天我們學習PyTorch的網絡模型創建，全面概括該怎么創建模型！
神經網絡的創建步驟
- 定義模型類，需要繼承
nn.Module - 定義各種層，包括卷積層、池化層、全連接層、激活函數等等
- 編寫前向傳播，規定信號是如何傳輸的
可以用?torchsummary?查看網絡結構，如果沒有的話，使用pip命令進行安裝?
pip install torchsummary
?Module: 神經網絡的模板
所有的神經網絡模塊都應該繼承該模塊
import torch.nn as nn
import torch.nn.functional as F
class Model(nn.Module):def _init__( self):super()._init_()self.conv1 = nn.Conv2d(1，20，5)self.conv2 = nn.Conv2d( 20，20，5)def forward( self， x):x = F.relu( self.conv1(x))return F.relu(self.conv2(x) )神經網絡中常見的各種層
常見的層包括：卷積層，池化層，全連接層，正則化層，激活層
導入層有兩種方法：
一種是將其看作一個類，在
torch.nn里面另一種是將其看作一個函數，在
torch.nn.functional里面可以調用
全連接層?
全連接層又稱為線性層，所以函數名叫?Linear，執行的操作是𝑦=𝑥𝐴𝑇+𝑏
torch.nn.Linear(in_features, out_features, bias=True, device=None, dtype=None)- in_feature代表輸入數
- out_features代表輸出數，即神經元數量
import torch
import torch.nn as nn
m = nn.Linear(2,3)
input = torch.randn(5,2)
output = m( input)
print(output.size() )
輸出：
torch.Size([5, 3])先搭建個只有一層的網絡，用?torchsummry?查看網絡結構
import torch
import torch.nn as nn
from torchsummary import summary class NeuralNetwork(nn.Module): def __init__(self): super().__init__() self.fc = nn.Linear(10, 1, bias=False) def forward(self, x): x = self.fc(x) return x if __name__ == '__main__': network = NeuralNetwork() summary(network, (10,)) # 這里調用 torchsummary 來打印網絡結構?輸出：
----------------------------------------------------------------Layer (type) Output Shape Param #
================================================================Linear-1 [-1, 1] 10
================================================================
Total params: 10
Trainable params: 10
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.00
Params size (MB): 0.00
Estimated Total Size (MB): 0.00
----------------------------------------------------------------Process finished with exit code 0
我們再自定義輸入到網絡中：
if __name__ == '__main__':network = NeuralNetwork()network.to('cuda')input = torch.randn(10)input = input.to('cuda')print('input=', input)output = network(input)print('output=', output)result = output.detach().cpu().numpy()print('result=', result)- ?
detach()用于從計算圖中分離出一個張量（Tensor），使其成為一個新的張量，這個新張量不再需要計算梯度（即不會參與反向傳播）
打印結果：
input= tensor([ 0.5767, -1.2199, 0.4407, 0.6083, -0.1758, 0.2291, -0.8924, 1.1664,0.3445, 0.7242], device='cuda:0')
output= tensor([-0.0183], device='cuda:0', grad_fn=<SqueezeBackward4>)
result= [-0.01834533]激活函數
常見的激活函數包括?sigmoid，relu，以及softmax，學會他們怎么用，就會用其他激活函數了
Sigmoid
sigmoid是早期的激活函數
?函數的示意圖：
m = nn.Sigmoid()
input = torch.randn(5)
output = m(input)print(input)
print(output)
# 輸出tensor([ 0.6759, -0.8753, -0.3187, 0.0088, 2.0625])
tensor([0.6628, 0.2942, 0.4210, 0.5022, 0.8872])
ReLU
ReLU激活函數常放在全連接層、以及卷積層后面
m = nn.ReLU() # 或m = F.ReLU()
input = torch.randn(5)
output = m(input)print(input)
print(output)# 輸出
tensor([-2.8164, 0.8885, -0.9526, 0.3723, -0.2637])
tensor([0.0000, 0.8885, 0.0000, 0.3723, 0.0000])Softmax
softmax是在分類當中經常用到的激活函數，用來放在全連接網絡的最后一層
?
m = nn.Softmax(dim=1)
input = torch.randn(4,3)
output = m(input)print(input)
print(output)# 輸出
tensor([[ 0.1096, 0.7095, 0.5996],[-0.6431, -0.0555, 0.5332],[-0.2367, -0.1851, 0.4029],[-1.0242, 1.9747, 2.0828]])
tensor([[0.2245, 0.4090, 0.3665],[0.1655, 0.2979, 0.5366],[0.2532, 0.2667, 0.4801],[0.0230, 0.4621, 0.5149]])隨機失活Dropout
當?FC層過多，容易對其中某條路徑產生依賴，從而使得某些參數未能訓練起來
為了防止上述問題，在?FC層之間通常還會加入隨機失活功能，也就是Dropout層
dropout的作用是隨機失活的，通常加載FC層之間
m = nn.Dropout(p=0.5)
input = torch.randn(6,6)
output = m(input)print(input)
print(output)輸出：
tensor([[-2.1174, 0.1180, -1.2979, 0.3600, -1.0417, -1.3583],[-0.2945, 1.0038, -0.9205, 2.5044, -1.2789, 0.4402],[-0.4641, 1.3378, 0.1766, 0.1972, 1.6867, -1.7123],[-1.1137, 1.1291, -0.1404, 0.6881, 0.3442, 0.7479],[ 2.4966, -2.5837, 2.0277, -1.0195, 0.2140, -0.1453],[-0.9259, 1.2443, -0.2939, 0.0304, -0.1057, -0.7959]])
tensor([[-4.2347, 0.0000, -0.0000, 0.0000, -0.0000, -2.7165],[-0.5890, 2.0076, -0.0000, 0.0000, -2.5579, 0.0000],[-0.0000, 0.0000, 0.3533, 0.3945, 3.3733, -3.4246],[-0.0000, 2.2581, -0.0000, 1.3763, 0.0000, 0.0000],[ 0.0000, -0.0000, 4.0554, -0.0000, 0.0000, -0.0000],[-0.0000, 2.4887, -0.5878, 0.0608, -0.0000, -0.0000]])至此，一個全連接網絡就可以構建了。
案例1：全連接網絡處理一維信息
搭建以下的網絡結構
組合全連接層，dropout層，激活函數，我們就可以構建出一個完整的全連接網絡結構，代碼如下
import torch
import torch.nn as nn
from torchsummary import summaryclass NeuralNetwork(nn.Module):def __init__(self):super().__init__()self.relu = nn.ReLU()self.softmax = nn.Softmax(dim=1)self.dropout = nn.Dropout(p=0.5)self.fc_1 = nn.Linear(1000, 100)self.fc_2 = nn.Linear(100, 50)self.fc_3 = nn.Linear(50, 10)def forward(self,x):x = x.view(-1, 1000) # 將輸入的維度變成1000x = self.dropout(self.relu(self.fc_1(x)))x = self.dropout(self.relu(self.fc_2(x)))x = self.softmax(self.fc_3(x))return xif __name__ == '__main__':network = NeuralNetwork()network.to('cuda')input = torch.randn(10, 1000)input = input.to('cuda')output = network(input)result = output.detach().cpu().numpy()print('result=', result)summary(network, (1000,))輸出：
result= [[0.08132502 0.0739548 0.09398187 0.10661174 0.12098686 0.115986820.09127808 0.11483455 0.10602687 0.0950134 ][0.09192658 0.08138597 0.07189317 0.12415235 0.11198585 0.116253770.11482875 0.09960157 0.11294526 0.07502676][0.09182167 0.05779037 0.14180492 0.09080649 0.11460604 0.096480750.10017563 0.08380282 0.10664819 0.11606318][0.07540213 0.09515596 0.11200604 0.11029708 0.14663948 0.087270780.06854413 0.07956128 0.10746382 0.1176593 ][0.07536343 0.091349 0.1040979 0.08714981 0.11877389 0.144979750.08420233 0.08688229 0.11904272 0.08815894][0.08312867 0.05986795 0.12148032 0.10792468 0.10400964 0.12383830.11305461 0.08796311 0.11383145 0.08490121][0.07948367 0.09183787 0.08272586 0.11967309 0.12150185 0.108538620.09249827 0.10322765 0.102726 0.09778718][0.09022301 0.09465341 0.08689808 0.08957365 0.14267558 0.10252120.08516254 0.08472932 0.12696771 0.09659547][0.08116906 0.12094414 0.09831021 0.12145476 0.12512349 0.109310410.09090355 0.08238174 0.07898384 0.0914188 ][0.10484971 0.08653011 0.09862521 0.1086348 0.09272213 0.09912340.08527588 0.10124511 0.10974825 0.11324544]]
----------------------------------------------------------------Layer (type) Output Shape Param #
================================================================Linear-1 [-1, 100] 100,100ReLU-2 [-1, 100] 0Dropout-3 [-1, 100] 0Linear-4 [-1, 50] 5,050ReLU-5 [-1, 50] 0Dropout-6 [-1, 50] 0Linear-7 [-1, 10] 510Softmax-8 [-1, 10] 0
================================================================
Total params: 105,660
Trainable params: 105,660
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.00
Params size (MB): 0.40
Estimated Total Size (MB): 0.41
----------------------------------------------------------------
案例2：全連接網絡處理二維圖像
搭建以下的網絡結構
使用全連接網絡處理二維圖像信息，當二維特征（Feature Map）轉為一維特征時，需要從高維壓縮成一維
這時候可以用?tensor.view()，或者用nn.Flatten(start_dim=1)
import torch
import torch.nn as nn
from torchsummary import summaryclass NeuralNetwork(nn.Module):def __init__(self):super().__init__()self.relu = nn.ReLU()self.softmax = nn.Softmax(dim=1)self.dropout = nn.Dropout(p=0.5)self.fc_1 = nn.Linear(3*256*256, 100)self.fc_2 = nn.Linear(100, 10)self.fc_3 = nn.Linear(10,5)def forward(self,x):x = x.view(-1, 3*256*256)x = self.dropout(self.relu(self.fc_1(x)))x = self.dropout(self.relu(self.fc_2(x)))x = self.softmax(self.fc_3(x))return xif __name__ == '__main__':network = NeuralNetwork()network.to('cuda')input = torch.randn((4,3,256,256)) # 4個樣本,每個樣本3通道,256*256像素input = input.to('cuda')output = network(input)result = output.detach().cpu().numpy()print('result=', result)summary(network, (3, 256, 256))輸出：
result= [[0.17621297 0.14625552 0.19215888 0.2527377 0.23263492][0.16786984 0.16124012 0.1907313 0.2352923 0.24486642][0.17400946 0.16431957 0.18192714 0.23585317 0.2438907 ][0.1535219 0.18567167 0.18704179 0.16786435 0.30590025]]
----------------------------------------------------------------Layer (type) Output Shape Param #
================================================================Linear-1 [-1, 100] 19,660,900ReLU-2 [-1, 100] 0Dropout-3 [-1, 100] 0Linear-4 [-1, 10] 1,010ReLU-5 [-1, 10] 0Dropout-6 [-1, 10] 0Linear-7 [-1, 5] 55Softmax-8 [-1, 5] 0
================================================================
Total params: 19,661,965
Trainable params: 19,661,965
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.75
Forward/backward pass size (MB): 0.00
Params size (MB): 75.00
Estimated Total Size (MB): 75.76
----------------------------------------------------------------
