【解讀Spikingjelly】使用單層全連接SNN識別MNIST

原文檔：使用單層全連接SNN識別MNIST — spikingjelly alpha 文檔

代碼地址：完整的代碼位于activation_based.examples.lif_fc_mnist.py

GitHub - fangwei123456/spikingjelly: SpikingJelly is an open-source deep learning framework for Spiking Neural Network (SNN) based on PyTorch.

ZhengyuanGao/spikingjelly: 開源脈沖神經網絡深度學習框架 - spikingjelly - OpenI - 啟智AI開源社區提供普惠算力！ (pcl.ac.cn)a

本文補充一些細節代碼以解決運行報錯問題，并提供可視化代碼，解釋核心代碼作用以輔助SNN初學者快速入門！

1.網絡定義

2.主函數

2.1參數設置

2.2主循環

?3.可視化

?3.1準確率

?3.2測試圖片與發放脈沖

3.3脈沖發放與電壓

4.完整代碼?

lif_fc_mnist.py（為減少運行耗時，迭代次數設置為1）

lif_fc_mnist_test.py

1.網絡定義

class SNN(nn.Module):def __init__(self, tau):super().__init__()self.layer = nn.Sequential(layer.Flatten(),layer.Linear(28 * 28, 10, bias=False),neuron.LIFNode(tau=tau, surrogate_function=surrogate.ATan()),)def forward(self, x: torch.Tensor):return self.layer(x)

（1）super：繼承父類torch.nn.Module的初始化方法

（2）Sequential：順序方式連接網絡結構，首先將輸入展平為一維，定義全連接層，輸入格式28*28，輸出10個神經元。Neuron.LIFNode為脈沖神經元層，用于對全連接層的激活，指定膜時間常數與替代函數（解決不可導問題）

（3）forward：重寫前向傳播函數，返回網絡輸出結果

2.主函數

2.1參數設置

（1）使用命令行設置LIF神經網絡的超參數

parser = argparse.ArgumentParser(description='LIF MNIST Training')parser.add_argument('-T', default=100, type=int, help='simulating time-steps')parser.add_argument('-device', default='cuda:0', help='device')parser.add_argument('-b', default=64, type=int, help='batch size')parser.add_argument('-epochs', default=100, type=int, metavar='N',help='number of total epochs to run')parser.add_argument('-j', default=4, type=int, metavar='N',help='number of data loading workers (default: 4)')
# 添加 default='./MNIST' 以解決無下載所需文件夾問題----------------------------------------parser.add_argument('-data-dir', type=str, default='./MNIST', help='root dir of MNIST dataset')
# -----------------------------------------------------------------------------------------parser.add_argument('-out-dir', type=str, default='./logs', help='root dir for saving logs and checkpoint')parser.add_argument('-resume', type =str, help='resume from the checkpoint path')parser.add_argument('-amp', action='store_true', help='automatic mixed precision training')parser.add_argument('-opt', type=str, choices=['sgd', 'adam'], default='adam', help='use which optimizer. SGD or Adam')parser.add_argument('-momentum', default=0.9, type=float, help='momentum for SGD')parser.add_argument('-lr', default=1e-3, type=float, help='learning rate')parser.add_argument('-tau', default=2.0, type=float, help='parameter tau of LIF neuron')

注：在代碼上述標記位置添加? default='./MNIST' 以解決無下載所需文件夾問題

超參數含義如下圖所示：

（2）參數代入：是否自動混合精度訓練（PyTorch的自動混合精度（AMP） - 知乎 (zhihu.com)）

scaler = Noneif args.amp:scaler = amp.GradScaler()

（3）參數代入：優化器類型

optimizer = Noneif args.opt == 'sgd':optimizer = torch.optim.SGD(net.parameters(), lr=args.lr, momentum=args.momentum)elif args.opt == 'adam':optimizer = torch.optim.Adam(net.parameters(), lr=args.lr)else:raise NotImplementedError(args.opt)

?（4）是否恢復斷點訓練（if args.resume:從斷點處開始繼續訓練模型）

    if args.resume:checkpoint = torch.load(args.resume, map_location='cpu')net.load_state_dict(checkpoint['net'])optimizer.load_state_dict(checkpoint['optimizer'])start_epoch = checkpoint['epoch'] + 1max_test_acc = checkpoint['max_test_acc']

?（5）泊松編碼

encoder = encoding.PoissonEncoder()

2.2主循環

（1）在主循環之前補充創建兩個空數組，用于保存訓練過程中的準確率，以便后續繪制曲線

?（2）加載訓練數據（測試數據代碼大同小異，不另外分析）

        for img, label in train_data_loader:optimizer.zero_grad()img = img.to(args.device)label = label.to(args.device)label_onehot = F.one_hot(label, 10).float()

循環讀取訓練數據，在每次循環前，清空優化器梯度
將img、label放置到GPU上訓練
對標簽進行獨熱編碼，10個類別（獨熱編碼(One-Hot Encoding) - 知乎 (zhihu.com)）

（3）判斷是否使用混合精度訓練

            if scaler is not None:with amp.autocast():out_fr = 0.for t in range(args.T):encoded_img = encoder(img)out_fr += net(encoded_img)out_fr = out_fr / args.Tloss = F.mse_loss(out_fr, label_onehot)scaler.scale(loss).backward()scaler.step(optimizer)scaler.update()else:out_fr = 0.for t in range(args.T):encoded_img = encoder(img)out_fr += net(encoded_img)out_fr = out_fr / args.Tloss = F.mse_loss(out_fr, label_onehot)loss.backward()optimizer.step()

如果使用:
???- 用amp.autocast()包裹前向計算,使其在浮點16位計算
???- 用scaler縮放損失scale(loss)
???- 損失回傳
???- 通過scaler更新優化器

如果不使用混合精度：
???- 正常進行前向計算
???- 損失函數計算
???- 反向傳播
???- 優化器更新

（4）重置網絡

functional.reset_net(net)

SNN中的脈沖神經元在前向傳播時會積累狀態,比如膜電位、釋放的脈沖等。重置可以清空這些狀態,使網絡回到初始狀態。

（5）在下圖位置添加對應代碼保存.npy文件

?3.可視化

?3.1準確率

?在examples文件夾下創建一個.py文件，用于對結果的可視化

?代碼如下：

import numpy as np
import matplotlib.pyplot as plttest_accs = np.load("./train_accs.npy")
x = []
y = []
maxy = -1
maxx = -1
for t in range(len(test_accs)):if test_accs[t] > maxy:maxy = test_accs[t]maxx = tx.append(t)y.append(test_accs[t])
plt.plot(x, y)
# plt.plot(test_accs)
plt.xlabel('Iteration')
plt.ylabel('Acc')
plt.title('Train Acc')
plt.annotate(r'(%d,%f)' % (maxx, maxy), xy=(maxx, maxy), xycoords='data', xytext=(+10, +20), fontsize=16,arrowprops=dict(arrowstyle='->'), textcoords='offset points')
plt.show()
test_accs = np.load("./test_accs.npy")
x = []
y = []
maxy = -1
maxx = -1
for t in range(len(test_accs)):if test_accs[t] > maxy:maxy = test_accs[t]maxx = tx.append(t)y.append(test_accs[t])
# plt.plot(x, y)
plt.plot(test_accs)
plt.xlabel('Epoch')
plt.ylabel('Acc')
plt.title('Test Acc')
plt.annotate(r'(%d,%f)' % (maxx, maxy), xy=(maxx, maxy), xycoords='data', xytext=(+10, +20), fontsize=16,arrowprops=dict(arrowstyle='->'), textcoords='offset points')
plt.show()

效果：

?3.2測試圖片與發放脈沖

?添加如下代碼至main()函數的末尾：

 img = img.cpu().numpy().reshape(28, 28)plt.subplot(221)plt.imshow(img)plt.subplot(222)plt.imshow(img, cmap='gray')plt.subplot(223)plt.imshow(img, cmap=plt.cm.gray)plt.subplot(224)plt.imshow(img, cmap=plt.cm.gray_r)plt.show()

效果：?

3.3脈沖發放與電壓

?新建文件夾，運行如下代碼：

    test_spike = np.load("./s_t_array.npy")test_mem = np.load('./v_t_array.npy')visualizing.plot_2d_heatmap(array=np.asarray(test_mem), title='Membrane Potentials', xlabel='Simulating Step',ylabel='Neuron Index', int_x_ticks=True, x_max=100, dpi=200)visualizing.plot_1d_spikes(spikes=np.asarray(test_spike), title='Membrane Potentials', xlabel='Simulating Step',ylabel='Neuron Index', dpi=200)plt.show()

效果：

4.完整代碼?

lif_fc_mnist.py（為減少運行耗時，迭代次數設置為1）

import os
import time
import argparse
import sys
import datetimeimport torch
import torch.nn as nn
import torch.nn.functional as F
import torch.utils.data as data
from torch.cuda import amp
from torch.utils.tensorboard import SummaryWriter
import torchvision
import numpy as np
import matplotlib.pyplot as pltfrom spikingjelly.activation_based import neuron, encoding, functional, surrogate, layerclass SNN(nn.Module):def __init__(self, tau):super().__init__()self.layer = nn.Sequential(layer.Flatten(),layer.Linear(28 * 28, 10, bias=False),neuron.LIFNode(tau=tau, surrogate_function=surrogate.ATan()),)def forward(self, x: torch.Tensor):return self.layer(x)def main():''':return: None* :ref:`API in English <lif_fc_mnist.main-en>`.. _lif_fc_mnist.main-cn:使用全連接-LIF的網絡結構，進行MNIST識別。\n這個函數會初始化網絡進行訓練，并顯示訓練過程中在測試集的正確率。* :ref:`中文API <lif_fc_mnist.main-cn>`.. _lif_fc_mnist.main-en:The network with FC-LIF structure for classifying MNIST.\nThis function initials the network, starts trainingand shows accuracy on test dataset.'''parser = argparse.ArgumentParser(description='LIF MNIST Training')parser.add_argument('-T', default=100, type=int, help='simulating time-steps')parser.add_argument('-device', default='cuda:0', help='device')parser.add_argument('-b', default=64, type=int, help='batch size')# 100parser.add_argument('-epochs', default=1, type=int, metavar='N',help='number of total epochs to run')parser.add_argument('-j', default=4, type=int, metavar='N',help='number of data loading workers (default: 4)')parser.add_argument('-data-dir', type=str, default='./MNIST', help='root dir of MNIST dataset')parser.add_argument('-out-dir', type=str, default='./logs', help='root dir for saving logs and checkpoint')parser.add_argument('-resume', type =str, help='resume from the checkpoint path')parser.add_argument('-amp', action='store_true', help='automatic mixed precision training')parser.add_argument('-opt', type=str, choices=['sgd', 'adam'], default='adam', help='use which optimizer. SGD or Adam')parser.add_argument('-momentum', default=0.9, type=float, help='momentum for SGD')parser.add_argument('-lr', default=1e-3, type=float, help='learning rate')parser.add_argument('-tau', default=2.0, type=float, help='parameter tau of LIF neuron')args = parser.parse_args()print(args)net = SNN(tau=args.tau)print(net)net.to(args.device)# 初始化數據加載器train_dataset = torchvision.datasets.MNIST(root=args.data_dir,train=True,transform=torchvision.transforms.ToTensor(),download=True)test_dataset = torchvision.datasets.MNIST(root=args.data_dir,train=False,transform=torchvision.transforms.ToTensor(),download=True)train_data_loader = data.DataLoader(dataset=train_dataset,batch_size=args.b,shuffle=True,drop_last=True,num_workers=args.j,pin_memory=True)test_data_loader = data.DataLoader(dataset=test_dataset,batch_size=args.b,shuffle=False,drop_last=False,num_workers=args.j,pin_memory=True)scaler = Noneif args.amp:scaler = amp.GradScaler()start_epoch = 0max_test_acc = -1optimizer = Noneif args.opt == 'sgd':optimizer = torch.optim.SGD(net.parameters(), lr=args.lr, momentum=args.momentum)elif args.opt == 'adam':optimizer = torch.optim.Adam(net.parameters(), lr=args.lr)else:raise NotImplementedError(args.opt)if args.resume:checkpoint = torch.load(args.resume, map_location='cpu')net.load_state_dict(checkpoint['net'])optimizer.load_state_dict(checkpoint['optimizer'])start_epoch = checkpoint['epoch'] + 1max_test_acc = checkpoint['max_test_acc']out_dir = os.path.join(args.out_dir, f'T{args.T}_b{args.b}_{args.opt}_lr{args.lr}')if args.amp:out_dir += '_amp'if not os.path.exists(out_dir):os.makedirs(out_dir)print(f'Mkdir {out_dir}.')with open(os.path.join(out_dir, 'args.txt'), 'w', encoding='utf-8') as args_txt:args_txt.write(str(args))writer = SummaryWriter(out_dir, purge_step=start_epoch)with open(os.path.join(out_dir, 'args.txt'), 'w', encoding='utf-8') as args_txt:args_txt.write(str(args))args_txt.write('\n')args_txt.write(' '.join(sys.argv))encoder = encoding.PoissonEncoder()# 創建保存數組train_accs = []test_accs = []for epoch in range(start_epoch, args.epochs):start_time = time.time()net.train()train_loss = 0train_acc = 0train_samples = 0for img, label in train_data_loader:optimizer.zero_grad()img = img.to(args.device)label = label.to(args.device)label_onehot = F.one_hot(label, 10).float()if scaler is not None:with amp.autocast():out_fr = 0.for t in range(args.T):encoded_img = encoder(img)out_fr += net(encoded_img)out_fr = out_fr / args.Tloss = F.mse_loss(out_fr, label_onehot)scaler.scale(loss).backward()scaler.step(optimizer)scaler.update()else:out_fr = 0.for t in range(args.T):encoded_img = encoder(img)out_fr += net(encoded_img)out_fr = out_fr / args.Tloss = F.mse_loss(out_fr, label_onehot)loss.backward()optimizer.step()train_samples += label.numel()train_loss += loss.item() * label.numel()train_acc += (out_fr.argmax(1) == label).float().sum().item()functional.reset_net(net)train_time = time.time()train_speed = train_samples / (train_time - start_time)train_loss /= train_samplestrain_acc /= train_sampleswriter.add_scalar('train_loss', train_loss, epoch)writer.add_scalar('train_acc', train_acc, epoch)net.eval()test_loss = 0test_acc = 0test_samples = 0with torch.no_grad():for img, label in test_data_loader:img = img.to(args.device)label = label.to(args.device)label_onehot = F.one_hot(label, 10).float()out_fr = 0.for t in range(args.T):encoded_img = encoder(img)out_fr += net(encoded_img)out_fr = out_fr / args.Tloss = F.mse_loss(out_fr, label_onehot)test_samples += label.numel()test_loss += loss.item() * label.numel()test_acc += (out_fr.argmax(1) == label).float().sum().item()functional.reset_net(net)test_time = time.time()test_speed = test_samples / (test_time - train_time)test_loss /= test_samplestest_acc /= test_sampleswriter.add_scalar('test_loss', test_loss, epoch)writer.add_scalar('test_acc', test_acc, epoch)save_max = Falseif test_acc > max_test_acc:max_test_acc = test_accsave_max = Truecheckpoint = {'net': net.state_dict(),'optimizer': optimizer.state_dict(),'epoch': epoch,'max_test_acc': max_test_acc}if save_max:torch.save(checkpoint, os.path.join(out_dir, 'checkpoint_max.pth'))torch.save(checkpoint, os.path.join(out_dir, 'checkpoint_latest.pth'))print(args)print(out_dir)print(f'epoch ={epoch}, train_loss ={train_loss: .4f}, train_acc ={train_acc: .4f}, test_loss ={test_loss: .4f}, test_acc ={test_acc: .4f}, max_test_acc ={max_test_acc: .4f}')print(f'train speed ={train_speed: .4f} images/s, test speed ={test_speed: .4f} images/s')print(f'escape time = {(datetime.datetime.now() + datetime.timedelta(seconds=(time.time() - start_time) * (args.epochs - epoch))).strftime("%Y-%m-%d %H:%M:%S")}\n')#     保存數據至數組train_accs = np.append(train_accs, train_acc)test_accs = np.append(test_accs, test_acc)# print(train_accs)# 寫入npynp.save("./test_accs.npy", test_accs)np.save("./train_accs.npy", train_accs)# 保存繪圖用數據net.eval()# 注冊鉤子output_layer = net.layer[-1] # 輸出層output_layer.v_seq = []output_layer.s_seq = []def save_hook(m, x, y):m.v_seq.append(m.v.unsqueeze(0))m.s_seq.append(y.unsqueeze(0))output_layer.register_forward_hook(save_hook)with torch.no_grad():img, label = test_dataset[0]img = img.to(args.device)out_fr = 0.for t in range(args.T):encoded_img = encoder(img)out_fr += net(encoded_img)out_spikes_counter_frequency = (out_fr / args.T).cpu().numpy()print(f'Firing rate: {out_spikes_counter_frequency}')output_layer.v_seq = torch.cat(output_layer.v_seq)output_layer.s_seq = torch.cat(output_layer.s_seq)v_t_array = output_layer.v_seq.cpu().numpy().squeeze()  # v_t_array[i][j]表示神經元i在j時刻的電壓值np.save("v_t_array.npy",v_t_array)s_t_array = output_layer.s_seq.cpu().numpy().squeeze()  # s_t_array[i][j]表示神經元i在j時刻釋放的脈沖，為0或1np.save("s_t_array.npy",s_t_array)img = img.cpu().numpy().reshape(28, 28)plt.subplot(221)plt.imshow(img)plt.subplot(222)plt.imshow(img, cmap='gray')plt.subplot(223)plt.imshow(img, cmap=plt.cm.gray)plt.subplot(224)plt.imshow(img, cmap=plt.cm.gray_r)plt.show()if __name__ == '__main__':main()

lif_fc_mnist_test.py

import numpy as np
import matplotlib.pyplot as plttest_accs = np.load("./train_accs.npy")
x = []
y = []
maxy = -1
maxx = -1
for t in range(len(test_accs)):if test_accs[t] > maxy:maxy = test_accs[t]maxx = tx.append(t)y.append(test_accs[t])
plt.plot(x, y)
# plt.plot(test_accs)
plt.xlabel('Iteration')
plt.ylabel('Acc')
plt.title('Train Acc')
plt.annotate(r'(%d,%f)' % (maxx, maxy), xy=(maxx, maxy), xycoords='data', xytext=(+10, +20), fontsize=16,arrowprops=dict(arrowstyle='->'), textcoords='offset points')
plt.show()
test_accs = np.load("./test_accs.npy")
x = []
y = []
maxy = -1
maxx = -1
for t in range(len(test_accs)):if test_accs[t] > maxy:maxy = test_accs[t]maxx = tx.append(t)y.append(test_accs[t])
# plt.plot(x, y)
plt.plot(test_accs)
plt.xlabel('Epoch')
plt.ylabel('Acc')
plt.title('Test Acc')
plt.annotate(r'(%d,%f)' % (maxx, maxy), xy=(maxx, maxy), xycoords='data', xytext=(+10, +20), fontsize=16,arrowprops=dict(arrowstyle='->'), textcoords='offset points')
plt.show()