YOLOv9改進策略 | 低照度圖像篇 | 2024最新改進CPA-Enhancer鏈式思考網絡（適用低照度、圖像去霧、雨天、雪天）

一、本文介紹

本文給大家帶來的2024.3月份最新改進機制，由CPA-Enhancer: Chain-of-Thought Prompted Adaptive Enhancer for Object Detection under Unknown Degradations論文提出的CPA-Enhancer鏈式思考網絡，CPA-Enhancer通過引入鏈式思考提示機制，實現了對未知退化條件下圖像的自適應增強。該方法的核心在于能夠利用CoT提示對圖像退化進行動態分析和適應，從而顯著提升物體檢測性能。其適用的場景非常多低照度、圖像去霧、雨天、雪天均有提點效果，本文內容由我獨家整理！

?歡迎大家訂閱我的專欄一起學習YOLO！??

?專欄地址：YOLOv9有效漲點專欄-持續復現各種頂會內容-有效漲點-全網改進最全的專欄?

一、本文介紹

二、原理介紹

三、核心代碼

四、手把手教你添加本文機制?

4.1 修改一

4.2 修改二?

4.3 修改三?

五、CPA-Enhancer的yaml文件和運行記錄

5.1 CPA-Enhancer的yaml文件

5.2 運行記錄

五、本文總結

二、原理介紹

官方論文地址：官方論文地址點擊此處即可跳轉

官方代碼地址：官方代碼地址點擊此處即可跳轉

CPA-Enhancer的創新點和改進機制可以從以下幾個方面進行概括：
1. 鏈式思考（CoT）提示：首次將鏈式思考（CoT）提示機制應用于物體檢測任務中，通過逐步引導的方式處理未知退化圖像的問題。
2. 自適應增強策略：提出了一種能夠根據CoT提示動態調整其增強策略的自適應增強器，無需事先了解圖像的退化類型。
3. 插件式模型設計：CPA-Enhancer設計為一個插件式模塊，可以輕松地與任何現有的通用物體檢測器集成，提升在退化圖像上的檢測性能。

改進機制
CoT提示生成模塊（CGM）：通過CoT提示生成模塊動態生成與圖像退化相關的上下文信息，使模型能夠識別并適應不同類型的圖像退化。
內容驅動提示塊（CPB）：利用內容驅動提示塊加強輸入特征與CoT提示之間的交互，允許模型根據退化的類型調整其增強策略。
端到端訓練：CPA-Enhancer能夠與目標檢測器一起端到端地訓練，無需單獨的預訓練過程或額外的監督信號。

總結
CPA-Enhancer通過引入鏈式思考提示機制，實現了對未知退化條件下圖像的自適應增強。該方法的核心在于能夠利用CoT提示對圖像退化進行動態分析和適應，從而顯著提升物體檢測性能。其插件式設計使其可以無縫集成到現有的檢測框架中，為處理實際應用中遇到的各種退化條件提供了一種有效的解決方案。通過實驗驗證，CPA-Enhancer不僅在物體檢測任務上設立了新的性能標準，還證明了其對其他下游視覺任務性能的提升作用，展示了廣泛的應用潛力。

三、核心代碼

核心代碼的使用方式看章節四！

import torch
import torch.nn as nn
import torch.nn.functional as F
import numbers
from einops import rearrange
from einops.layers.torch import Rearrange__all__ = ['CPA_arch']class RFAConv(nn.Module):  # 基于Group Conv實現的RFAConvdef __init__(self, in_channel, out_channel, kernel_size=3, stride=1):super().__init__()self.kernel_size = kernel_sizeself.get_weight = nn.Sequential(nn.AvgPool2d(kernel_size=kernel_size, padding=kernel_size // 2, stride=stride),nn.Conv2d(in_channel, in_channel * (kernel_size ** 2), kernel_size=1,groups=in_channel, bias=False))self.generate_feature = nn.Sequential(nn.Conv2d(in_channel, in_channel * (kernel_size ** 2), kernel_size=kernel_size, padding=kernel_size // 2,stride=stride, groups=in_channel, bias=False),nn.BatchNorm2d(in_channel * (kernel_size ** 2)),nn.ReLU())self.conv = nn.Sequential(nn.Conv2d(in_channel, out_channel, kernel_size=kernel_size, stride=kernel_size),nn.BatchNorm2d(out_channel),nn.ReLU())def forward(self, x):b, c = x.shape[0:2]weight = self.get_weight(x)h, w = weight.shape[2:]weighted = weight.view(b, c, self.kernel_size ** 2, h, w).softmax(2)  # b c*kernel**2,h,w ->  b c k**2 h wfeature = self.generate_feature(x).view(b, c, self.kernel_size ** 2, h,w)  # b c*kernel**2,h,w ->  b c k**2 h w   獲得感受野空間特征weighted_data = feature * weightedconv_data = rearrange(weighted_data, 'b c (n1 n2) h w -> b c (h n1) (w n2)', n1=self.kernel_size,# b c k**2 h w ->  b c h*k w*kn2=self.kernel_size)return self.conv(conv_data)class Downsample(nn.Module):def __init__(self, n_feat):super(Downsample, self).__init__()self.body = nn.Sequential(nn.Conv2d(n_feat, n_feat // 2, kernel_size=3, stride=1, padding=1, bias=False),nn.PixelUnshuffle(2))def forward(self, x):return self.body(x)class Upsample(nn.Module):def __init__(self, n_feat):super(Upsample, self).__init__()self.body = nn.Sequential(nn.Conv2d(n_feat, n_feat * 2, kernel_size=3, stride=1, padding=1, bias=False),nn.PixelShuffle(2))def forward(self, x):  # (b,c,h,w)return self.body(x)  # (b,c/2,h*2,w*2)class SpatialAttention(nn.Module):def __init__(self):super(SpatialAttention, self).__init__()self.sa = nn.Conv2d(2, 1, 7, padding=3, padding_mode='reflect', bias=True)def forward(self, x):  # x:[b,c,h,w]x_avg = torch.mean(x, dim=1, keepdim=True)  # (b,1,h,w)x_max, _ = torch.max(x, dim=1, keepdim=True)  # (b,1,h,w)x2 = torch.concat([x_avg, x_max], dim=1)  # (b,2,h,w)sattn = self.sa(x2)  # 7x7conv (b,1,h,w)return sattn * xclass ChannelAttention(nn.Module):def __init__(self, dim, reduction=8):super(ChannelAttention, self).__init__()self.gap = nn.AdaptiveAvgPool2d(1)self.ca = nn.Sequential(nn.Conv2d(dim, dim // reduction, 1, padding=0, bias=True),nn.ReLU(inplace=True),  # Relunn.Conv2d(dim // reduction, dim, 1, padding=0, bias=True),)def forward(self, x):  # x:[b,c,h,w]x_gap = self.gap(x)  #  [b,c,1,1]cattn = self.ca(x_gap)  # [b,c,1,1]return cattn * xclass Channel_Shuffle(nn.Module):def __init__(self, num_groups):super(Channel_Shuffle, self).__init__()self.num_groups = num_groupsdef forward(self, x):batch_size, chs, h, w = x.shapechs_per_group = chs // self.num_groupsx = torch.reshape(x, (batch_size, self.num_groups, chs_per_group, h, w))# (batch_size, num_groups, chs_per_group, h, w)x = x.transpose(1, 2)  # dim_1 and dim_2out = torch.reshape(x, (batch_size, -1, h, w))return outclass TransformerBlock(nn.Module):def __init__(self, dim, num_heads, ffn_expansion_factor, bias, LayerNorm_type):super(TransformerBlock, self).__init__()self.norm1 = LayerNorm(dim, LayerNorm_type)self.attn = Attention(dim, num_heads, bias)self.norm2 = LayerNorm(dim, LayerNorm_type)self.ffn = FeedForward(dim, ffn_expansion_factor, bias)def forward(self, x):x = x + self.attn(self.norm1(x))x = x + self.ffn(self.norm2(x))return xdef to_3d(x):return rearrange(x, 'b c h w -> b (h w) c')def to_4d(x, h, w):return rearrange(x, 'b (h w) c -> b c h w', h=h, w=w)class BiasFree_LayerNorm(nn.Module):def __init__(self, normalized_shape):super(BiasFree_LayerNorm, self).__init__()if isinstance(normalized_shape, numbers.Integral):normalized_shape = (normalized_shape,)normalized_shape = torch.Size(normalized_shape)assert len(normalized_shape) == 1self.weight = nn.Parameter(torch.ones(normalized_shape))self.normalized_shape = normalized_shapedef forward(self, x):sigma = x.var(-1, keepdim=True, unbiased=False)return x / torch.sqrt(sigma + 1e-5) * self.weightclass WithBias_LayerNorm(nn.Module):def __init__(self, normalized_shape):super(WithBias_LayerNorm, self).__init__()if isinstance(normalized_shape, numbers.Integral):normalized_shape = (normalized_shape,)normalized_shape = torch.Size(normalized_shape)assert len(normalized_shape) == 1self.weight = nn.Parameter(torch.ones(normalized_shape))self.bias = nn.Parameter(torch.zeros(normalized_shape))self.normalized_shape = normalized_shapedef forward(self, x):device = x.devicemu = x.mean(-1, keepdim=True)sigma = x.var(-1, keepdim=True, unbiased=False)result = (x - mu) / torch.sqrt(sigma + 1e-5) * self.weight.to(device) + self.bias.to(device)return resultclass LayerNorm(nn.Module):def __init__(self, dim, LayerNorm_type):super(LayerNorm, self).__init__()if LayerNorm_type == 'BiasFree':self.body = BiasFree_LayerNorm(dim)else:self.body = WithBias_LayerNorm(dim)def forward(self, x):h, w = x.shape[-2:]return to_4d(self.body(to_3d(x)), h, w)class FeedForward(nn.Module):def __init__(self, dim, ffn_expansion_factor, bias):super(FeedForward, self).__init__()hidden_features = int(dim * ffn_expansion_factor)self.project_in = nn.Conv2d(dim, hidden_features * 2, kernel_size=1, bias=bias)self.dwconv = nn.Conv2d(hidden_features * 2, hidden_features * 2, kernel_size=3, stride=1, padding=1,groups=hidden_features * 2, bias=bias)self.project_out = nn.Conv2d(hidden_features, dim, kernel_size=1, bias=bias)def forward(self, x):device = x.deviceself.project_in = self.project_in.to(device)self.dwconv = self.dwconv.to(device)self.project_out = self.project_out.to(device)x = self.project_in(x)x1, x2 = self.dwconv(x).chunk(2, dim=1)x = F.gelu(x1) * x2x = self.project_out(x)return xclass Attention(nn.Module):def __init__(self, dim, num_heads, bias):super(Attention, self).__init__()self.num_heads = num_headsself.temperature = nn.Parameter(torch.ones(num_heads, 1, 1, dtype=torch.float32), requires_grad=True)self.qkv = nn.Conv2d(dim, dim * 3, kernel_size=1, bias=bias)self.qkv_dwconv = nn.Conv2d(dim * 3, dim * 3, kernel_size=3, stride=1, padding=1, groups=dim * 3,bias=bias)self.project_out = nn.Conv2d(dim, dim, kernel_size=1, bias=bias)def forward(self, x):b, c, h, w = x.shapedevice = x.deviceself.qkv = self.qkv.to(device)self.qkv_dwconv = self.qkv_dwconv.to(device)self.project_out = self.project_out.to(device)qkv = self.qkv(x)qkv = self.qkv_dwconv(qkv)q, k, v = qkv.chunk(3, dim=1)q = rearrange(q, 'b (head c) h w -> b head c (h w)', head=self.num_heads)k = rearrange(k, 'b (head c) h w -> b head c (h w)', head=self.num_heads)v = rearrange(v, 'b (head c) h w -> b head c (h w)', head=self.num_heads)q = torch.nn.functional.normalize(q, dim=-1)k = torch.nn.functional.normalize(k, dim=-1)attn = (q @ k.transpose(-2, -1)) * self.temperature.to(device)attn = attn.softmax(dim=-1)out = (attn @ v)out = rearrange(out, 'b head c (h w) -> b (head c) h w', head=self.num_heads, h=h, w=w)out = self.project_out(out)return outclass resblock(nn.Module):def __init__(self, dim):super(resblock, self).__init__()# self.norm = LayerNorm(dim, LayerNorm_type='BiasFree')self.body = nn.Sequential(nn.Conv2d(dim, dim, kernel_size=3, stride=1, padding=1, bias=False),nn.PReLU(),nn.Conv2d(dim, dim, kernel_size=3, stride=1, padding=1, bias=False))def forward(self, x):res = self.body((x))res += xreturn res#########################################################################
# Chain-of-Thought Prompt Generation Module (CGM)
class CotPromptParaGen(nn.Module):def __init__(self,prompt_inch,prompt_size, num_path=3):super(CotPromptParaGen, self).__init__()# (128,32,32)->(64,64,64)->(32,128,128)self.chain_prompts=nn.ModuleList([nn.ConvTranspose2d(in_channels=prompt_inch if idx==0 else prompt_inch//(2**idx),out_channels=prompt_inch//(2**(idx+1)),kernel_size=3, stride=2, padding=1) for idx in range(num_path)])def forward(self,x):prompt_params = []prompt_params.append(x)for pe in self.chain_prompts:x=pe(x)prompt_params.append(x)return prompt_params#########################################################################
# Content-driven Prompt Block (CPB)
class ContentDrivenPromptBlock(nn.Module):def __init__(self, dim, prompt_dim, reduction=8, num_splits=4):super(ContentDrivenPromptBlock, self).__init__()self.dim = dimself.num_splits = num_splitsself.pa2 = nn.Conv2d(2 * dim, dim, 7, padding=3, padding_mode='reflect', groups=dim, bias=True)self.sigmoid = nn.Sigmoid()self.conv3x3 = nn.Conv2d(prompt_dim, prompt_dim, kernel_size=3, stride=1, padding=1, bias=False)self.conv1x1 = nn.Conv2d(dim, prompt_dim, kernel_size=1, stride=1, bias=False)self.sa = SpatialAttention()self.ca = ChannelAttention(dim, reduction)self.myshuffle = Channel_Shuffle(2)self.out_conv1 = nn.Conv2d(prompt_dim + dim, dim, kernel_size=1, stride=1, bias=False)self.transformer_block = [TransformerBlock(dim=dim // num_splits, num_heads=1, ffn_expansion_factor=2.66, bias=False,LayerNorm_type='WithBias') for _ in range(num_splits)]def forward(self, x, prompt_param):# latent: (b,dim*8,h/8,w/8)  prompt_param3: (1, 256, 16, 16)x_ = xB, C, H, W = x.shapecattn = self.ca(x)  # channel-wise attnsattn = self.sa(x)  # spatial-wise attnpattn1 = sattn + cattnpattn1 = pattn1.unsqueeze(dim=2)  # [b,c,1,h,w]x = x.unsqueeze(dim=2)  # [b,c,1,h,w]x2 = torch.cat([x, pattn1], dim=2)  #  [b,c,2,h,w]x2 = Rearrange('b c t h w -> b (c t) h w')(x2)  # [b,c*2,h,w]x2 = self.myshuffle(x2)  # [c1,c1_att,c2,c2_att,...]pattn2 = self.pa2(x2)pattn2 = self.conv1x1(pattn2)  # [b,prompt_dim,h,w]prompt_weight = self.sigmoid(pattn2)  # Sigmodprompt_param = F.interpolate(prompt_param, (H, W), mode="bilinear")# (b,prompt_dim,prompt_size,prompt_size) -> (b,prompt_dim,h,w)prompt = prompt_weight * prompt_paramprompt = self.conv3x3(prompt)  # (b,prompt_dim,h,w)inter_x = torch.cat([x_, prompt], dim=1)  # (b,prompt_dim+dim,h,w)inter_x = self.out_conv1(inter_x)  # (b,dim,h,w) dim=64splits = torch.split(inter_x, self.dim // self.num_splits, dim=1)transformered_splits = []for i, split in enumerate(splits):transformered_split = self.transformer_block[i](split)transformered_splits.append(transformered_split)result = torch.cat(transformered_splits, dim=1)return result#########################################################################
# CPA_Enhancer
class CPA_arch(nn.Module):def __init__(self, c_in=3, c_out=3, dim=4, prompt_inch=128, prompt_size=32):super(CPA_arch, self).__init__()self.conv0 = RFAConv(c_in, dim)self.conv1 = RFAConv(dim, dim)self.conv2 = RFAConv(dim * 2, dim * 2)self.conv3 = RFAConv(dim * 4, dim * 4)self.conv4 = RFAConv(dim * 8, dim * 8)self.conv5 = RFAConv(dim * 8, dim * 4)self.conv6 = RFAConv(dim * 4, dim * 2)self.conv7 = RFAConv(dim * 2, c_out)self.down1 = Downsample(dim)self.down2 = Downsample(dim * 2)self.down3 = Downsample(dim * 4)self.prompt_param_ini = nn.Parameter(torch.rand(1, prompt_inch, prompt_size, prompt_size)) # (b,c,h,w)self.myPromptParamGen = CotPromptParaGen(prompt_inch=prompt_inch,prompt_size=prompt_size)self.prompt1 = ContentDrivenPromptBlock(dim=dim * 2 ** 1, prompt_dim=prompt_inch // 4, reduction=8)  # !!!!self.prompt2 = ContentDrivenPromptBlock(dim=dim * 2 ** 2, prompt_dim=prompt_inch // 2, reduction=8)self.prompt3 = ContentDrivenPromptBlock(dim=dim * 2 ** 3, prompt_dim=prompt_inch , reduction=8)self.up3 = Upsample(dim * 8)self.up2 = Upsample(dim * 4)self.up1 = Upsample(dim * 2)def forward(self, x):  # (b,c_in,h,w)prompt_params = self.myPromptParamGen(self.prompt_param_ini)prompt_param1 = prompt_params[2] # [1, 64, 64, 64]prompt_param2 = prompt_params[1]  # [1, 128, 32, 32]prompt_param3 = prompt_params[0]  # [1, 256, 16, 16]x0 = self.conv0(x)  # (b,dim,h,w)x1 = self.conv1(x0)  # (b,dim,h,w)x1_down = self.down1(x1)  # (b,dim,h/2,w/2)x2 = self.conv2(x1_down)  # (b,dim,h/2,w/2)x2_down = self.down2(x2)x3 = self.conv3(x2_down)x3_down = self.down3(x3)x4 = self.conv4(x3_down)device = x4.deviceself.prompt1 = self.prompt1.to(device)self.prompt2 = self.prompt2.to(device)self.prompt3 = self.prompt3.to(device)x4_prompt = self.prompt3(x4, prompt_param3)x3_up = self.up3(x4_prompt)x5 = self.conv5(torch.cat([x3_up, x3], 1))x5_prompt = self.prompt2(x5, prompt_param2)x2_up = self.up2(x5_prompt)x2_cat = torch.cat([x2_up, x2], 1)x6 = self.conv6(x2_cat)x6_prompt = self.prompt1(x6, prompt_param1)x1_up = self.up1(x6_prompt)x7 = self.conv7(torch.cat([x1_up, x1], 1))return x7if __name__ == "__main__":# Generating Sample imageimage_size = (1, 3, 640, 640)image = torch.rand(*image_size)out = CPA_arch(3, 3, 4)out = out(image)print(out.size())

四、手把手教你添加本文機制?

4.1 修改一

第一還是建立文件，我們找到如下yolov9-main/models文件夾下建立一個目錄名字呢就是'modules'文件夾(用群內的文件的話已經有了無需新建)！然后在其內部建立一個新的py文件將核心代碼復制粘貼進去即可。

4.2 修改二?

第二步我們在該目錄下創建一個新的py文件名字為'__init__.py'(用群內的文件的話已經有了無需新建)，然后在其內部導入我們的檢測頭如下圖所示。

4.3 修改三?

第三步我門中到如下文件'yolov9-main/models/yolo.py'進行導入和注冊我們的模塊(用群內的文件的話已經有了無需重新導入直接開始第四步即可)！

從今天開始以后的教程就都統一成這個樣子了，因為我默認大家用了我群內的文件來進行修改！！

到此就修改完成了，大家可以復制下面的yaml文件運行，無需修改parse_model方法。。

五、CPA-Enhancer的yaml文件和運行記錄

5.1 CPA-Enhancer的yaml文件

注意本文的代碼計算量很高但是參數量不高，所以大家可能運行失敗因為電腦算力不足而導致！

# YOLOv9# parameters
nc: 80  # number of classes
depth_multiple: 1  # model depth multiple
width_multiple: 1  # layer channel multiple
#activation: nn.LeakyReLU(0.1)
#activation: nn.ReLU()# anchors
anchors: 3# YOLOv9 backbone
backbone:[[-1, 1, CPA_arch, []],[-1, 1, Silence, []],# conv down[-1, 1, Conv, [64, 3, 2]],  # 1-P1/2# conv down[-1, 1, Conv, [128, 3, 2]],  # 2-P2/4# elan-1 block[-1, 1, RepNCSPELAN4, [256, 128, 64, 1]],  # 3# conv down[-1, 1, Conv, [256, 3, 2]],  # 4-P3/8# elan-2 block[-1, 1, RepNCSPELAN4, [512, 256, 128, 1]],  # 5# conv down[-1, 1, Conv, [512, 3, 2]],  # 6-P4/16# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 7# conv down[-1, 1, Conv, [512, 3, 2]],  # 8-P5/32# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 9]# YOLOv9 head
head:[# elan-spp block[-1, 1, SPPELAN, [512, 256]],  # 10# up-concat merge[-1, 1, nn.Upsample, [None, 2, 'nearest']],[[-1, 8], 1, Concat, [1]],  # cat backbone P4# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 13# up-concat merge[-1, 1, nn.Upsample, [None, 2, 'nearest']],[[-1, 6], 1, Concat, [1]],  # cat backbone P3# elan-2 block[-1, 1, RepNCSPELAN4, [256, 256, 128, 1]],  # 16 (P3/8-small)# conv-down merge[-1, 1, Conv, [256, 3, 2]],[[-1, 14], 1, Concat, [1]],  # cat head P4# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 19 (P4/16-medium)# conv-down merge[-1, 1, Conv, [512, 3, 2]],[[-1, 11], 1, Concat, [1]],  # cat head P5# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 22 (P5/32-large)# routing[6, 1, CBLinear, [[256]]], # 23[8, 1, CBLinear, [[256, 512]]], # 24[10, 1, CBLinear, [[256, 512, 512]]], # 25# conv down[0, 1, Conv, [64, 3, 2]],  # 26-P1/2# conv down[-1, 1, Conv, [128, 3, 2]],  # 27-P2/4# elan-1 block[-1, 1, RepNCSPELAN4, [256, 128, 64, 1]],  # 28# conv down fuse[-1, 1, Conv, [256, 3, 2]],  # 29-P3/8[[24, 25, 26, -1], 1, CBFuse, [[0, 0, 0]]], # 30# elan-2 block[-1, 1, RepNCSPELAN4, [512, 256, 128, 1]],  # 31# conv down fuse[-1, 1, Conv, [512, 3, 2]],  # 32-P4/16[[25, 26, -1], 1, CBFuse, [[1, 1]]], # 33# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 34# conv down fuse[-1, 1, Conv, [512, 3, 2]],  # 35-P5/32[[26, -1], 1, CBFuse, [[2]]], # 36# elan-2 block[-1, 1, RepNCSPELAN4, [512, 512, 256, 1]],  # 37# detect[[32, 35, 38, 17, 20, 23], 1, DualDDetect, [nc]],  # DualDDetect(A3, A4, A5, P3, P4, P5)]