以下分析都以A100硬件架構為例;

Theoretical Max Active Warps per SM: 64

Register number: 512 (規定每個thread不能超過256)

1. Theoretical Active Warps per SM [warp]：512//registers_per_thread*4, which defines theoretical active warp occupancy

2. Waves Per SM(equals waves per GPU)：grid_size/a_wave_perf_GPU，which defines tail effect

   1. a_wave_perf_GPU：Theoretical Active Warps per SM // (block_size//32) * 108

   2. A wave of thread blocks is defined as the maximum number of blocks that can be executed in parallel on the target GPU

3. ncu/nsys/torch_profiler 計算threadBlock register file使用情況（threadBlock share mem使用情況也有顯示）：

registers_mem_used = blockSize x registers_per_thread

hopper 白皮書中文版本