為啥直接關注riscv-arch-test，是因為RISCOF 測試框架使用的是riscv-arch-test
1. The architectural test
架構測試是一個單一的測試，代表了可編譯和運行的最小測試代碼。它是用匯編代碼編寫的，其產品是test signature。一個架構測試可能由多個測試用例組成。
2. The RISC-V architectural test pool
RISC-V 架構測試池由所有經批準的架構測試組成，這些測試可由測試框架編譯，形成架構測試套件。RISC-V 架構測試庫必須與測試目標無關（因此，應能在任何符合要求的目標上正確運行）。請注意，這種非功能測試不能替代驗證或器件測試。
2.1 Test pool structure
architectural-tests-suite (root)
|-- <architecture>_<mode>/<feature(s)>, where
<architecture> is [ RV32I | RV64I | RV32E ]
<mode> is [ M | MU | MS | MSU ], whereM Machine mode tests - tests execute in M-mode onlyMU Machine/User mode tests - tests execute in both M- & U-modes (S-mode may exist)MS Machine/Supv mode tests - tests execute in both M- & S-modes (not U-mode)MSU All mode tests - tests execute in all of M-, S-, & U-Modes
<feature(s)> are the lettered extension [A | B | C | M ...] or subextension [Zifencei | Zam | ...] when the tests involve extensions, or more general names when tests cut across extension definitionss (e.g. Priv, Interrupt, Vm). The feature string consists of an initial capital letter, followed by any further letters in lower case.
機器模式處理器指令集特性寄存器（MISA）
misa = 0x800000000094112f
二進制表示:1000000000000000000000000000000000000000100101000001000100101111
riscv-arch-test/riscv-test-suite$ ls
env/ Makefile.include README.md rv32e_m/ rv32i_m/ rv64i_m/
riscv-arch-test/riscv-test-suite/rv64i_m$ ls
A/ B/ C/ CMO/ D/ F/ I/ K/ M/ privilege/ P_unratified/ Zfh/ Zfinx/ Zicond/ Zifencei/
3. The RISC-V architectural test suite
RISC-V 架構測試套件是從架構測試池中選出的一組測試，用于測試特定 RISC-V 配置的一致性。測試結果以 test suite signature的形式獲得。測試的選擇基于目標的斷言配置、規范、執行環境或平臺要求。合規的處理器或處理器模型應顯示與所測試的特定配置的黃金參考測試套件簽名相同。
4. The test case
測試用例是架構測試的一部分，只測試規范的一個功能。
注意：一個測試可以包含多個測試用例，每個測試用例都有自己的測試包含條件（由 RVTEST_CASE 宏的 cond_str 參數定義）。
4.1 Test命名
<test objective>-<test number>.S
riscv-arch-test/riscv-test-suite/rv64i_m/I/src$ ls
add-01.S and-01.S bge-01.S bne-01.S lb-align-01.S lhu-align-01.S misalign1-jalr-01.S sd-align-01.S slliw-01.S sltiu-01.S sraiw-01.S srliw-01.S sw-align-01.S
addi-01.S andi-01.S bgeu-01.S fence-01.S lbu-align-01.S lui-01.S or-01.S sh-align-01.S sllw-01.S sltu-01.S sraw-01.S srlw-01.S xor-01.S
addiw-01.S auipc-01.S blt-01.S jal-01.S ld-align-01.S lw-align-01.S ori-01.S sll-01.S slt-01.S sra-01.S srl-01.S sub-01.S xori-01.S
addw-01.S beq-01.S bltu-01.S jalr-01.S lh-align-01.S lwu-align-01.S sb-align-01.S slli-01.S slti-01.S srai-01.S srli-01.S subw-01.S
5. Assembly Test Infrastructure (以./rv32i_m/I/src/add-01.S為例)
//
// This assembly file tests the add instruction of the RISC-V I extension for the add covergroup.
//
#include "model_test.h"
#include "arch_test.h"
RVTEST_ISA("RV32I").section .text.init
.globl rvtest_entry_point
rvtest_entry_point:
RVMODEL_BOOT
RVTEST_CODE_BEGIN#ifdef TEST_CASE_1RVTEST_CASE(0,"//check ISA:=regex(.*32.*);check ISA:=regex(.*I.*);def TEST_CASE_1=True;",add)RVTEST_SIGBASE( x3,signature_x3_1)inst_0:
// rs2 == rd != rs1, rs1==x4, rs2==x24, rd==x24, rs1_val > 0 and rs2_val > 0, rs2_val == 1, rs1_val == (2**(xlen-1)-1), rs1_val != rs2_val, rs1_val == 2147483647
// opcode: add ; op1:x4; op2:x24; dest:x24; op1val:0x7fffffff; op2val:0x1
TEST_RR_OP(add, x24, x4, x24, 0x80000000, 0x7fffffff, 0x1, x3, 0, x18)inst_1:
// rs1 == rs2 != rd, rs1==x10, rs2==x10, rd==x28, rs1_val > 0 and rs2_val < 0, rs2_val == -257, rs1_val == 131072
// opcode: add ; op1:x10; op2:x10; dest:x28; op1val:0x20000; op2val:0x20000
TEST_RR_OP(add, x28, x10, x10, 0x40000, 0x20000, 0x20000, x3, 4, x18)inst_2:
// rs1 == rs2 == rd, rs1==x21, rs2==x21, rd==x21, rs1_val < 0 and rs2_val < 0, rs1_val == -16777217
// opcode: add ; op1:x21; op2:x21; dest:x21; op1val:-0x1000001; op2val:-0x1000001
TEST_RR_OP(add, x21, x21, x21, 0xfdfffffe, -0x1000001, -0x1000001, x3, 8, x18)
......
......
......
RVTEST_CODE_END
RVMODEL_HALTRVTEST_DATA_BEGIN
.align 4rvtest_data:
.word 0xbabecafe
.word 0xbabecafe
.word 0xbabecafe
.word 0xbabecafe
RVTEST_DATA_ENDRVMODEL_DATA_BEGIN
rvtest_sig_begin:
sig_begin_canary:
CANARY;signature_x3_0:.fill 0*(XLEN/32),4,0xdeadbeefsignature_x3_1:.fill 17*(XLEN/32),4,0xdeadbeefsignature_x8_0:.fill 16*(XLEN/32),4,0xdeadbeefsignature_x1_0:.fill 512*(XLEN/32),4,0xdeadbeefsignature_x1_1:.fill 43*(XLEN/32),4,0xdeadbeef#ifdef rvtest_mtrap_routinetsig_begin_canary:
CANARY;
mtrap_sigptr:.fill 64*(XLEN/32),4,0xdeadbeef
tsig_end_canary:
CANARY;#endif#ifdef rvtest_gpr_savegpr_save:.fill 32*(XLEN/32),4,0xdeadbeef#endifsig_end_canary:
CANARY;
rvtest_sig_end:
RVMODEL_DATA_END
簡單對上面的匯編代碼解釋如下：
- Header to inlcude comments
#This assembly file tests the add instruction of the RISC-V I extension for the add covergroup.
- Includes header files
#include “model_test.h”
#include “arch_test.h”
每個測試應只包括以下頭文件：
model_test.h - 定義特定于目標的宏，包括必填的宏和可選的宏：（如 RVMODEL_xxx）
arch_test.h - 定義預定義的測試宏，包括必填的宏和可選的宏：（如 RVTEST_xxx）
- Set the TVM of the test
RVTEST_ISA(“RV32I”)
- Test target specific boot-code
RVMODEL_BOOT
- Start of GPR initialization routine and test code
RVTEST_CODE_BEGIN
- Define the RVTEST_CASE string and conditions
#ifdef TEST_CASE_1
// this test is meant for devices implementing rv32I extension and requires enabling the compile
// macro TEST_CASE_1. This test will contribute to the “add” coverage label.
RVTEST_CASE(0,“//check ISA:=regex(.32.);check ISA:=regex(.I.);def TEST_CASE_1=True;”,add)
- Initialize pointer to the signature region
RVTEST_SIGBASE( x16,signature_x16_1) // x16 will point to signature_x16_1 label in the signature region
- Define the test cases
inst_0:
// rs2 == rd != rs1, rs1==x4, rs2==x24, rd==x24, rs1_val > 0 and rs2_val > 0, rs2_val == 1, rs1_val == (2**(xlen-1)-1), rs1_val != rs2_val, rs1_val == 2147483647
// opcode: add ; op1:x4; op2:x24; dest:x24; op1val:0x7fffffff; op2val:0x1
TEST_RR_OP(add, x24, x4, x24, 0x80000000, 0x7fffffff, 0x1, x3, 0, x18)inst_1:
// rs1 == rs2 != rd, rs1==x10, rs2==x10, rd==x28, rs1_val > 0 and rs2_val < 0, rs2_val == -257, rs1_val == 131072
// opcode: add ; op1:x10; op2:x10; dest:x28; op1val:0x20000; op2val:0x20000
TEST_RR_OP(add, x28, x10, x10, 0x40000, 0x20000, 0x20000, x3, 4, x18)
...
...
//Tests for a instructions with register-register operand
#define TEST_RR_OP(inst, destreg, reg1, reg2, correctval, val1, val2, swreg, offset, testreg) \TEST_CASE(testreg, destreg, correctval, swreg, offset, \LI(reg1, MASK_XLEN(val1)) ;\LI(reg2, MASK_XLEN(val2)) ;\inst destreg, reg1, reg2 ;\)
#define TEST_CASE(testreg, destreg, correctval, swreg, offset, code... ) ;\code ;\RVTEST_SIGUPD(swreg,destreg,offset) ;\RVMODEL_IO_ASSERT_GPR_EQ(testreg, destreg, correctval)/* automatically adjust base and offset if offset gets too big, resetting offset *//* RVTEST_SIGUPD(basereg, sigreg) stores sigreg at offset(basereg) and updates offset by regwidth *//* RVTEST_SIGUPD(basereg, sigreg,newoff) stores sigreg at newoff(basereg) and updates offset to regwidth+newoff */
#define RVTEST_SIGUPD(_BR,_R,...) ;\.if NARG(__VA_ARGS__) == 1 ;\.set offset,_ARG1(__VA_OPT__(__VA_ARGS__,0)) ;\.endif ;\CHK_OFFSET(_BR, REGWIDTH,0) ;\SREG _R,offset(_BR) ;\.set offset,offset+REGWIDTHRVMODEL_IO_ASSERT_GPR_EQ 定義在target的model_test.h中```在咱們得model_test.h中將RVMODEL_IO_ASSERT_GPR_EQ 宏定義如下：比較錯誤的話，往0xF0000080寫1```c
#define RVMODEL_IO_ASSERT_GPR_EQ(_S, _R, _I) \li _S, 0xF0000080; \mv t0, _R; \li t3, _I; \beq t0, t3, 1f; \li t2, 1; \sw t2, 0(_S); \j 2f; \
1: \li t2, 0; \sw t2, 0(_S); \
2: \nop;```在tb.v中加入監測對AXI 寫地址總線，地址0xF0000080的監測，如果出現fail_cnt >0，可以判斷該testcase錯誤```c
// RVMODEL_IO_ASSERT_GPR_EQ(testreg, destreg, correctval) used to check destreg == correctval
// destreg != correctva write testreg 1, else write testreg 0
always @(posedge `CPU_CLK) beginif ((cpu_awaddr[31:0] == 32'hF000_0080) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif(`SOC_TOP.biu_pad_wdata == 1'b1) beginfail_cnt ++;endend
end
- Change signature base register
// this will change the signature base register to x3. x3 will not point to signature_x3_0 in
// the signature region
RVTEST_SIGBASE( x3,signature_x3_0)// continue with new test cases ..
TEST_RR_OP(add, x4, x24, x27, 0x55555955, 0x00000400, 0x55555555, x3, 0, x5)
...
...
- End the test and halt the test-target
RVTEST_CODE_END
RVMODEL_HALT
- Create test input data section
RVTEST_DATA_BEGIN
rvtest_data:
.word 0xbabecafe
RVTEST_DATA_END
- Create pre-loaded signature region
RVMODEL_DATA_BEGIN
rvtest_sig_begin:
sig_begin_canary:
CANARY;signature_x3_0:.fill 0*(XLEN/32),4,0xdeadbeefsignature_x3_1:.fill 17*(XLEN/32),4,0xdeadbeefsignature_x8_0:.fill 16*(XLEN/32),4,0xdeadbeefsignature_x1_0:.fill 512*(XLEN/32),4,0xdeadbeefsignature_x1_1:.fill 43*(XLEN/32),4,0xdeadbeef#ifdef rvtest_mtrap_routinetsig_begin_canary:
CANARY;
mtrap_sigptr:.fill 64*(XLEN/32),4,0xdeadbeef
tsig_end_canary:
CANARY;#endif#ifdef rvtest_gpr_savegpr_save:.fill 32*(XLEN/32),4,0xdeadbeef#endifsig_end_canary:
CANARY;
rvtest_sig_end:
RVMODEL_DATA_END
6. The test case signature
測試用例signature由單個或多個值表示。值將以 RVMODEL_DATA_BEGIN 指定的地址為起點，以 RVMODEL_DATA_END 指定的地址為終點寫入內存。使用 RVTEST_SIGUPD 宏很容易生成signature。
7. The test signature
測試 signature 是由architectural test運行生成的特征值。測試 signature可能由多個測試用例 signature 組成，前綴是一個單獨的行，其中包含測試的名稱和表示其版本的唯一值。測試target負責從內存中提取值并適當地格式化它們，使用由框架提供的元數據，使用 RVMODEL_DATA_BEGIN 和 RVMODEL_DATA_END 宏。測試用例 signature 值按行寫入，從最左邊的最高有效字節開始，格式為 <hex_value>，其中值的長度將為 32 位（因此為 8 個字符），而實際測試計算的值長度不考慮。文件應從 signature 的最低地址處的值開始存儲（即從 RVMODEL_DATA_BEGIN 到 RVMODEL_DATA_END）。此外，signature 應始終從 16 字節（128 位）邊界開始，signature的大小應為 4 字節的倍數（即也應以 4 字節邊界結束）。
8. The test suite signature
測試套件 signature 被定義為一組對于給定的架構測試套件有效的測試 signature。它代表了選擇的特定 RISC-V 配置的架構測試套件的測試 signature。
9. RISCOF 測試框架
RISCOF - RISC-V 兼容性框架是一個基于 Python 的框架，它使得可以使用一套 RISC-V 架構測試集來測試 RISC-V 目標（硬件或軟件實現）與標準的 RISC-V 黃金參考模型的兼容性。
RISC-V Configuration Validator : RISCV-Config
RISC-V Compliance Test Generator : RISC-V CTG
RISC-V ISA Coverage : RISC-V ISAC
為了RISCOF能正常運行測試，需要提供以下內容：
- config.ini：這個文件是一個基本的配置文件，遵循 ini 語法。這個文件將捕獲信息，比如：DUT/reference 插件的名稱，插件的路徑，基于 riscv-config 的 YAML 文件的路徑等。
- dut-plugin 目錄：RISCOF 要求測試的 DUT 模型以 Python 插件的形式提供。Python 插件實際上就是一個包含某些標準和定義函數的 Python 文件，用于執行測試編譯、執行和簽名提取的活動。這個 Python 文件的名稱需要以 riscof_ 為前綴，并且必須存在于 dut-plugin 目錄中。可以參考 Python 插件文件部分，了解如何編寫這個 Python 文件。
該目錄還需要包含基于 riscv-config 的 isa 和 platform YAML 文件，這些文件提供了 DUT 的定義。這些 YAML 文件將用于過濾需要在 DUT 上運行的測試。
最后，在 dut-plugin 目錄中還需要存在一個 env 目錄，其中包含環境文件，如 model_test.h，這是編譯和運行測試所需的文件。請參考 TestFormat 規范，了解可以在 model_test.h 文件中使用的宏的定義。env 目錄還可能包含其他文件，如鏈接腳本、用戶可能需要的后處理腳本。 - reference-plugin 目錄：與 DUT 插件類似，RISCOF 也需要一個參考模型插件。目錄和文件的結構與 DUT 的相同。但是，不需要 isa 和 platform YAML 文件，因為 RISCOF 將始終從 DUT 插件中選擇所有目的的 YAML 文件。
為了簡化操作，RISCOF 通過設置命令為用戶生成標準的 DUT 和參考模型預置模板，如下圖所示：
$ riscof setup --dutname=spike
上述命令將在當前目錄下生成以下文件和目錄：
├──config.ini # configuration file for riscof
├──spike/ # DUT plugin templates
├── env
│ ├── link.ld # DUT linker script
│ └── model_test.h # DUT specific header file
├── riscof_spike.py # DUT python plugin
├── spike_isa.yaml # DUT ISA yaml based on riscv-config
└── spike_platform.yaml # DUT Platform yaml based on riscv-config
├──sail_cSim/ # reference plugin templates
├── env
│ ├── link.ld # Reference linker script
│ └── model_test.h # Reference model specific header file
├── init.py
└── riscof_sail_cSim.py # Reference model python plugin.
將上面的spike改為C920的plugin就可以了，當然需要修改各個配置文件和python文件
c920_isa.yaml
hart_ids: [0]
hart0:ISA: RV64IMAFDCVZicsr_Zicbom_Zicbop_Zicboz_Zihintpause_Zfh_Zca_Zcb_Zcd_Zba_Zbb_Zbc_Zbsphysical_addr_sz: 40User_Spec_Version: "2.2"Privilege_Spec_Version: "1.10"hw_data_misaligned_support: falsepmp_granularity: 4supported_xlen: [64]
c920_platform.yaml
mtime:implemented: trueaddress: 0xBFF8
mtimecmp:implemented: trueaddress: 0x4000
nmi:label: nmi_vector
reset:address: 0x000000000
執行下面的命令，會從test_list.yaml 提取case列表跑各個case，可以注釋掉其中一些，只跑部分case
riscof run --config=config.ini
–suite=riscv-arch-test/riscv-test-suite/
–env=riscv-arch-test/riscv-test-suite/env
–testfile=riscof_work/test_list.yaml
執行下面的命令會生成test_list.yaml 并且跑regression
riscof run --config=config.ini
–suite=riscv-arch-test/riscv-test-suite/
–env=riscv-arch-test/riscv-test-suite/env
10. 測試結果
錯誤原因分析：
第一種：
是因為編譯的時候遇到錯誤
INFO | Compiling test: /ssd_fes/jiongz/desktop/github/c920_riscof1/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/beq-01.SERROR | /opt/picocom/ThirdParty_Libs/T-head/C920_R2S0P21/C920_R2S0_manuals_and_tools/manuals_and_tools/08_toolchain_900_series_cpu_toolchain/V2.8.0/Xuantie-900-gcc-elf-newlib-x86_64-V2.8.0/bin/../lib/gcc/riscv64-unknown-elf/10.4.0/../../../../riscv64-unknown-elf/bin/ld: main.elf section `.text' will not fit in region `MEM1'
collect2: error: ld returned 1 exit status
第二種：
就是c920默認沒使能某個指令
11. rv64i_m/I/src/add-01.S 波形
model_test.h中還有一個宏，用來dump signature和finish simulation
// This will dump the test results (signature) via the testbench dump module.
#define RVMODEL_HALT \signature_dump: \la a0, begin_signature; \la a1, end_signature; \li a2, 0xF0000040; \signature_dump_loop: \bge a0, a1, signature_dump_end; \lw t0, 0(a0); \sw t0, 0(a2); \addi a0, a0, 4; \j signature_dump_loop; \signature_dump_end: \nop; \terminate_simulation: \li a0, 0xF0000000; \li a1, 0xCAFECAFE; \sw a1, 0(a0); \j terminate_simulation
對應tb.v中有
always @(posedge `CPU_CLK or negedge `CPU_RST) beginif (!`CPU_RST) beginmsi <= 1'b0;mei <= 1'b0;mti <= 1'b0;end else begin//if ((wb_cpu.cyc == 1'b1) && (wb_cpu.stb == 1'b1) && (wb_cpu.we == 1'b1) && (cpu_awaddr[31:0] == 32'hF000_0000)) beginif ((cpu_awaddr[31:0] == 32'hF000_0000) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) begincase (`SOC_TOP.biu_pad_wdata[31:0])32'hCAFE_CAFE: begin // end simulation$display("Finishing simulation.");#100;if(fail_cnt >0) begin$error("This case test failed!");end$finish;end......end// Signature Dump
int dump_file; // Declare file handle
always @(posedge `CPU_CLK) beginif ((cpu_awaddr[31:0] == 32'hF000_0040) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif (!dump_file) begin // Check if file is already opendump_file = $fopen("DUT-c920.signature", "w"); // Open file if not already openedend//for (int i = 7; i >= 0; i--) begin// $fwrite(dump_file, "%h\n", `SOC_TOP.biu_pad_wdata[i*4 +: 4]); // Write data//end$fwrite(dump_file, "%h\n", `SOC_TOP.biu_pad_wdata[31:0]); // Write dataendelse if((cpu_awaddr[31:0] == 32'hF000_0000) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif (dump_file) begin // If file is open, close it$fclose(dump_file);dump_file = 0; // Reset file handle to 0 indicating file is closedendend
end// RVMODEL_IO_ASSERT_GPR_EQ(testreg, destreg, correctval) used to check destreg == correctval
// destreg != correctva write testreg 1, else write testreg 0
always @(posedge `CPU_CLK) beginif ((cpu_awaddr[31:0] == 32'hF000_0080) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif(`SOC_TOP.biu_pad_wdata == 1'b1) beginfail_cnt ++;endend
end
#include "model_test.h"
#include "arch_test.h"
RVTEST_ISA("RV64I").section .text.init
.globl rvtest_entry_point
rvtest_entry_point:
RVMODEL_BOOT
RVTEST_CODE_BEGIN#ifdef TEST_CASE_1RVTEST_CASE(0,"//check ISA:=regex(.*64.*);check ISA:=regex(.*I.*);def TEST_CASE_1=True;",add)RVTEST_SIGBASE( x8,signature_x8_1)inst_0:
// rs1 == rs2 != rd, rs1==x0, rs2==x0, rd==x20, rs1_val > 0 and rs2_val > 0, rs1_val == 4, rs1_val==4 and rs2_val==6148914691236517206, rs1_val != rs2_val
// opcode: add ; op1:x0; op2:x0; dest:x20; op1val:0x0; op2val:0x0
TEST_RR_OP(add, x20, x0, x0, 0x0, 0x0, 0x0, x8, 0, x16)inst_1:
// rs2 == rd != rs1, rs1==x2, rs2==x26, rd==x26, rs1_val > 0 and rs2_val < 0, rs2_val == -1073741825
// opcode: add ; op1:x2; op2:x26; dest:x26; op1val:0x5; op2val:-0x40000001
TEST_RR_OP(add, x26, x2, x26, 0xffffffffc0000004, 0x5, -0x40000001, x8, 8, x16)inst_2:
// rs1 == rs2 == rd, rs1==x22, rs2==x22, rd==x22, rs1_val < 0 and rs2_val < 0, rs1_val == -8388609
// opcode: add ; op1:x22; op2:x22; dest:x22; op1val:-0x800001; op2val:-0x800001
TEST_RR_OP(add, x22, x22, x22, 0xfffffffffefffffe, -0x800001, -0x800001, x8, 16, x16)
“riscof_work/rv64i_m/I/src/add-01.S/dut/DUT-c920.signature” 內容如下：
e7d4b281
6f5ca309
00000000
00000000
c0000004
ffffffff
fefffffe
ffffffff
ffffffbf
007fffff
00000080
00000000
66666665
e6666666
00000001
00000000
0001ffff
80000000
10000001
00000000
fffffeff
