openxiangshan / difftest Goto Github PK

Currently I am workign on 3rd ysyx project with Chisel. I skip instruction commit for MMIO accessing. When we update the value of mtimecmp in CLINT, mip will also change, e.g., when we increment mtimecmp in interrupt handler, mtip field in mip register will change from 1 to 0, but difftest doesn't pass this info to NEMU.

My temporary solution (although it is not elegant) is to copy the value of the whole mip register from dut to ref when there's a "skip" instruction in src/test/csrc/difftest/difftest.cpp:217.

  // MMIO accessing should not be a branch or jump, just +2/+4 to get the next pc
  // to skip the checking of an instruction, just copy the reg state to reference design
  if (dut.commit[i].skip) {
    proxy->regcpy(ref_regs_ptr, REF_TO_DIFFTEST);
    ref.csr.this_pc += dut.commit[i].isRVC ? 2 : 4;
    if (dut.commit[i].wen && dut.commit[i].wdest != 0) {
      // TODO: FPR
      ref_regs_ptr[dut.commit[i].wdest] = dut.commit[i].wdata;
    }
    ref.csr.mip = dut.csr.mip;   // THIS IS WHAT I ADDED
    proxy->regcpy(ref_regs_ptr, DIFFTEST_TO_REF);
    return;
  }

When using Spike as the REF and conducting difftest with the rocket-chip, I noticed that REF records an incorrect value for mepc during exceptions.

The Commit Instr Trace is as follows：

- [468] commit pc 00000000800006ec inst 6141606f wen 1 dst 00 data 0000000000000000 idx 000 j       pc + 0x16614
- [469] commit pc 0000000080016d00 inst 00000013 wen 1 dst 00 data 0000000000000000 idx 000 nop
- [470] exception pc 0000000080016d04 inst 00000000 cause 0000000000000002 c.unimp//EXCEPTION
- [471] commit pc 0000000080002000 inst 0400006f wen 1 dst 00 data 0000000000000000 idx 000 j       pc + 0x40
- ...
- [502] commit pc 00000000800020b8 inst 0dbc3c23 wen 0 dst 24 data 0000000080001fe0 idx 000 sd      s11, 216(s8)
- [503] commit pc 00000000800020bc inst 0fcc3023 wen 0 dst 00 data 0000000000000000 idx 000 sd      t3, 224(s8) <--
- [504] commit pc 00000000800020d0 inst ffffe297 wen 1 dst 05 data 00000000800000d0 idx 000 auipc   t0, 0xffffe
- ...
- [508] commit pc 0000000080016d00 inst 00000013 wen 1 dst 00 data 0000000000000000 idx 000 nop

Difftest result：

- pc: 0x0000000080002000 mstatus: 0x0000000a00001800 mcause: 0x0000000000000002 mepc: 0x00000000800020bc
-                        sstatus: 0x0000000200000000 scause: 0xaa75e257df5869e5 sepc: 0x0000006f10958978
- satp: 0x0000000000000000
- mip: 0x0000000000000220 mie: 0x0000000000000000 mscratch: 0x6410e61c5e18874a sscratch: 0x6410e61c5e18874a
- mideleg: 0x0000000000000000 medeleg: 0x0000000000000000
- mtval: 0x0000000000000000 stval: 0xfffffff9ba1e5c49 mtvec: 0x0000000080002001 stvec: 0x00000000314fc1c8
- privilege mode:3
- priviledgeMode: 3
-    mepc different at pc = 0x00800020bc, right= 0x00000000800020bc, wrong = 0x0000000080016d04
- Core 0: ABORT at pc = 0x800020c0

The above log found that ref gave the wrong mepc. The correct value should be: 0x0000000080016d04

To verify whether it's an error within Spike itself, I ran it independently in Spike, and the debug log is as follows:

- core   0: 0x00000000800006ec (0x6141606f) j       pc + 0x16614
- (spike) 
- core   0: >>>>  _real_end
- core   0: 0x0000000080016d00 (0x00000013) nop
- (spike) 
- core   0: >>>>  _end
- core   0: 0x0000000080016d04 (0x00000000) c.unimp
- core   0: exception trap_illegal_instruction, epc 0x0000000080016d04
- core   0:           tval 0x0000000000000000
- (spike) 
- core   0: >>>>  mtvec_handler
- core   0: 0x0000000080002000 (0x0400006f) j       pc + 0x40
- (spike) reg 0 mepc
- 0x0000000080016d04

I found that Spike itself is correct, so the issue might be with the synchronization mechanism of difftest when it encounters exceptions.

Img and log files:
test.zip

你好，目前看到，Difftest适配了nutshell、rocketchip和xiangshan三个设计工程，想问一下，Difftest从nutshell迁移到rocketchip上，都需要做哪几个方面的修改？或者说，基于一个新的开源核，如果想使用Difftest的话，需要修改哪些内容？

In the current version of difftest, even though some internal signals are directly connected to AXI4 IO signals directly through wire (no registers involved), in the waveform, there's 1 clock cycle delay for these internal signals.

The solution is simple, and works in my OSCPU project. Exchange the order of following two code blocks in difftest/src/test/csrc/verilator/emu.cpp, around line 220~240.

#ifdef WITH_DRAMSIM3
  axi_copy_from_dut_ptr(dut_ptr, axi);
  axi.aw.addr -= 0x80000000UL;
  axi.ar.addr -= 0x80000000UL;
  dramsim3_helper_falling(axi);
  axi_set_dut_ptr(dut_ptr, axi);
#endif

#if VM_TRACE == 1
  if (enable_waveform) {
    auto trap = difftest[0]->get_trap_event();
    uint64_t cycle = trap->cycleCnt;
    uint64_t begin = dut_ptr->io_logCtrl_log_begin;
    uint64_t end   = dut_ptr->io_logCtrl_log_end;
    bool in_range  = (begin <= cycle) && (cycle <= end);
    if (in_range) { tfp->dump(cycle); }
  }
#endif

I'm not sure whether it is appropriate to directly submit a pull request, so I raise an issue here :)
(Thank TA Zhibo Hong for helping me on debugging this issue in the OSCPU project.)

AbstractRefProxy 的函数声明在这里，其中以ref_regcpy为例，从宏定义可知，函数原型包含3个参数
f(ref_regcpy, difftest_regcpy, void, void*, bool, bool)

但是在NEMU中的 void difftest_regcpy(void *dut, bool direction, bool restore, uint64_t restore_count) 原型是有4个参数？

这里是不是有点问题，还请帮忙澄清一下

README.md的“Difftest: detailed usage (Chinese)”这个地址无效

Chisel has experimental features of Definition/Instance to avoid multiple elaboration of the same hardware. However, this brings issues to the elaboration of Difftest, which relies on the generational information during Chisel elaboration. See OpenXiangShan/XiangShan#2338

Don't know how to fix it yet.

$:~/riscv/NutShell$ ./build/emu -b 0 -e 0 -i ./ready-to-run/linux.bin --diff ./ready-to-run/riscv64-nemu-interpreter-so

I pulled NutShell and difftest rep to my local envierment, NutShell switched to difftest branch. After running "make emu" in difftest dirctory, i run the command like above. Terminal promopts :

Emu compiled at Jul 23 2021, 16:27:44
The image is ./ready-to-run/linux.bin
Using simulated 8192MB RAM
[warning] sdcard img not found
Using ./ready-to-run/riscv64-nemu-interpreter-so for difftest
emu: /home/xxxxx/riscv/NutShell/difftest/src/test/csrc/difftest/nemuproxy.cpp:48: NemuProxy::NemuProxy(int): Assertion `memcpy_from_ref' failed.
Aborted (core dumped)

// if (proxy->store_commit(&addr, &data, &mask)) {
if (proxy->store_commit(&addr, &data, &mask, &nemu_addr, &nemu_data, &nemu_mask)) { // jiasen. 12/8
  eprintf("HJS Difftest::do_store_check proxy->store_commit i = %d success\n", i); // jiasen. 12/7

  // display();
  // printf("Mismatch for store commits %d: \n", i);
  // printf("  REF commits addr 0x%lx, data 0x%lx, mask 0x%x\n", addr, data, mask);
  // printf("  DUT commits addr 0x%lx, data 0x%lx, mask 0x%x\n",
  //   dut.store[i].addr, dut.store[i].data, dut.store[i].mask);
  return 1;
}
else { // jiasen. 12/7
  eprintf("HJS Difftest::do_store_check proxy->store_commit i = %d failed\n", i); // jiasen. 12/7
  display();
  eprintf("Mismatch for store commits %d: \n", i);
  eprintf("  DUT commits addr 0x%lx, data 0x%lx, mask 0x%x\n", addr, data, mask);
  eprintf("  REF(NEMU) commits addr 0x%lx, data 0x%lx, mask 0x%x\n", nemu_addr, nemu_data, nemu_mask);
}

修改如上所示
为了方便调试，同步修改NEMU中的difftest_store_commit以及check_store_commit，令check_store_commit的端口新增NEMU的addr, data, mask信息，以便直接在difftest中监测

你好，想问下咱们difftest环境是否有尝试接入特定VIP来辅助验证，从环境结构看，能否通过DPI来与VIP通信并启动VIP来进行验证？

我在使用rocket chip的 dev-difftest分支遇到如下error
（1） 准备工程：
https://github.com/OpenXiangShan/rocket-chip/tree/dev-difftest?tab=readme-ov-file
（2） 配置riscv-gcc
（3） 工程配置信息：
cp1@cp1:~/rocket-chip$ git branch

• dev-difftest

cp1@cp1:~/rocket-chip$ git log -1
commit 82c0f3e03152b1851c6b505d98237d0b2dc05883 (HEAD -> dev-difftest, origin/dev-difftest)
Author: Yinan Xu [email protected]
Date: Sun Apr 21 21:04:22 2024 +0800

Bump Chisel 6.3.0 and DiffTest

（4）生成verilog报错：
make verilog
cd /home/cp1/rocket-chip && mill -i emulator[freechips.rocketchip.system.TestHarness,freechips.rocketchip.system.DefaultConfig].mfccompiler.compile
[206/206] emulator[freechips.rocketchip.system.TestHarness,freechips.rocketchip.system.DefaultConfig].mfccompiler.compile
1 targets failed
emulator[freechips.rocketchip.system.TestHarness,freechips.rocketchip.system.DefaultConfig].mfccompiler.compile java.io.IOException: Cannot run program "firtool" (in directory "/home/cp1/rocket-chip/out/emulator/freechips.rocketchip.system.TestHarness/freechips.rocketchip.system.DefaultConfig/mfccompiler/compile.dest"): error=2, 没有那个文件或目录
java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
os.proc.proc$lzycompute$1(ProcessOps.scala:123)
os.proc.proc$1(ProcessOps.scala:122)
os.proc.spawn(ProcessOps.scala:129)
os.proc.call(ProcessOps.scala:87)
millbuild.build$Emulator$mfccompiler$.$anonfun$compile$3(build.sc:225)
java.io.IOException: error=2, 没有那个文件或目录
java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
java.base/java.lang.ProcessImpl.(ProcessImpl.java:340)
java.base/java.lang.ProcessImpl.start(ProcessImpl.java:271)
java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
os.proc.proc$lzycompute$1(ProcessOps.scala:123)
os.proc.proc$1(ProcessOps.scala:122)
os.proc.spawn(ProcessOps.scala:129)
os.proc.call(ProcessOps.scala:87)
millbuild.build$Emulator$mfccompiler$.$anonfun$compile$3(build.sc:225)
make: *** [Makefile:73：verilog] 错误 1

这里没看找到 compile.dest的生成条件
麻烦帮忙确认下rocket-chip 的dev-difftest 分支目前是否正常可用！

有这方面的文档吗？

• Difftest with Forkwait
• Difftest with DRAMSim3
• Difftest in oscpu-framework

我使用了Difftest作为riscv处理器的代码比对工具，修改了difftest/config/verilator.mk中关于波形的配置，设置EMU_TRACE为1，并且在执行时添加了--dump-wave参数。执行参数如下：
./build/emu -b 0 -e 10000 -i ./ready-to-run/rtthread.bin --dump-wave
在执行过程中，控制台显示
dump wave to /home/xx/rvcpu/1/rvcpu/build/2022-09-27@04:21:31.vcd...
在这个目录下，我找到了这个波形文件，使用gtkwave打开后，所有的信号名均能正常显示，但是波形图显示只有一个时间点而不是完整的执行周期，并且为difftest执行最后一刻的值。我想咨询一下这是否因为我的使用方法有问题，正确的显示波形应该如何设置。

dut.csr.this_pc = a3703baed8a56
dut.csr.mstatus = 40e00333e4728626
dut.csr.mcause = 88e6e06203f75793
dut.csr.mepc = 854e85ee86ea8876
dut.csr.sstatus = 908ff0ef0a676733
dut.csr.scause = 4e89bac18a5684aa

上述字段全部是不定值

应该注释掉这一行

检测到if (dut_regs_ptr[i] != ref_regs_ptr[i])之后，增加assert，可以让程序立即停止，如下所示
============== REF Regs ==============
$0: 0x0000000000000000 ra: 0x0000000000000000 sp: 0x0000000000000000 gp: 0x0000000000000000
tp: 0x0000000000000000 t0: 0x0000000000000001 t1: 0x0000000000000000 t2: 0x0000000000000000
s0: 0x0000000000000000 s1: 0x0000000000000000 a0: 0x0000000000000000 a1: 0x0000000000000000
a2: 0x0000000000000000 a3: 0x0000000000000000 a4: 0x0000000000000000 a5: 0x0000000000000000
a6: 0x0000000000000000 a7: 0x0000000000000000 s2: 0x0000000000000000 s3: 0x0000000000000000
s4: 0x0000000000000000 s5: 0x0000000000000000 s6: 0x0000000000000000 s7: 0x0000000000000000
s8: 0x0000000000000000 s9: 0x0000000000000000 s10: 0x0000000000000000 s11: 0x0000000000000000
t3: 0x0000000000000000 t4: 0x0000000000000000 t5: 0x0000000000000000 t6: 0x0000000000000000
ft0: 0x6603cf0dd7057a4d ft1: 0x5b3e034c01d9df56 ft2: 0xc6faa71cb4882fca ft3: 0xac8f2d18b44e3033
ft4: 0x0dd3e856b0bc131c ft5: 0xbae35a9da9f92940 ft6: 0x7f6df64bf1c5b186 ft7: 0xb4720fc58e09f578
fs0: 0xf08842af1e5f21a8 fs1: 0x24a8f0a9ed1aef52 fa0: 0x12a41cf9f928afb0 fa1: 0x72665d147ca91633
fa2: 0x952e6bd65d297284 fa3: 0xeb11e7c7bc91cd9c fa4: 0x72f0cf343b5e4b9c fa5: 0x0e35c4a60d3f3f97
fa6: 0x0cfb7ffb34673dfb fa7: 0xba73cb93ec983dca fs2: 0x77fff78baaf4d972 fs3: 0x17effba07ffbefa9
fs4: 0xecd198f490e7a599 fs5: 0xcb1e2003519283d0 fs6: 0x28f1913370366398 fs7: 0xb7f1318e7d9d235c
fs8: 0x9c9a8046c3a3d81d fs9: 0x51d3e953b5ab7aa7 fs10: 0x48e97cc9994c6ddb fs11: 0x8ecb12eaa2b1c1e7
ft8: 0xb21043888c806c22 ft9: 0xdba911c265bea167 ft10: 0x770fe52f68f001f2 ft11: 0xb52e981edc45e9ea
pc: 0x0000000010000004 mstatus: 0x0000000a00002000 mcause: 0x0000000000000000 mepc: 0x261286e2e492930c
sstatus: 0x0000000200002000 scause: 0x0000000000000000 sepc: 0x0000000000000000
satp: 0x0000000000000000
mip: 0x0000000000000000 mie: 0x0000000000000000 mscratch: 0x0000000000000000 sscratch: 0x0000000000000000
mideleg: 0x0000000000000000 medeleg: 0x0000000000000000
mtval: 0x0000000000000000 stval: 0x8ad80a3f8fae78b6 mtvec: 0x0000000000000000 stvec: 0x0000000000000000
privilege mode:3 pmp: below
0: cfg:0x00 addr:0x0000000000000000| 1: cfg:0x00 addr:0x0000000000000000
2: cfg:0x00 addr:0x0000000000000000| 3: cfg:0x00 addr:0x0000000000000000
4: cfg:0x00 addr:0x0000000000000000| 5: cfg:0x00 addr:0x0000000000000000
6: cfg:0x00 addr:0x0000000000000000| 7: cfg:0x00 addr:0x0000000000000000
8: cfg:0x00 addr:0x0000000000000000| 9: cfg:0x00 addr:0x0000000000000000
10: cfg:0x00 addr:0x0000000000000000|11: cfg:0x00 addr:0x0000000000000000
12: cfg:0x00 addr:0x0000000000000000|13: cfg:0x00 addr:0x0000000000000000
14: cfg:0x00 addr:0x0000000000000000|15: cfg:0x00 addr:0x0000000000000000
emu: /data/jiasen/XiangShan/difftest/src/test/csrc/difftest/difftest.cpp:281: virtual int Difftest::step(): Assertion `dut_regs_ptr[i] == ref_regs_ptr[i] && "Error: dup_regs_ptr[i] != ref_regs_ptr[i]"' failed.
priviledgeMode: 3
t0 different at pc = 0x0010000000, right= 0x0000000000000000, wrong = 0x0000000000000001
t0 different at pc = 0x0010000000, right= 0x0000000000000000, wrong = 0x0000000000000001

Hi, maintainers of Difftest.

Recently, I've been trying to use the Rocket core with Difftest and found that it seems to only support the default extension rv64gc. How can I enable other extensions? (e.g., B extension) . In other words, what efforts should I make to enable Difftest to test the Rocket core with the B extension?

Thanks!

In verilator.mk, we have a feature that SSH to the remote host to build if we have a $REMOTE set. However, this feature is not well maintained.

For example, we didn't pass the RTL_SUFFIX variable to the remote host, and it will use .v by default, but currently, NutShell generated RTL with .sv, so this feature does not work. Similar problems are also caused by WITH_CHISELDB and WITH_CONSTANTIN for NetShell.

Although it can be simply fixed by adding these parameters. However, some features are easy to forget and cause some problems. For example, we currently have PGO_EMU_ARGS, which may have space in the string. If we use some command like ssh $(REMOTE) make PGO_EMU_ARGS=$(PGO_EMU_ARGS), the space will be parsed as the splitter for each argument. I don't know how we need this feature today since the compile time is mostly bounded to single-thread performance since verialtor generates C code takes time. A server with more core helps, but it is not critical.

Thus, I think removing this feature will reduce the cost of maintaining it.

Timestamp t is initialized at this line. In the while loop for simulation, t is used at this line for generating snapshot, but is never updated in the while loop, so no snapshot is actually generated.

我想使用Difftest对自己设计的riscv64处理器进行检测，由于我们没有实现C扩展指令集，我选择了NutShell下的ready-to-run/microbench.bin进行测试，但是结果显示

address (0x0000000000000000) is out of bound {???} [0x0000000000000000, 0x0000000000000000] at pc = 0x0000000000000000

以及

emu: src/device/io/map.c:21: check_bound: Assertion `map != ((void *)0) && addr <= map->high && addr >= map->low' failed.
Aborted (core dumped)

我想请问一下是我在有哪里设置的不对，还是说NutShell版本的bin文件不支持最新的difftest了。
如果不兼容的话，请问一下有不带C扩展指令集的microbench的C源码可以方便我重新编译吗？

感谢您的帮助。

NEMU daffiest_memcpy的定义中前两个参数是固定的němu_addr (ref), *dut_addr，通过最后一个表示方向的参数在函数体内决定调用memcpy的时候的传参

因此，Difftest框架中调用memcpy的时候也同样应该遵循这一原则，永远固定的第一个参数是ref, 第二个是dut_addr

应该统一修改成proxy->memcpy(golden, &dut.load[i].paddr, len, DUT_TO_DIFFTEST);

假设我只改了${NOOP_HOME}/difftest/src/test/csrc/verilator/emu.cpp

如题

请问difftest中为什么会对move指令跳过比对？无论是否消除move在提交后一拍的逻辑寄存器应该是一致的

man shmt show that:

On success, shmat() returns the address of the attached shared memory segment; on error, (void *) -1 is returned, and errno is set to indicate the cause of the error.