thu-dsp-lab / ventus-gpgpu Goto Github PK

在异构计算方面，目前gnu只对amdgcn与nvptx做了支持，他们都是闭源的硬件平台。gnu缺乏对一个完整公开的gpgpu的支持。虽然之前有一些工作(https://github.com/pulp-platform/hero)，但是后续似乎都转向了基于llvm的方案。
ventus-gpgpu 是一个雄心勃勃的计划，如果在开发过程中，同步支持gnu的gcc/gomp与llvm/opencl的方案，并在完善后合入gcc主线，这对于开放的异构计算研究将是个很好的消息。
目前，作为个人开发者，我最近在尝试做这方面的工作：

mxlol233-ventus/gcc#1
https://github.com/orgs/mxlol233-ventus/repositories

但是ventus公开的文档比较欠缺，同时我也想与您的项目组保持沟通，并对未来的计划有更详细的了解。所以，想请问下是否存在更加完善的资料，或者工作群可以加一下？

您好，IDE生成Verilog时，报了俩错：
1 .scala: Error: Error compiling the sbt component 'compiler-bridge-2.12.15-66.0'
sbt.internal.inc.CompileFailed: Error compiling the sbt component 'compiler-bridge-2.12.15-66.0'
at sbt.internal.inc.AnalyzingCompiler$.handleCompilationError$1(AnalyzingCompiler.scala:436)
at sbt.internal.inc.AnalyzingCompiler$.$anonfun$compileSources$5(AnalyzingCompiler.scala:453)
at sbt.internal.inc.AnalyzingCompiler$.$anonfun$compileSources$5$adapted(AnalyzingCompiler.scala:448)
at sbt.io.IO$.withTemporaryDirectory(IO.scala:488)
2. scala: Error: org.jetbrains.jps.incremental.scala.remote.ServerException
java.lang.StackOverflowError
at scala.runtime.Statics.anyHash(Statics.java:127)
at scala.util.hashing.MurmurHash3.listHash(MurmurHash3.scala:285)
at scala.util.hashing.MurmurHash3$.seqHash(MurmurHash3.scala:355)
at scala.collection.Seq.hashCode(Seq.scala:41)
at scala.collection.Seq.hashCode$(Seq.scala:41)
at scala.collection.AbstractSeq.hashCode(Seq.scala:1190)

查了查是关于Jdk版本问题，请问你们会遇到这个问题吗，你们使用的JDK版本是什么呢？

指令还未执行完就报错，vfmacc.vf执行完后pc值为undefine

在执行make verilog，出现了如上图的报错。请教一下，是否有解决方法呢？

Hi Ventus GPGPGU team:

I'm setting development environment for Ventus.

I'm flowing this link: https://zhuanlan.zhihu.com/p/586445036

When i execute below command:
make init

I got below Errors:

git submodule update --init --recursive
Cloning into '/home/leonz/hw/ventus-gpgpu/dependencies/berkeley-hardfloat'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:ucb-bar/berkeley-hardfloat.git' into submodule path '/home/leonz/hw/ventus-gpgpu/dependencies/berkeley-hardfloat' failed
Failed to clone 'dependencies/berkeley-hardfloat'. Retry scheduled
Cloning into '/home/leonz/hw/ventus-gpgpu/dependencies/cde'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:chipsalliance/cde.git' into submodule path '/home/leonz/hw/ventus-gpgpu/dependencies/cde' failed
Failed to clone 'dependencies/cde'. Retry scheduled
Cloning into '/home/leonz/hw/ventus-gpgpu/dependencies/fpuv2'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:liuxd17thu/fpuv2.git' into submodule path '/home/leonz/hw/ventus-gpgpu/dependencies/fpuv2' failed
Failed to clone 'dependencies/fpuv2'. Retry scheduled
Cloning into '/home/leonz/hw/ventus-gpgpu/dependencies/rocket-chip'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:chipsalliance/rocket-chip.git' into submodule path '/home/leonz/hw/ventus-gpgpu/dependencies/rocket-chip' failed
Failed to clone 'dependencies/rocket-chip'. Retry scheduled
Cloning into '/home/leonz/hw/ventus-gpgpu/dependencies/rocket-chip-inclusive-cache'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:chipsalliance/rocket-chip-inclusive-cache.git' into submodule path '/home/leonz/hw/ventus-gpgpu/dependencies/rocket-chip-inclusive-cache' failed
Failed to clone 'dependencies/rocket-chip-inclusive-cache'. Retry scheduled
Cloning into '/home/leonz/hw/ventus-gpgpu/dependencies/berkeley-hardfloat'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:ucb-bar/berkeley-hardfloat.git' into submodule path '/home/leonz/hw/ventus-gpgpu/dependencies/berkeley-hardfloat' failed
Failed to clone 'dependencies/berkeley-hardfloat' a second time, aborting
make: *** [Makefile:2: init] Error 1

Do i need set some other things so i can clone above code?

I've downloaded software part from https://opengpgpu.org.cn/html/web/project/release/index.html,
while trying to run some example OpenCL program these problems are encountered:

1. Path used by pocl is absolute:

chmod: cannot access '/home/kl/kl_ws/tsinghua_ws/ventus_ws/llvm-project/install/bin/../../assemble.sh': No such file or directory
sh: 1: /home/kl/kl_ws/tsinghua_ws/ventus_ws/llvm-project/install/bin/../../assemble.sh: not found

which relates to

	  std::string assembler_path = CLANG;
    assembler_path = assembler_path.substr(0,assembler_path.length()-6);
	  assembler_path += "/../../assemble.sh";

https://github.com/THU-DSP-LAB/pocl/blob/414145738992eccdb830c55d4af0501917326ae3/lib/CL/devices/ventus/pocl_ventus.cc#L689

which seems caused by
https://github.com/THU-DSP-LAB/pocl/blob/414145738992eccdb830c55d4af0501917326ae3/config.h.in.cmake#L160
then cannot be used without manually putting the script there.
Moreover, assemble.sh is not included in the package.

2. object.riscv not found

nm: 'object.riscv': No such file
llvm-objdump: error: 'object.riscv': No such file or directory

not sure what the situation is, could you please explain how to get object.riscv?

3. poclcc gives "still have references to IRs - can't release LLVM context !"

➜  /tmp cat vector_add.cl
 // ACL kernel for adding two input vectors
__kernel void vectorAdd(__global float *x, 
                        __global float *y, 
                        __global float *z)
{
    // get index of the work item
    int index = get_global_id(0);

    // add the vector elements
    z[index] = x[index] + y[index];
}
➜  /tmp poclcc vector_add.cl
[INFO]: [HW DRIVER] in [FILE] ventus.cpp,[LINE]25,[fn] vt_dev_open: vt_dev_open : hello world from ventus.cpp
spike device initialize: allocating local memory: to allocate at 0x70000000 with 268435456 bytes 
spike device initialize: allocating pc source memory: to allocate at 0x80000000 with 268435456 bytes 
### options: -DPOCL_DEVICE_ADDRESS_BITS=32 -D__USE_CLANG_OPENCL_C_H -xcl -Dinline= -I. -cl-kernel-arg-info  -D__ENDIAN_LITTLE__=1 -DCL_DEVICE_MAX_GLOBAL_VARIABLE_SIZE=0 -D__OPENCL_VERSION__=200 -cl-std=CL2.0 -D__OPENCL_C_VERSION__=200 -Dcl_khr_fp64=1 -D__opencl_c_generic_address_space=1 -D__opencl_c_named_address_space_builtins=1 -cl-ext=-all,+cl_khr_fp64,+__opencl_c_generic_address_space,+__opencl_c_named_address_space_builtins -fno-builtin -triple=riscv32 -target-cpu ventus-gpgpu user_options: 
### Triple: riscv32, CPU: ventus-gpgpu
still have references to IRs - can't release LLVM context !
[1]    62536 IOT instruction (core dumped)  poclcc vector_add.cl

also not sure what's happening...

when I try the way in the /ventus/fpga_test/read.me ,I find that
1)The folder does not contain the imports mentioned in read.me.
2)After I run the command: make verilog, I get the file whose name is GPGPU_top.v instead of GPGPU_axi_top.v as mentioned in read.me
3)When I run the command: source project_gpgpu.tcl in vivado, I get the error:
WARNING: [IP_Flow 19-2248] Failed to load user IP repository 'c:/Users/25305/Desktop/ip_repo/ CTA_Schedular_1.0'; Can't find the specified path.
I found that the CTA_Schedular_1.0 mentioned in the .tcl file really doesn't show up
And, I would like to ask if you can write the FPGA tutorials in more detail.
Thanks in advance,and this is really a good project

我做了如下修改能跑了，不知道改的对不对

is there SIMT-deadlock issue for the SIMT-stack based divergence? how to deal with it if yes?

Can these test benches run using any other simulator backend besides Treadle provided by chiseltest like Icarus and Veilator? Simulating with IcarusBackendAnnotation will generate a lot of Error messages. Is there any way to run simulations using these simulators?

done.

开发团队你们好
我的环境是Ubuntu20.04,使用java17，make Verilog也是可以执行出来的
但是当我执行命令make tests时，会出现错误：

请问你们能否给出一些建议，非常感谢你们的帮助

我在阅读源码时，有一个疑惑，想请教一下：

class host2CTA_data extends Bundle{
  val host_wg_id            = (UInt(WG_ID_WIDTH.W))
  val host_num_wf           = (UInt(WF_COUNT_WIDTH.W))
  val host_wf_size          = (UInt(WAVE_ITEM_WIDTH.W))
  val host_start_pc         = (UInt(MEM_ADDR_WIDTH.W))
  val host_vgpr_size_total  = (UInt((VGPR_ID_WIDTH + 1).W))
  val host_sgpr_size_total  = (UInt((SGPR_ID_WIDTH + 1).W))
  val host_lds_size_total   = (UInt((LDS_ID_WIDTH + 1).W))
  val host_gds_size_total   = (UInt((GDS_ID_WIDTH + 1).W))
  val host_vgpr_size_per_wf = (UInt((VGPR_ID_WIDTH + 1).W))
  val host_sgpr_size_per_wf = (UInt((SGPR_ID_WIDTH + 1).W))
  val host_gds_baseaddr = UInt(MEM_ADDR_WIDTH.W)
}

貌似host2CTA_data所包含的信息中，并没有一个CUDA中三维网格、线程块ID的概念，那么它们是如何映射到硬件上的呢？即：

• blockIdx.x
• blockIdx.y
• blockIdx.z
• threadIdx.x
• threadIdx.y
• threadIdx.z

你好，请问当前版本实现了MMU了吗？

Environment:
ThisBuild / scalaVersion := "2.13.10"
libraryDependencies += "edu.berkeley.cs" %% "chisel3" % "3.5.6"
addCompilerPlugin("edu.berkeley.cs" % "chisel3-plugin" % "3.5.6" cross CrossVersion.full)

1. Error1
   Code: mask:=Vec(num_fetch,1.U(1.W)).asUInt
   ERROR: vec type 'UInt<1>(1)' must be a Chisel type, not hardware

It seems Vec[T <: Data ] , U(1.W) is hardware, so we should use below code:
mask:=VecInit(Seq.fill(num_fetch)(1.U(1.W))).asUInt

2. Error2
   Code: val pc_mask = Vec(num_fetch, Bool())
   ERROR: data to be connected 'Bool' must be hardware, not a bare Chisel type. Perhaps you forgot to wrap it in Wire() or IO()?

According to : pc_mask(i) := Mux(pc_aligned + (i * 4).U >= pc, true.B, false.B)
pc_mask use ture.B and false.B, these all hardware, so we should use below code:
val pc_mask = VecInit(Seq.fill(num_fetch)(false.B))

BTW, i just learn Chisel a few days. Maybe your Chisel or Chisel compiler version don't have these ERRORs

I'm a little confused because when we check in code, we should make sure our code don't have above similar problem. Right?

So it is probably we use different Chisel version which lead to above compile ERROR.

The README file for fpga_test is 2 years old and so are the instructions and other code files int that folder.
Where can I find the latest instructions/guide on this?

I am trying to figure out the workaround it but so far I haven't been successful. Can someone guide me on how did they deploy this project on a board using these instructions?
To replicate the procedure, I made the directory structure similar to what is mentioned in the README file. I am currently stuck with generating the VIVADO project using project_gpgpu.tcl script as most of the commands in the script are deprecated (the dsa one).

Also, is there some specific FPGA board that we have to use for this project?

当前承影GPU是否有类似传统处理器那样的C模拟器比对验证？
如果不是的话，承影目前有指令粒度的正确性验证吗？目前是如何证明设计的正确性的呢？

As a question, I would like to know where I can configure the number of GPGPU cores?

1、make bsp生成的GPGPU_top.v与GPGPU_axi_top接口完全不同；
2、有没有最新版本的GPGPU_axi_adapter.v与其适配；
3、是否支持fpga_test？

你好，当我在运行 make verilog时，程序出现了错误：
[error] /home/wyk/ventus/ventus-gpgpu (copy)/dependencies/fpuv2/fudian/src/main/scala/fudian/Generator.scala:13:29: value getPackageName is not a member of Class[T]
[error] val pkg = this.getClass.getPackageName
[error] ^
[error] one error found
1 targets failed
fudian.compile Compilation failed
make: *** [Makefile:33: verilog] Error 1
请问你知道这可能是哪些原因导致的吗，感谢

This is a report for a completely new ubuntu environment. I format the machine and install all package from scratch.
Maybe chipsalliance path is not included in file build.sc. Please correct it.

• Configuration as following:

1. Ubuntu version: 20.04 LTS

(base) wx@wx-TianYi510Pro-14IOB:~/git/ventus-gpgpu$ uname -a
Linux wx-TianYi510Pro-14IOB 5.14.0-1058-oem #66-Ubuntu SMP Fri Feb 10 09:46:18 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

1. java version:

(base) wx@wx-TianYi510Pro-14IOB:~/git/ventus-gpgpu$ java -version
openjdk version "11.0.20" 2023-07-18
OpenJDK Runtime Environment (build 11.0.20+8-post-Ubuntu-1ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.20+8-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)

1. mill version

(base) wx@wx-TianYi510Pro-14IOB:~/git/ventus-gpgpu$ ./mill version
[1/1] version
0.10.8

• reports:

(base) wx@wx-TianYi510Pro-14IOB:~/git/ventus-gpgpu$ ./mill -i -j 0 __.compile
[#8] [344/374] foreign-modules.dependencies.fpuv2.fudian.fudian.compile
[#6] [info] compiling 1 Scala source to /home/wx/git/ventus-gpgpu/out/foreign-modules/dependencies/cde/cde/tests/compile.dest/classes ...
[#7] [info] compiling 339 Scala sources to /home/wx/git/ventus-gpgpu/out/myrocketchip/compile.dest/classes ...
[#11] [info] compiling 13 Scala sources to /home/wx/git/ventus-gpgpu/out/foreign-modules/dependencies/berkeley-hardfloat/hardfloat/test/compile.dest/classes ...
[#8] [info] compiling 17 Scala sources to /home/wx/git/ventus-gpgpu/out/foreign-modules/dependencies/fpuv2/fudian/fudian/compile.dest/classes ...

[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/package.scala:6:16: not found: value chipsalliance
[#7] [error] val config = chipsalliance.rocketchip.config
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/amba/ahb/AHBLite.scala:9:29: not found: type Parameters
[#7] [error] class AHBLite()(implicit p: Parameters) extends LazyModule {
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/diplomacy/LazyModule.scala:28:45: not found: type Parameters
[#7] [error] abstract class LazyModule()(implicit val p: Parameters) {
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/diplomacy/LazyModule.scala:269:19: not found: type Parameters
[#7] [error] implicit val p: Parameters = wrapper.p
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/amba/ahb/Parameters.scala:127:11: not found: type Parameters
[#7] [error] params: Parameters,
[#7] [error] ^
[#6] [info] done compiling
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/amba/ahb/AHBLite.scala:38:27: not found: type Parameters
[#7] [error] def apply()(implicit p: Parameters) = {
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/amba/ahb/Nodes.scala:10:44: not found: type Field
[#7] [error] case object AHBSlaveMonitorBuilder extends Field[AHBSlaveMonitorArgs => AHBSlaveMonitorBase]
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/amba/ahb/Nodes.scala:15:72: not found: type Parameters
[#7] [error] def edge(pd: AHBMasterPortParameters, pu: AHBSlavePortParameters, p: Parameters, sourceInfo: SourceInfo) = AHBEdgeParameters(pd, pu, p, sourceInfo)
[#7] [error] ^
[#7] [error] /home/wx/git/ventus-gpgpu/dependencies/rocket-chip/src/main/scala/diplomacy/Nodes.scala:173:29: not found: type Parameters
[#7] [error] def edge(pd: D, pu: U, p: Parameters, sourceInfo: SourceInfo): E
[#7] [error] ^

Chisel will shortly end support to 3.5.X, suggesting upgrade Chisel versions to Chisel 5/6, and migrate to CIRCT flow.

./mill -i ventus.tests.testOnly play.hello_test2
[10/315] mill.scalalib.ZincWorkerModule.worker
1 targets failed
mill.scalalib.ZincWorkerModule.worker java.lang.ClassNotFoundException: mill.scalalib.worker.ZincWorkerImpl
java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
mill.api.ClassLoader$$anon$1.findClass(ClassLoader.scala:47)
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
mill.scalalib.ZincWorkerModule.$anonfun$worker$3(ZincWorkerModule.scala:58)
Makefile:30: recipe for target 'test' failed

Java version:
openjdk version "17.0.7" 2023-04-18
OpenJDK Runtime Environment (build 17.0.7+7-Ubuntu-0ubuntu118.04)
OpenJDK 64-Bit Server VM (build 17.0.7+7-Ubuntu-0ubuntu118.04, mixed mode, sharing)