sx-aurora / veda Goto Github PK

[egonzalez@XAIJPVE1 build]$ make
[  3%] Performing download step (git clone) for 'aveo'
Cloning into 'src'...
error: pathspec 'd2b04de' did not match any file(s) known to git.
CMake Error at tmp/aveo-gitclone.cmake:40 (message):
  Failed to checkout tag: 'd2b04de'

//once: vedaCtxCreate(&ctx, VEDA_CONTEXT_MODE_OMP, 0)

vedaDevicePrimaryCtxRetain(&ctx, device);
vedaCtxPushCurrent(ctx);
vedaMemAllocAsync
vedaMemcpyHtoDAsync
...
call kernel
 vedaCtxPopCurrent(&ctx);

I keep Veda calls async and synchronize them when I need them inside the CPU operations.

to my surprise, my async Veda operations took more time than I expected, as it was synchronizing inside (vedaMemcpyHtoDAsync).

[ 85%] Linking CXX executable veda_test
libfill_lib.so: undefined reference to `veraInit()'

I understand I should be linking my host library with libvera.so if I'm going to use VERA functionality. For the VEDA functionality, CMake defines VEDA_LIBRARY:

grep -r "FIND_LIBRARY"
veda-0.9.5.1/cmake/FindVE.cmake:  FIND_LIBRARY(VEDA_LIBRARY "libveda.so" "libveda.a" PATHS "${VEDA_DIR}/lib64")

/usr/local/ve/veda-0.9.5/include/vera.h:4:24: fatal error: vera_enums.h: No such file or directory  
 #include "vera_enums.h"

As a side note, when searching the aurora system (and the github repository), I can't find the next file included in vera.h either:

• vera_types.h

#0  0x00007ffff5e21143 in std::__cxx11::_List_base<veda::Context*, std::allocator<veda::Context*> >::_M_clear (this=0x7ffff7fcb980) at /usr/include/c++/8/bits/list.tcc:74
#1  0x00007ffff5e21030 in std::__cxx11::list<veda::Context*, std::allocator<veda::Context*> >::clear (this=0x7ffff7fcb980) at /usr/include/c++/8/bits/stl_list.h:1508
#2  0x00007ffff5e2093f in veda::Contexts::shutdown () at /home/qwr/veda/src/veda/Contexts.cpp:54
#3  0x00007ffff5e09041 in vedaExit () at /home/qwr/veda/src/veda.cpp:36
#4  0x0000000001e500e6 in VEDA_HANDLE::~VEDA_HANDLE() ()
#5  0x00007ffff4ae4b0c in __run_exit_handlers () from /lib64/libc.so.6
#6  0x00007ffff4ae4c40 in exit () from /lib64/libc.so.6
#7  0x00007ffff4ace49a in __libc_start_main () from /lib64/libc.so.6

here, I wrote simple thread_local + std::list to show problem
https://godbolt.org/z/7jPWWexGP

[VE] ERROR: sigactionHandler() Interrupt signal 11 received
0x600fffe00000
0x60001004cf40 -> (null)
0x600c01583b60 -> __vthr$_pcall_va
[VH] [TID 3611133] ERROR: unpack_call_result() VE exception 11
�@$+U?�^\s�?(*f@xϾ?�v�A]
[VH] [TID 3611133] ERROR: _progress_nolock() Internal error on executing a command(-4)
[VEDA_ERROR_VEO_COMMAND_EXCEPTION] /home/qwr/veda/src/veda/Context.cpp (435)

I'm using the veda this way: for now I integrated your vednn library through veda, so it has few ops support
java->javacpp-> nd4jcpp+veda-> device vednn lib

Veda handle cpp class {
     function,
     module, 
     ctx

}

Singleton Veda{
   list<veda_handle> ;
  //for now I'm using one device
  init(){ 
  init and load libs per device, 
 also I pop
vedaCtxPopCurrent;
//this way I'm solving vedaExit problem that I had because of thread local std::list
 }
exit() { vedaExit;}

}

    vedaDevicePrimaryCtxRetain(&ctx, device);
    vedaCtxPushCurrent(ctx);

    veda Mem calls..
   veda Launch

   sync

   vedaCtxPopCurrent(&ctx);

inline VEDAresult vedaArgsSet(VEDAargs args, const int idx, const int32_t value) {
	return vedaArgsSetI32(args, idx, value);
}

inline VEDAresult vedaArgsSet(VEDAargs args, const int idx, const int64_t value) {
	return vedaArgsSetI64(args, idx, value);
}

inline VEDAresult vedaArgsSet(VEDAargs args, const int idx, const float value) {
	return vedaArgsSetF32(args, idx, value);
}

inline VEDAresult vedaArgsSet(VEDAargs args, const int idx, const double value) {
	return vedaArgsSetF64(args, idx, value);
}
.....
....
template<typename T, typename... Args>
inline VEDAresult __vedaLaunchKernel(VEDAfunction func, VEDAstream stream, uint64_t* result, VEDAargs args, const int idx, const T value, Args... vargs) {
    static_assert(!std::is_same<T, bool>::value, "Don't use bool as data-type when calling a VE function, as it defined as 1B on VH and 4B on VE!");
	vedaArgsSet(args, idx, value);
	return __vedaLaunchKernel(func, stream, result, args, idx+1, vargs...);
}

VEDA(vedaMemcpyDtoHAsync(cpu_return, kern_sum, N*sizeof(int), stream));

VEDA(vedaMemFreeAsync(x_device, stream));
VEDA(vedaMemFreeAsync(y_device, stream));
VEDA(vedaMemFreeAsync(kern_sum,  stream));

vedaCtxSynchronize();

Now. If I replace the device allocation by a pre-allocation, then the array returned has some garbage in the first few elements. The only way to get a proper data return is to put the vedaCtxSynchronize() call before the vedaMemFreeAsync() calls.

Did I do something wrong, or is this a bug? Intuitively I would have put a sync() before a free() to start with, but perhaps that's then missing the point of the vedaMemFreeAsync() functionality?

See attachment for a minimal test case.

simple_add_kernel.tar.gz