shadensmith / splatt Goto Github PK
View Code? Open in Web Editor NEWThe Surprisingly ParalleL spArse Tensor Toolkit.
License: MIT License
The Surprisingly ParalleL spArse Tensor Toolkit.
License: MIT License
The command
X = splatt_load('mytensor.tns');
throws an MATLAB:unassignedOutputs
exception when the tensor cannot be loaded, e.g., because the file is not found. I suggest to either throw a "file not found" exception, or to return an empty tensor. (Probably the former is preferable.)
Running splatt_load
(and probably any other splatt function as well), results in a static TLS error:
dlopen: cannot load any more object with static TLS
(On a Ubuntu 14.04 system with Matlab 2016b.)
I traced the error back to the openMPI library used. There seem to be two solutions to mitigate this problem:
startup.m
file. This way the library is loaded soon enough.LD_PRELOAD=libgomp.so.1 matlab
export LD_PRELOAD=libgomp.so.1
matlab
The latter export function can be put in your .bashrc
file (or equivalent).
Both approaches seemed to work for me. The former approach is somewhat messy and will always load SPLATT. The latter approach made loading MATLAB extremely slow (couple of minutes). This can be a licensing error though.
I get the following error in parallel (with mpi) but not on a single node. Any suggestions.
Gram matrix is not SPD. Trying 'GELSS'
Hi,
I am going to implement parallelized tensor decomposition. When I tried the C/C++ API, I found there are several issues:
#include <splatt.h>
int main()
{
/* allocate default options */
double * cpd_opts = splatt_default_opts();
/* load the tensor from a file */
int ret;
splatt_idx_t nmodes;
splatt_csf * tt; //not "splatt_csf_t"!
ret = splatt_csf_load("mytensor.tns", &nmodes, &tt, cpd_opts);
/* do the factorization! */
splatt_kruskal factored; //not "splatt_kruskal_t"!
ret = splatt_cpd_als(tt, 10, cpd_opts, &factored);
/* do some processing */
for(splatt_idx_t m = 0; m < nmodes; ++m) {
/* access factored.lambda and factored.factors[m] */
}
/* cleanup */
splatt_free_csf(tt, cpd_opts);
splatt_free_kruskal(&factored);
splatt_free_opts(cpd_opts);
}
/tmp/cc8AfqHz.o: In function main:
splatt_example.cpp:(.text+0xc): undefined reference to splatt_default_optssplatt_example.cpp:(.text+0x29): undefined reference to splatt_csf_load
splatt_example.cpp:(.text+0x48): undefined reference to splatt_cpd_alssplatt_example.cpp:(.text+0x77): undefined reference to splatt_free_csf
splatt_example.cpp:(.text+0x86): undefined reference to splatt_free_kruskalsplatt_example.cpp:(.text+0x92): undefined reference to splatt_free_opts
collect2: error: ld returned 1 exit status`I'm wondering if there is some approach to fix the "undefined reference" issues. Thanks.
Can someone explain the output below??
splatt v2.0.0
Tensor information ---------------------------------------------
FILE=med.tns
DIMS=2425x10816x29567 NNZ=100000 DENSITY=1.289479e-07
COORD-STORAGE=3.05MB
MPI information ------------------------------------------------
DISTRIBUTION=MEDIUM DIMS=1x1x4
AVG NNZ=25000
MAX NNZ=25006 (0.02% diff)
AVG COMMUNICATION VOL=22531
MAX COMMUNICATION VOL=23352 (3.52% diff)
Factoring ------------------------------------------------------
NFACTORS=800 MAXITS=50 TOL=1.0e-05 REG=0.0e+00 RANKS=4 THREADS=1
CSF-ALLOC=TWOMODE TILE=NO
CSF-STORAGE=6.07MB FACTOR-STORAGE=396.64MB
Hi,
I got the following error when run ./configure --with-mpi && make in my ubuntu 16.04 terminal:
"In function ‘splatt_mpi_cpd_cmd’:
/home/zrahimi/splatt-tucker/src/cmds/mpi_cmd_cpd.c:247:35: error: ‘CSF_SORTED_MINUSONE’ undeclared (first use in this function)
csf_alloc_mode(tt_filtered, CSF_SORTED_MINUSONE, m, csf+m, args.opts);
^
/home/zrahimi/splatt-tucker/src/cmds/mpi_cmd_cpd.c:247:35: note: each undeclared identifier is reported only once for each function it appears in".
I think it is returning to csf.h which "CSF_SORTED_MINUS_ONE" is not declared in typedefs. I added "CSF_SORTED_MINUSONE" to "./src/csf.h" and the make was done without error.
In the main Splatt repository "CSF_SORTED_MINUSONE" exists in "csf.h" but in the tucker branch, it does not. But after I run "./splatt tucker --alloc=greedy ./tests/med5.tns" I got a "segmentation fault" error. The logfiles for --alloc=iter and alloc=greedy are attached.
Thanks in advance
Can splatt support SpMTTKRP, which means the two input matrices are also sparse? Thank you!
Hi, I'm trying to build the sc16-sgd branch as a CP-SGD benchmark. But the make stage failed with the following error. I'm using cmake version 3.24.0-rc3, icc and icpc version 2021.6.0.
make -C build/Linux-x86_64 all
make[1]: Entering directory '/home/zywu/bench/splatt/build/Linux-x86_64'
make[2]: Entering directory '/home/zywu/bench/splatt/build/Linux-x86_64'
make[3]: Entering directory '/home/zywu/bench/splatt/build/Linux-x86_64'
make[3]: Leaving directory '/home/zywu/bench/splatt/build/Linux-x86_64'
make[3]: Entering directory '/home/zywu/bench/splatt/build/Linux-x86_64'
[ 1%] Building C object lib/CMakeFiles/splatt.dir/__/src/base.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 3%] Building C object lib/CMakeFiles/splatt.dir/__/src/bench.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 5%] Building CXX object lib/CMakeFiles/splatt.dir/__/src/bfs.cc.o
icpc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 6%] Building C object lib/CMakeFiles/splatt.dir/__/src/ccp/ccp.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 8%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/als.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
remark #11074: Inlining inhibited by limit max-size
remark #11076: To get full report use -qopt-report=4 -qopt-report-phase ipo
[ 10%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/ccd.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
/home/zywu/bench/splatt/src/completion/ccd.c(153): warning #191: type qualifier is meaningless on cast type
mats[m] = (val_t const * restrict) model->factors[csf->dim_perm[m]] +
^
/home/zywu/bench/splatt/src/completion/ccd.c(294): warning #191: type qualifier is meaningless on cast type
mats[m] = (val_t const * const restrict) model->factors[csf->dim_perm[m]] +
^
/home/zywu/bench/splatt/src/completion/ccd.c(418): warning #191: type qualifier is meaningless on cast type
mats[m] = (val_t const * const restrict) model->factors[csf->dim_perm[m]] +
^
/home/zywu/bench/splatt/src/completion/ccd.c(551): warning #191: type qualifier is meaningless on cast type
mats[m] = (val_t const * const restrict) model->factors[csf->dim_perm[m]] +
^
remark #11074: Inlining inhibited by limit max-size
remark #11076: To get full report use -qopt-report=4 -qopt-report-phase ipo
[ 11%] Building CXX object lib/CMakeFiles/splatt.dir/__/src/completion/completion.cc.o
icpc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 13%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/gd.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 15%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/gradient.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
remark #11074: Inlining inhibited by limit max-size
remark #11076: To get full report use -qopt-report=4 -qopt-report-phase ipo
[ 16%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/lbfgs.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 18%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/liblbfgs/lbfgs.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 20%] Building C object lib/CMakeFiles/splatt.dir/__/src/completion/nlcg.c.o
icc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
[ 21%] Building CXX object lib/CMakeFiles/splatt.dir/__/src/completion/sgd.cc.o
icpc: command line remark #10412: option '-mkl' is deprecated and will be removed in a future release. Please use the replacement option '-qmkl'
/home/zywu/bench/splatt/src/completion/sgd.cc(2315): error: identifier "sgd_comm_t" is undefined
sgd_comm_t * sgd_comm,
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2315): error: identifier "sgd_comm" is undefined
sgd_comm_t * sgd_comm,
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2316): error: type name is not allowed
tc_model const * const model,
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2316): error: too many initializer values
tc_model const * const model,
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2316): error: expected a ")"
tc_model const * const model,
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2318): error: expected a ";"
{
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2325): error: this pragma must immediately precede a statement
#pragma omp parallel reduction(+:reg_obj)
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2335): error: this pragma must immediately precede a statement
#pragma omp for schedule(static) nowait
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2340): warning #12: parsing restarts here after previous syntax error
reg_obj += ws->regularization[m] * accum;
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2341): error: expected a declaration
}
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2344): warning #12: parsing restarts here after previous syntax error
MPI_Allreduce(MPI_IN_PLACE, ®_obj, 1, SPLATT_MPI_VAL, MPI_SUM, MPI_COMM_WORLD);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2346): error: expected an identifier
assert(reg_obj > 0);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2346): error: expected a type specifier
assert(reg_obj > 0);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2347): error: expected a declaration
return reg_obj;
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2348): error: expected a declaration
}
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2373): warning #12: parsing restarts here after previous syntax error
sizeof(val_t)*nlocalrow*nfactors);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2375): error: expected a declaration
for(int p=0; p < rinfo->layer_size[m]; ++p) {
^
/home/zywu/bench/splatt/src/completion/sgd.cc(2377): error: this pragma must immediately precede a statement
#pragma omp parallel for
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3194): warning #12: parsing restarts here after previous syntax error
splatt_csf *csf = NULL;
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3197): error: expected a declaration
if(ws->csf) {
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3221): warning #12: parsing restarts here after previous syntax error
val_t loss = tc_loss_sq(train, model, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3222): error: identifier "model" is undefined
val_t frobsq = tc_frob_sq(model, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3222): error: identifier "ws" is undefined
val_t frobsq = tc_frob_sq(model, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3223): error #77: this declaration has no storage class or type specifier
tc_converge(train, validate, model, loss, frobsq, 0, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3223): error: declaration is incompatible with "bool splatt_tc_converge(const sptensor_t *, const sptensor_t *, const tc_model *, splatt_val_t={double}, splatt_val_t={double}, splatt_idx_t={uint64_t={__uint64_t={unsigned long}}}, tc_ws *)" (declared at line 377 of "/home/zywu/bench/splatt/src/completion/completion.h")
tc_converge(train, validate, model, loss, frobsq, 0, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3223): error: identifier "train" is undefined
tc_converge(train, validate, model, loss, frobsq, 0, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3223): error: identifier "validate" is undefined
tc_converge(train, validate, model, loss, frobsq, 0, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3223): error: identifier "loss" is undefined
tc_converge(train, validate, model, loss, frobsq, 0, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3223): error: too many initializer values
tc_converge(train, validate, model, loss, frobsq, 0, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3229): error #77: this declaration has no storage class or type specifier
timer_start(&ws->tc_time);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3229): error: declaration is incompatible with "void timer_start(sp_timer_t *)" (declared at line 132 of "/home/zywu/bench/splatt/src/completion/../timer.h")
timer_start(&ws->tc_time);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3232): error: expected a declaration
for(e=1; e < ws->max_its+1; ++e) {
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3243): error: this pragma must immediately precede a statement
#pragma omp parallel for
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3253): error: this pragma must immediately precede a statement
#pragma omp parallel for
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3259): warning #12: parsing restarts here after previous syntax error
timer_stop(&ws->train_time);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3262): error #77: this declaration has no storage class or type specifier
timer_start(&ws->test_time);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3262): error: variable "timer_start" has already been defined
timer_start(&ws->test_time);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3263): error #77: this declaration has no storage class or type specifier
loss = tc_loss_sq(train, model, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3264): error #77: this declaration has no storage class or type specifier
frobsq = tc_frob_sq(model, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3264): error: variable "frobsq" has already been defined
frobsq = tc_frob_sq(model, ws);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3265): error #77: this declaration has no storage class or type specifier
obj = loss + frobsq;
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3265): error: variable "obj" has already been defined
obj = loss + frobsq;
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3266): error: expected a declaration
if(tc_converge(train, validate, model, loss, frobsq, e, ws)) {
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3270): warning #12: parsing restarts here after previous syntax error
timer_stop(&ws->test_time);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3273): error: expected a declaration
if(e > 1) {
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3281): warning #12: parsing restarts here after previous syntax error
prev_obj = obj;
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3282): error: expected a declaration
}
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3285): error #77: this declaration has no storage class or type specifier
printf(" update_time %g (%g)\n", (ws->train_time.seconds - ws->shuffle_time.seconds), (ws->train_time.seconds - ws->shuffle_time.seconds)/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3285): error: declaration is incompatible with "int printf(const char *, ...)" (declared at line 318 of "/usr/include/stdio.h")
printf(" update_time %g (%g)\n", (ws->train_time.seconds - ws->shuffle_time.seconds), (ws->train_time.seconds - ws->shuffle_time.seconds)/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3285): error: too many initializer values
printf(" update_time %g (%g)\n", (ws->train_time.seconds - ws->shuffle_time.seconds), (ws->train_time.seconds - ws->shuffle_time.seconds)/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3285): error: a value of type "const char *" cannot be used to initialize an entity of type "int"
printf(" update_time %g (%g)\n", (ws->train_time.seconds - ws->shuffle_time.seconds), (ws->train_time.seconds - ws->shuffle_time.seconds)/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3286): error #77: this declaration has no storage class or type specifier
printf(" shuffle_time %g (%g)\n", ws->shuffle_time.seconds, ws->shuffle_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3286): error: variable "printf" has already been defined
printf(" shuffle_time %g (%g)\n", ws->shuffle_time.seconds, ws->shuffle_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3286): error: too many initializer values
printf(" shuffle_time %g (%g)\n", ws->shuffle_time.seconds, ws->shuffle_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3286): error: a value of type "const char *" cannot be used to initialize an entity of type "int"
printf(" shuffle_time %g (%g)\n", ws->shuffle_time.seconds, ws->shuffle_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3287): error #77: this declaration has no storage class or type specifier
printf(" test_time %g (%g)\n", ws->test_time.seconds, ws->test_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3287): error: variable "printf" has already been defined
printf(" test_time %g (%g)\n", ws->test_time.seconds, ws->test_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3287): error: too many initializer values
printf(" test_time %g (%g)\n", ws->test_time.seconds, ws->test_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3287): error: a value of type "const char *" cannot be used to initialize an entity of type "int"
printf(" test_time %g (%g)\n", ws->test_time.seconds, ws->test_time.seconds/e);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3295): error #77: this declaration has no storage class or type specifier
splatt_free(perm);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3295): error: declaration is incompatible with "void splatt_free(void *)" (declared at line 70 of "/home/zywu/bench/splatt/src/completion/../base.h")
splatt_free(perm);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3295): error: a value of type "splatt_idx_t={uint64_t={__uint64_t={unsigned long}}} *" cannot be used to initialize an entity of type "int"
splatt_free(perm);
^
/home/zywu/bench/splatt/src/completion/sgd.cc(3299): error: expected a declaration
}
^
compilation aborted for /home/zywu/bench/splatt/src/completion/sgd.cc (code 2)
lib/CMakeFiles/splatt.dir/build.make:243: recipe for target 'lib/CMakeFiles/splatt.dir/__/src/completion/sgd.cc.o' failed
make[3]: *** [lib/CMakeFiles/splatt.dir/__/src/completion/sgd.cc.o] Error 2
make[3]: Leaving directory '/home/zywu/bench/splatt/build/Linux-x86_64'
CMakeFiles/Makefile2:178: recipe for target 'lib/CMakeFiles/splatt.dir/all' failed
make[2]: *** [lib/CMakeFiles/splatt.dir/all] Error 2
make[2]: Leaving directory '/home/zywu/bench/splatt/build/Linux-x86_64'
Makefile:135: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/home/zywu/bench/splatt/build/Linux-x86_64'
Makefile:11: recipe for target 'all' failed
make: *** [all] Error 2
Seems there are some bugs with the code. How can I solve this problem?
I'm attempting to install on Windows because attempting to install on a Linux machine results in errors where cmake has trouble parsing the \r carriage return.
When I run .configure I get this output:
Found CMAKE: '/c/Program Files/CMake/bin/cmake'
Removing old build directory 'build/MINGW64_NT-10.0-19045-x86_64'...
mkdir: created directory 'build/MINGW64_NT-10.0-19045-x86_64'
~/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64 ~/Documents/CSC548/project/Ndblock_higher/splatt
Calling cmake with arguments ''
-- Building for: Visual Studio 17 2022
CMake Warning (dev) at CMakeLists.txt:1 (project):
cmake_minimum_required() should be called prior to this top-level project()
call. Please see the cmake-commands(7) manual for usage documentation of
both commands.
This warning is for project developers. Use -Wno-dev to suppress it.
-- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.35.32216.1
-- The CXX compiler identification is MSVC 19.35.32216.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Deprecation Warning at CMakeLists.txt:2 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Building in RELEASE mode.
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Could NOT find BLAS (missing: BLAS_LIBRARIES)
-- Could NOT find LAPACK (missing: LAPACK_LIBRARIES)
Reason given by package: LAPACK could not be found because dependency BLAS could not be found.
CMake Error at cmake/lapack.cmake:39 (message):
Could not find LAPACK library. Run `./configure --help` for assistance.
Call Stack (most recent call first):
CMakeLists.txt:27 (include)
-- Configuring incomplete, errors occurred!
CMake failed with '0'
Since it recommended to run .configure --help to see what I could do about the LAPACK library, I found that the command --download-blas-lapack would take care of the dependencies for me. So I tried running:
Found CMAKE: '/c/Program Files/CMake/bin/cmake'
Removing old build directory 'build/MINGW64_NT-10.0-19045-x86_64'...
mkdir: created directory 'build/MINGW64_NT-10.0-19045-x86_64'
~/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64 ~/Documents/CSC548/project/Ndblock_higher/splatt
Calling cmake with arguments ' -DDOWNLOAD_BLAS_LAPACK=TRUE'
-- Building for: Visual Studio 17 2022
CMake Warning (dev) at CMakeLists.txt:1 (project):
cmake_minimum_required() should be called prior to this top-level project()
call. Please see the cmake-commands(7) manual for usage documentation of
both commands.
This warning is for project developers. Use -Wno-dev to suppress it.
-- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.35.32216.1
-- The CXX compiler identification is MSVC 19.35.32216.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Deprecation Warning at CMakeLists.txt:2 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Building in RELEASE mode.
-- Building BLIS & reference LAPACK.
Using user supplied LAPACK=C:/Users/abirm/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64/blis/lib/liblapack.a
Using user supplied BLAS=C:/Users/abirm/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64/blis/lib/libblis.a
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- The Fortran compiler identification is unknown
CMake Error at cmake/fortran.cmake:6 (enable_language):
No CMAKE_Fortran_COMPILER could be found.
Call Stack (most recent call first):
CMakeLists.txt:31 (include)
-- Configuring incomplete, errors occurred!
CMake failed with '0'
Since it said CMAKE_Fortran_COMPILER was not found, I decided to set the DCMAKE_Fortran_COMPILER flag in the configure file like so: CONFIG_FLAGS="-DCMAKE_Fortran_COMPILER=C:/MinGW/bin/gfortran.exe"
Then I ran the command again to this Visual Studio related error and I am unsure where to continue from here:
Found CMAKE: '/c/Program Files/CMake/bin/cmake'
Removing old build directory 'build/MINGW64_NT-10.0-19045-x86_64'...
mkdir: created directory 'build/MINGW64_NT-10.0-19045-x86_64'
~/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64 ~/Documents/CSC548/project/Ndblock_higher/splatt
Calling cmake with arguments '-DCMAKE_Fortran_COMPILER=C:/MinGW/bin/gfortran.exe -DDOWNLOAD_BLAS_LAPACK=TRUE'
-- Building for: Visual Studio 17 2022
CMake Warning (dev) at CMakeLists.txt:1 (project):
cmake_minimum_required() should be called prior to this top-level project()
call. Please see the cmake-commands(7) manual for usage documentation of
both commands.
This warning is for project developers. Use -Wno-dev to suppress it.
-- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.35.32216.1
-- The CXX compiler identification is MSVC 19.35.32216.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Deprecation Warning at CMakeLists.txt:2 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Building in RELEASE mode.
-- Building BLIS & reference LAPACK.
Using user supplied LAPACK=C:/Users/abirm/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64/blis/lib/liblapack.a
Using user supplied BLAS=C:/Users/abirm/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64/blis/lib/libblis.a
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- The Fortran compiler identification is unknown
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - failed
-- Check for working Fortran compiler: C:/MinGW/bin/gfortran.exe
-- Check for working Fortran compiler: C:/MinGW/bin/gfortran.exe - broken
CMake Error at C:/Program Files/CMake/share/cmake-3.26/Modules/CMakeTestFortranCompiler.cmake:59 (message):
The Fortran compiler
"C:/MinGW/bin/gfortran.exe"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: C:/Users/abirm/Documents/CSC548/project/Ndblock_higher/splatt/build/MINGW64_NT-10.0-19045-x86_64/CMakeFiles/CMakeScratch/TryCompile-dptjj0
Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/devenv.com CMAKE_TRY_COMPILE.sln /build Debug /project cmTC_7116d &&
Microsoft Visual Studio 2022 Version 17.5.3.
Copyright (C) Microsoft Corp. All rights reserved.
Invalid project
Use:
devenv [solutionfile | projectfile | folder | anyfile.ext] [switches]
The first argument for devenv is usually a solution file, project file or a folder.
You can also use any other file as the first argument if you want to have the
file open automatically in an editor. When you enter a project file, the IDE
looks for an .sln file with the same base name as the project file in the
parent directory for the project file. If no such .sln file exists, then the
IDE looks for a single .sln file that references the project. If no such single
.sln file exists, then the IDE creates an unsaved solution with a default .sln
file name that has the same base name as the project file.
Command line builds:
devenv solutionfile.sln /build [ solutionconfig ] [ /project projectnameorfile [ /projectconfig name ] ]
Available command line switches:
/Build Builds the solution or project with the specified solution
configuration. For example "Debug". If multiple platforms
are possible, the configuration name must be enclosed in quotes
and contain platform name. For example: "Debug|Win32".
/Clean Deletes build outputs.
/Command Starts the IDE and executes the command.
/Deploy Builds and then deploys the specified build configuration.
/DoNotLoadProjects Opens the specified solution without loading any projects.
/Edit Opens the specified files in a running instance of this
application. If there are no running instances, it will
start a new instance with a simplified window layout.
/LCID Sets the default language in the IDE for the UI.
/Log Logs IDE activity to the specified file for troubleshooting.
/NoVSIP Disables the VSIP developer's license key for VSIP testing.
/Out Appends the build log to a specified file.
/Project Specifies the project to build, clean, or deploy.
Must be used with /Build, /Rebuild, /Clean, or /Deploy.
/ProjectConfig Overrides the project configuration specified in the solution
configuration. For example "Debug". If multiple platforms are
possible, the configuration name must be enclosed in quotes
and contain platform name. For example: "Debug|Win32".
Must be used with /Project.
/Rebuild Cleans and then builds the solution or project with the
specified configuration.
/ResetSettings Restores the IDE's default settings, optionally resets to
the specified VSSettings file.
/ResetSkipPkgs Clears all SkipLoading tags added to VSPackages.
/Run Compiles and runs the specified solution.
/RunExit Compiles and runs the specified solution then closes the IDE.
/SafeMode Launches the IDE in safe mode loading minimal windows.
/Upgrade Upgrades the project or the solution and all projects in it.
A backup of these files will be created as appropriate. Please
see Help on 'Visual Studio Conversion Wizard' for more
information on the backup process.
Product-specific switches:
/debugexe Open the specified executable to be debugged. The remainder of
the command line is passed to this executable as its arguments.
/diff Compares two files. Takes four parameters:
SourceFile, TargetFile, SourceDisplayName(optional),
TargetDisplayName(optional)
/sqldbaction Start SQL Server Data Tools and perform the action specified in the argument string.
/TfsLink Opens Team Explorer and launches a viewer for the
provided artifact URI if one is registered.
/useenv Use PATH, INCLUDE, LIBPATH, and LIB environment variables
instead of IDE paths for VC++ builds.
To attach the debugger from the command line, use:
VsJITDebugger.exe -p <pid>
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
cmake/fortran.cmake:6 (enable_language)
CMakeLists.txt:31 (include)
-- Configuring incomplete, errors occurred!
CMake failed with '0'
Any suggestions?
Hi Shaden
The compilation in Matlab seems to work now on the master branch, after you fix a configure error. The README.md file suggests
./configure --blas-int=int64_t
while the types.cmake
file expects either 32
or 64
. Configuring with --blas-int=64
seems to work.
Cheers
Nico
Consider a basic CPD of a tensor using the matlab interface:
X = splatt_load('mytensor.tns');
U = splatt_cpd(X, 5);
If we choose mytensor.tns
to be the example given in the readme.md file, everything works well. If we choose the nips.tns
file from frostt-tensor instead, splatt_cpd
throws a segmentation fault. (splatt_load
works fine). I haven't tested the C interface.
During linking:
../lapack/lib/liblapack.a(xerbla.f.o): In function
xerbla_': xerbla.f:(.text+0x79): undefined reference to
_gfortran_st_write'
xerbla.f:(.text+0x90): undefined reference to_gfortran_string_len_trim' xerbla.f:(.text+0xb3): undefined reference to
_gfortran_transfer_character_write'
xerbla.f:(.text+0xd1): undefined reference to_gfortran_transfer_integer_write' xerbla.f:(.text+0xe0): undefined reference to
_gfortran_st_write_done'
xerbla.f:(.text+0xef): undefined reference to `_gfortran_stop_string'
collect2: error: ld returned 1 exit status
We need to use CMake's enable_language(Fortran)
and link against necessary libraries.
Hello,
I am trying to benchmark SPLATT MTTKRP with MPI, but am running into issues with memory scalability and wanted to check if I am doing anything wrong.
I configure with --with-mpi
and build via make (cmake). I have been setting OMP_NUM_THREADS=1
and running e.g., mpirun -np 4 splatt bench <my_tensor_file> -a splatt -i 1
or -a csf
. I am able to run both variants to completion for a 2K-by-2K-by-2K random tensor with density .01, on one KNL node of Stampede2 with one process. However, running the same problem with 64 processes per node on 8 nodes immediately returns an error that suggests more memory is being allocated than there is available (seems the tensor is not read in successfully).
I've also tried a problem on my laptop locally that is 800-by-800-by-800 with density .125, for which -a csf
works fine with 1,2,4 processes, but -a splatt
fails with 2 or 4 MPI processes with a segfault, but runs fine with 1 process.
Please let me know if I am doing things in the wrong way or if splatt bench
does not actually support MPI benchmarking.
Inconsistency in the example of README file and the website example, "splatt_csf_t" and "splatt_kruskal_t" you mean"splatt_csf" and "splatt_kruskal" I think. No "cpd_opts" in the website example, "splatt_opts".
Hi,
I have a 1000001000002 tensor which is sparse. I have used splatt to factorize it but I have got fits more than 1000. Could you please help me? I always thought that fits have to be less than one. How can I solve this?
thanks in advance
There's a seg. fault when running the mpi version with1K processors or more.
Config command:
./configure --with-mpi
Run command:
mpiexec -np 1024 splatt cpd enron.tns -t 1 -r 16
The input tensor can be found here
Some trace info:
in splatt_tt_get_slices (tt=0x5810640, m=0, nunique=0x7ffe7d607c20)
in p_greedy_mat_distribution (rinfo=0x7ffe7d607dd0, tt=0x5810640, perm=0x5a96b80)
splatt/src/mpi/mpi_mat_distribute.c:464 in splatt_mpi_distribute_mats (rinfo=0x7ffe7d607dd0, tt=0x5810640, distribution=SPLATT_DECOMP_MEDIUM)
splatt/src/mpi/mpi_mat_distribute.c:616 in splatt_mpi_cpd_cmd (argc=8, argv=0x7ffe7d6085f0)
splatt/src/cmds/mpi_cmd_cpd.c:219 in main (argc=9, argv=0x7ffe7d6085e8)
I have very little experience with Matlab -- is there a good unit testing framework that I could use?
Bonus points if it works in Octave as well, so I can run it on travis-ci.
Right now the CPD can only be initialized randomly and controlled via the command-line seed parameter (--seed=<integer>
). We should add the ability to initialize based on input files, etc.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.