Hello, I try to run the test suite but I got this error message <div class="snippe

what do you think about this <a href="https://bugs.debian.org/cgi-bi

howto run the test suite ?,about g1257/dmrgpp

Comments (19)

g1257 commented on August 19, 2024

Please use cd TestSuite/ rm -Rf tests/ ./ci.pl -n "0-*" -S bash to run it on the terminal with bash or ./ci.pl -n "0-*" -S bash to run it on separate nodes with torque/pbs. One can run only some tests with (say) ./ci.pl -n '10-20' -S bash or ./ci.pl -n '10-20' -S qsub After the testsuite finishes, one can compare with ./postCi.pl -g /path/to/OraclesDmrgpp/oracles/tests -n '0-*' (the -g is for gold directory) Any other issues please let me know. ~G.

…

On 12/20/21 08:08, picca wrote: Hello, I try to run the test suite but I got this error message ``` make[1]: Entering directory '/<<PKGBUILDDIR>>' cd TestSuite && ./testsuite.pl -a No such file or directory at ../psimaglite/TestSuite/TestSuiteGlobals.pm line 249. ``` When I am looking at the 249 line I find this ``` #Runs a single test sub testSuite { my ($testNum)=@_; #$tempNum -= 100 if($testNum >= 100); my $procFile = $TestSuiteGlobals::inputsDir."processing$testNum.txt"; my $procLib = $TestSuiteGlobals::inputsDir."processingLibrary.txt"; if(-r $procLib) { if(-r "$procFile") { print "*******START OF TEST $testNum*******\n"; $TestSuiteGlobals::specFile = getSpecFile($procFile); my @Analyses = extractAnalyses($procFile) ; ***@***.***) ? ***@***.***, $procLib)) : (print "Test $testNum does not includes any processing analyses.\n"); print "*******END OF TEST ".$testNum."*******\n"; } else { die "Could not find $procFile: $!"; } } else { die "$!"; } } ``` So it expect the ```processingLibrary.txt``` in the inputs directory. But I can not find this file in the inputs directory. What is wrong here ? thanks

from dmrgpp.

picca commented on August 19, 2024

Hello I end up with this ``` cp dmrg operator make[2]: Leaving directory '/<<PKGBUILDDIR>>/src' make[1]: Leaving directory '/<<PKGBUILDDIR>>' debian/rules override_dh_auto_test make[1]: Entering directory '/<<PKGBUILDDIR>>' cd TestSuite/ && rm -Rf tests/ && ./ci.pl -n "0-*" -S bash '../src/dmrg' -> 'tests/dmrg' '../src/observe' -> 'tests/observe' ./ci.pl: WARNING: Ignored test 0 because it's an SU(2) test and you did not specify -su2 ./ci.pl: WARNING: Ignored test 1 because it's an SU(2) test and you did not specify -su2 ./ci.pl: Cannot open ../batchDollarized.pbs: No such file or directory |2| has 1 observe lines |2|: postTest ./observe -f ../inputs/input2.ain "<gs|c';c|gs>,<gs|2.0*sz;2.0*sz|gs>,<gs|n;n|gs>" >> observe2.txt make[1]: *** [debian/rules:42: override_dh_auto_test] Error 2 make[1]: Leaving directory '/<<PKGBUILDDIR>>' make: *** [debian/rules:11: binary] Error 2 dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2 -------------------------------------------------------------------------------- Build finished at 2021-12-20T14:36:25Z ``` The name of the file is batchDollarized.pbs.sample and not batchDollarized.pbs Is it normal ?

from dmrgpp.

g1257 commented on August 19, 2024

I think one needs: cp batchDollarized.pbs.sample batchDollarized.pbs so that that file is there, but please let me know how it goes. ~G.

from dmrgpp.

picca commented on August 19, 2024

ok, with this cp I got this build log Do not hesitate to gives me your opinion about the tes suite.

from dmrgpp.

picca commented on August 19, 2024

So I try to run the postCI part now, but I get this error. It seems that the oracles files are missing in the repository. Indeed it seems that the size of all this is around 4Go. I am right ? ./postCi.pl: kompare tests/observe2.txt oracles/tests/observe2.txt ./postCi.pl: File oracles/tests/observe2.txt NOT FOUND work has 2 observe matrices -- gold has 0 observe matrices ----------------------------------------------- |3|: No oracles/tests/runForinput3.cout found ----------------------------------------------- |4|: No oracles/tests/runForinput4.cout found ----------------------------------------------- |5|: No oracles/tests/runForinput5.cout found ----------------------------------------------- ./postCi.pl: WARNING: Ignored test 6 because it's NOT an SU(2) test and you did not specify -su2 |7|: No oracles/tests/runForinput7.cout found ----------------------------------------------- |8|: No oracles/tests/runForinput8.cout found ----------------------------------------------- |9|: No oracles/tests/runForinput9.cout found ----------------------------------------------- |10|: No oracles/tests/runForinput10.cout found ----------------------------------------------- |11|: No oracles/tests/runForinput11.cout found ----------------------------------------------- |12|: No oracles/tests/runForinput12.cout found ----------------------------------------------- |15|: No oracles/tests/runForinput15.cout found ----------------------------------------------- |18|: No oracles/tests/runForinput18.cout found ----------------------------------------------- |20|: No oracles/tests/runForinput20.cout found ----------------------------------------------- ./postCi.pl: WARNING: Ignored test 21 because it's NOT an SU(2) test and you did not specify -su2

…

----- g1257 ***@***.***> a écrit :

I think one needs: cp batchDollarized.pbs.sample batchDollarized.pbs so that that file is there, but please let me know how it goes. ~G. -- Reply to this email directly or view it on GitHub: #38 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

from dmrgpp.

g1257 commented on August 19, 2024

On 12/20/21 15:56, picca wrote: So I try to run the postCI part now, but I get this error. It seems that the oracles files are missing in the repository. Indeed it seems that the size of all this is around 4Go. I am right ?

Yes, you probably need something like git clone https://code.ornl.gov/gonzalo_3/OraclesDmrg.git cd OraclesDmrg git pull origin features and you should have all those "golden" results to compare. Thanks, ~G.

from dmrgpp.

picca commented on August 19, 2024

Hello, Here the scrip I am suing in order to test dmrgpp with the Debian autopkgtest infra $ cat debian/tests/control Test-Command: set -efu ; cp -rf scripts TestSuite "$AUTOPKGTEST_TMP" ; cd "$AUTOPKGTEST_TMP" ; mkdir -p src ; cp -f /usr/bin/dmrg `find /usr/lib -name observe -type f` src ; git clone https://code.ornl.gov/gonzalo_3/OraclesDmrg.git ; cd "$AUTOPKGTEST_TMP"/OraclesDmrg ; git pull origin features ; cd "$AUTOPKGTEST_TMP"/TestSuite ; rm -Rf tests/ ; cp -f batchDollarized.pbs.sample batchDollarized.pbs ; ./ci.pl -n "0-*" -S bash ; ./postCi.pl -g "$AUTOPKGTEST_TMP"/OraclesDmrg/oracles/tests -n '0-*' Depends: dmrgpp, git, libmath-round-perl Restrictions: allow-stderr, needs-internet And here the first run https://salsa.debian.org/science-team/dmrgpp/-/jobs/2300932/raw Can you tell me if this seems ok for you. You can also find interesting information about the dmrgpp package here https://tracker.debian.org/pkg/dmrgpp Cheers Frederic

from dmrgpp.

g1257 commented on August 19, 2024

It looks good. Thanks for setting it up on debian.org. If there are any recommendations to improve quality or to meet Debian standards please let me know, and we'll see what we can do. Thanks again.

from dmrgpp.

picca commented on August 19, 2024

Hello, you can find here the CI log of dmrgpp for the armhf architecture. https://ci.debian.net/packages/d/dmrgpp/testing/armhf/ It seems thaht it time out on this test |6700| has 1 observe lines |6700|: postTest ./observe -f ../inputs/input6700.ain "<gs|c';c|gs>,<gs|2.0*sz;2.0*sz|gs>,<gs|n;n|gs>" >> observe6700.txt Engine/ObserverHelper.h read 0 out of 14 Engine/ObserverHelper.h read 1 out of 14 Engine/ObserverHelper.h read 2 out of 14 Engine/ObserverHelper.h read 3 out of 14 Engine/ObserverHelper.h read 4 out of 14 Engine/ObserverHelper.h read 5 out of 14 Engine/ObserverHelper.h read 6 out of 14 Engine/ObserverHelper.h read 7 out of 14 Engine/ObserverHelper.h read 8 out of 14 Engine/ObserverHelper.h read 9 out of 14 Engine/ObserverHelper.h read 10 out of 14 Engine/ObserverHelper.h read 11 out of 14 Engine/ObserverHelper.h read 12 out of 14 Engine/ObserverHelper.h read 13 out of 14 ./ci.pl: Submitted Batch3.pbs Batch3.pbs autopkgtest [14:28:56]: ERROR: timed out on command "su -s /bin/bash debci -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true; . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.s5jmciv0/downtmp/build.4ic/src"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.s5jmciv0/downtmp/command1-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.s5jmciv0/downtmp/command1-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.s5jmciv0/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.s5jmciv0/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=160; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; touch /tmp/autopkgtest-lxc.s5jmciv0/downtmp/command1-stdout /tmp/autopkgtest-lxc.s5jmciv0/downtmp/command1-stderr; bash -ec 'set -efu ; cp -rf scripts TestSuite "$AUTOPKGTEST_TMP" ; cd "$AUTOPKGTEST_TMP" ; mkdir -p src ; cp -f /usr/bin/dmrg `find /usr/lib -name observe -type f` src ; git clone https://code.ornl.gov/gonzalo_3/OraclesDmrg.git ; cd "$AUTOPKGTEST_TMP"/OraclesDmrg ; git pull origin features ; cd "$AUTOPKGTEST_TMP"/TestSuite ; rm -Rf tests/ ; cp -f batchDollarized.pbs.sample batchDollarized.pbs ; ./ci.pl -n "0-*" -S bash ; ./postCi.pl -g "$AUTOPKGTEST_TMP"/OraclesDmrg/oracles/tests -n '\''0-*'\''' 2> >(tee -a /tmp/autopkgtest-lxc.s5jmciv0/downtmp/command1-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.s5jmciv0/downtmp/command1-stdout);" (kind: test) Do you have an idea about this issue ? If this is just a timeout problem, is it possible to skip a test easily with a command line argument for memory, the script executed from my side is this one https://salsa.debian.org/science-team/dmrgpp/-/blob/master/debian/tests/control Cheers Fred

from dmrgpp.

picca commented on August 19, 2024

what do you think about this

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1026864

Source: dmrgpp
Version: 6.02-3
Severity: serious
User: [email protected]
Usertags: flaky timeout

Dear maintainer(s),

I looked at the results of the autopkgtest of your package. I noticed 
that it regularly fails on amd64 because it times out after 2 hours and 
47 minutes. Apparently this only happens on our most powerful worker: 
ci-worker13, but not all runs there time out. I'm wondering if this 
might be a race condition that the test only experiences with lots of 
CPUs (64) or lots of memory (256GB) (and depends on the load of the system).

Because the unstable-to-testing migration software now blocks on
regressions in testing, flaky tests, i.e. tests that flip between
passing and failing without changes to the list of installed packages,
are causing people unrelated to your package to spend time on these
tests.

Don't hesitate to reach out if you need help and some more information
from our infrastructure.

Paul

It mades the packages excluded from Debian stable...

from dmrgpp.

picca commented on August 19, 2024

In fact the tests timeout in our infrastructure afer 10000s. The load on the computer is high and makes the dmrgpp test run in more than this timeout. Is it possible to reduce the number of test to reduce this duration without reducing too much the coverage of the tests ?

tanks for your advice.

I have the feeling that I can reduce this with -n but I do not know what is the right range to achieve thie quite good coverage.

from dmrgpp.

picca commented on August 19, 2024

In fatc this is not about the load of the computer. Whe run on a computer with more than 100 cores, the code becomes really slow. Is there a way to limit the number of core used by dmrgpp ?

from dmrgpp.

picca commented on August 19, 2024

I mean globally in the test suite

from dmrgpp.

g1257 commented on August 19, 2024

On 8/3/23 02:22, picca wrote: In fatc this is not about the load of the computer. Whe run on a computer with more than 100 cores, the code becomes really slow. Is there a way to limit the number of core used by dmrgpp ?

Yes, if you run: ./ci.pl -n '0-*' -o "-S 1" this will add -S 1 to ./dmrg and ./observe as you can see in the batch files created by the ci.pl The meaning of -S is to set DMRG++ threads to 1. ~G.

from dmrgpp.

utkarsh2102 commented on August 19, 2024

Oh, sweet. This is great. Thanks, @g1257 (and @picca for driving this!).

from dmrgpp.

picca commented on August 19, 2024

Hello, with the new 6.06 upload into Debian I now have a more reliable test suite. thanks

I limits the number of core used by the openmp part via this snipset

# Limit the number of threads in OpenMP
OMP_NUM_THREADS=$(test $(nproc) -gt 4 && echo 4 || echo $(nproc))
export OMP_NUM_THREADS
echo "Limiting the number of OpenMP thread via OMP_NUM_THREADS=$OMP_NUM_THREADS"

Now I have a bunch of missing oracles files, is it normal ?

I use the master/main branch of https://code.ornl.gov/gonzalo_3/OraclesDmrg.git

which I embed in the Debian package (this way no need to have an internet connection during the tests)

Cheers

3264s |5502|: No tests/runForinput5502.cout found
3264s |5502|: MaxEnergyDiff = NEW ENERGIES UNDEFINED   .NGST.
3264s |5502|: MaxRAM: UNDEFINED 939244 kB
3264s |5502|: UserTime: UNDEFINED 0.796
3264s |5502|: Version: UNDEFINED 5.55 24c483dc956d80ca9b0d349b8ee589ab065affee
3264s |5502| has 1 dmrg lines
3264s |5502| ignoring dmrg label in postCi mode
3264s |5502| has 1 CollectBrakets lines
3264s ./postCi.pl: Could not open tests/CollectBrakets5502_0.txt : No such file or directory
3264s ./postCi.pl: kompare tests/CollectBrakets5502_0.txt /tmp/autopkgtest-lxc.79d7q79s/downtmp/autopkgtest_tmp/OraclesDmrg/oracles/tests/CollectBrakets5502_0.txt
3264s WARNING: Braket n. of labels differs 0 != 6
3264s Warning, Braket with label <gs|c'|P4> not in working dir
3264s Warning, Braket with label <gs|c'|P0> not in working dir
3264s Warning, Braket with label <gs|c'|P3> not in working dir
3264s Warning, Braket with label <gs|c'|P2> not in working dir
3264s Warning, Braket with label <gs|c'|P1> not in working dir
3264s Warning, Braket with label <gs|c'|P5> not in working dir
3264s -----------------------------------------------
3264s |5503|: No tests/runForinput5503.cout found
3264s |5503|: MaxEnergyDiff = NEW ENERGIES UNDEFINED   .NGST.
3264s |5503|: MaxRAM: UNDEFINED 939160 kB
3264s |5503|: UserTime: UNDEFINED 0.399
3264s |5503|: Version: UNDEFINED 5.55 24c483dc956d80ca9b0d349b8ee589ab065affee
3264s |5503| has 1 dmrg lines
3264s |5503| ignoring dmrg label in postCi mode
3264s |5503| has 1 CollectBrakets lines
3264s ./postCi.pl: Could not open tests/CollectBrakets5503_0.txt : No such file or directory
3264s ./postCi.pl: kompare tests/CollectBrakets5503_0.txt /tmp/autopkgtest-lxc.79d7q79s/downtmp/autopkgtest_tmp/OraclesDmrg/oracles/tests/CollectBrakets5503_0.txt
3264s WARNING: Braket n. of labels differs 0 != 2
3264s Warning, Braket with label <gs|A|P2> not in working dir
3264s Warning, Braket with label <gs|A|P3> not in working dir

from dmrgpp.

g1257 commented on August 19, 2024

Sorry for my late reply. If you'd like to list the items you'd want in mainstream, we can take a look. The repo https://code.ornl.gov/gonzalo_3/OraclesDmrg.git is probably too big to include in DMRG++, but if you can include it yourself, that could be the solution then. I wonder also if moving to cmake may make things simpler and more familiar to people. To answer your question about missing tests, that should be fine, and I may keep decreasing full pipeline tests in favor of unit tests, which we don't yet have any. We would still run all full pipeline tests before a version release, but having smaller unit tests may help testing; perhaps using the gtest framework is the way to go here. Any other suggestions you may have, please let me know. Best, ~Gonzalo.

…

On 8/30/23 02:55, picca wrote: Hello, with the new 6.06 upload into Debian I now have a more reliable test suite. thanks I limits the number of core used by the openmp part via this snipset ``` # Limit the number of threads in OpenMP OMP_NUM_THREADS=$(test $(nproc) -gt 4 && echo 4 || echo $(nproc)) export OMP_NUM_THREADS echo "Limiting the number of OpenMP thread via OMP_NUM_THREADS=$OMP_NUM_THREADS" ``` Now I have a bunch of missing oracles files, is it normal ? I use the master/main branch of https://code.ornl.gov/gonzalo_3/OraclesDmrg.git which I embed in the Debian package (this way no need to have an internet connection during the tests) Cheers ``` 3264s |5502|: No tests/runForinput5502.cout found 3264s |5502|: MaxEnergyDiff = NEW ENERGIES UNDEFINED .NGST. 3264s |5502|: MaxRAM: UNDEFINED 939244 kB 3264s |5502|: UserTime: UNDEFINED 0.796 3264s |5502|: Version: UNDEFINED 5.55 24c483d 3264s |5502| has 1 dmrg lines 3264s |5502| ignoring dmrg label in postCi mode 3264s |5502| has 1 CollectBrakets lines 3264s ./postCi.pl: Could not open tests/CollectBrakets5502_0.txt : No such file or directory 3264s ./postCi.pl: kompare tests/CollectBrakets5502_0.txt /tmp/autopkgtest-lxc.79d7q79s/downtmp/autopkgtest_tmp/OraclesDmrg/oracles/tests/CollectBrakets5502_0.txt 3264s WARNING: Braket n. of labels differs 0 != 6 3264s Warning, Braket with label <gs|c'|P4> not in working dir 3264s Warning, Braket with label <gs|c'|P0> not in working dir 3264s Warning, Braket with label <gs|c'|P3> not in working dir 3264s Warning, Braket with label <gs|c'|P2> not in working dir 3264s Warning, Braket with label <gs|c'|P1> not in working dir 3264s Warning, Braket with label <gs|c'|P5> not in working dir 3264s ----------------------------------------------- 3264s |5503|: No tests/runForinput5503.cout found 3264s |5503|: MaxEnergyDiff = NEW ENERGIES UNDEFINED .NGST. 3264s |5503|: MaxRAM: UNDEFINED 939160 kB 3264s |5503|: UserTime: UNDEFINED 0.399 3264s |5503|: Version: UNDEFINED 5.55 24c483d 3264s |5503| has 1 dmrg lines 3264s |5503| ignoring dmrg label in postCi mode 3264s |5503| has 1 CollectBrakets lines 3264s ./postCi.pl: Could not open tests/CollectBrakets5503_0.txt : No such file or directory 3264s ./postCi.pl: kompare tests/CollectBrakets5503_0.txt /tmp/autopkgtest-lxc.79d7q79s/downtmp/autopkgtest_tmp/OraclesDmrg/oracles/tests/CollectBrakets5503_0.txt 3264s WARNING: Braket n. of labels differs 0 != 2 3264s Warning, Braket with label <gs|A|P2> not in working dir 3264s Warning, Braket with label <gs|A|P3> not in working dir ```

from dmrgpp.

picca commented on August 19, 2024

If you'd like to list the items you'd want in mainstream, we can take a look.

For now I just want to be able to run the full suite easily on the Debian autopkgtest CI. It seems that this is possible https://ci.debian.net/packages/d/dmrgpp/unstable/amd64/40237885/ can you tell me if this looks ok for you ?

The repo https://code.ornl.gov/gonzalo_3/OraclesDmrg.git is probably too big to include in DMRG++, but if you can include it yourself, that could be the solution then.

yes, this is what I am doing

I wonder also if moving to cmake may make things simpler and more familiar to people.

I personnaly use autotools for my hkl library https://tracker.debian.org/pkg/hkl but I am investigating meson, which seems nicer to me than cmake. https://mesonbuild.com/index.html I know that I have more difficulties with cmake when packaging than with other build systems. I like particularly the pkg-config system instead of all these cmake Find scripts...

To answer your question about missing tests, that should be fine, and I may keep decreasing full pipeline tests in favor of unit tests, which we don't yet have any.

In my case unit test or full pipelies, I would run all of them in autopkgtest to gfarantie that nothing changed when building with new dependencies

perhaps using the gtest framework is the way to go here.

I do not know, I am using the make check target and a tap system https://testanything.org/ this one https://github.com/rra/c-tap-harness embeded in my hkl library Cheers Frederic

from dmrgpp.

g1257 commented on August 19, 2024

On 11/29/23 03:53, picca wrote: For now I just want to be able to run the full suite easily on the Debian autopkgtest CI. It seems that this is possible https-3A__ci.debian.net_packages_d_dmrgpp_unstable_amd64_40237885 can you tell me if this looks ok for you ?

Yes, it does look good and same as my own run. In the future, we'll have a cleaner process hopefully :-)

> I wonder also if moving to cmake may make things simpler and more > familiar to people. I personnaly use autotools for my hkl library https-3A__tracker.debian.org_pkg_hkl but I am investigating meson, which seems nicer to me than cmake. https-3A__mesonbuild.com_index.html I know that I have more difficulties with cmake when packaging than with other build systems. I like particularly the pkg-config system instead of all these cmake Find scripts...

Interesting. I'll have a look. Thanks for the suggestion.

> To answer your question about missing tests, that should be fine, > and I may keep decreasing full pipeline tests in favor of unit > tests, which we don't yet have any. In my case unit test or full pipelies, I would run all of them in autopkgtest to gfarantie that nothing changed when building with new dependencies > perhaps using the gtest framework is the way to go here.> I do not know, I am using the make check target and a tap system https-3A__testanything.org this one https-3A__github.com_rra_c-2Dtap-2Dharness embeded in my hkl library

Thanks for these ideas too. I'll also take look. Bye, ~Gonzalo.

from dmrgpp.

howto run the test suite ? about dmrgpp HOT 19 OPEN

Comments (19)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent