boutproject / bout-configs Goto Github PK

One of the nice things about having a repo like this with version-controlled configurations is that it makes it easier to do provenance tracking. Just having the repo exist is a pretty good start, as e.g. STORM can grab the commit hash (plus git diff if there is one) and save it. Just wondering if we could add any features to this repo (and/or BOUT-dev) to make provenance tracking easier... For example when configuring could we package up some info about BOUT-configs (git hash + git commit + which configuration [i.e. machine] is being used + ??) and inject it into some CMake variables that BOUT could then put into revision.hxx or something and save to dump files. Could maybe have some shared utility scripts that could be called in different configurations to create the CMake arguments.

ARCHER2 has just come back from a major system software upgrade. The bout.env needs updating, and it is possible other scripts do too. Should just be a case of updating to the new module versions.

If anyone wants to volunteer to test and update the archer branch that would be very welcome!

It would probably be good to update the BOUT-dev version at the same time to the latest release.

Note: we load the netcdf4 module for python. I had a problem on a different project loading the matplotlib module (which seemed to be incompatible with the new version of python provided on ARCHER2). The docs recommend pip-installing any extra packages needed (note this requires a little bit of setup in your .bashrc, see https://docs.archer2.ac.uk/user-guide/python/).

I have failed to get the Intel compilers (either 2018 or 2020 versions) to work on Marconi. The problem seems to be related to linking libstdc++. See failed attempts here https://github.com/boutproject/BOUT-configs/tree/marconi-intel-dev.

By default the Intel compilers use the 'system' libstdc++ in /usr/lib. On Marconi this seems to be old, and does not support C++11 features.

Loading the gnu/8.3.0 module helps a bit - compilation errors go away, but I get run-time errors which seem to happen because the executable gets linked to a libstdc++ that sits somewhere inside the intel/ module (in an inspector/ subdirectory) which does not have a recent enough version. I tried adding the gnu lib/ directory at the beginning of LD_LIBRARY_PATH and got different run-time errors (BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES, with EXIT CODE: 4).

I'm not sure what's changed, because I did manage to compile BOUT++ (with a pre-release 4.4 version, boutproject/BOUT-dev@154ebb7) previously (early 2021). I had set up modules for that by source'ing the following

module purge

export LD_LIBRARY_PATH=""

# Intel-2020 versions
module load profile/global profile/advanced profile/candidate env-skl intel/pe-xe-2020--binary intelmpi/2020--binary mkl/2020--binary szip/2.1--gnu--6.1.0 zlib/1.2.8--gnu--6.1.0 hdf5/1.12.0--intel--pe-xe-2020--binary fftw/3.3.8--intelmpi--2020--binary python vtune

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$WORK/$USER/intel-versions/pkg/sundials_inst/lib64:$WORK/$USER/intel-versions/pkg/netcdf_inst/lib64/

I'm giving up on the Intel compilers now, so leaving this issue in case anyone needs to try again in future.

I'm planning to set up BOUT++ on ARCHER2 for a few users, am thinking it would be nice to add the config(s) here. Wondering how it's best to organise all the different configs.

One possibility that comes to mind would be to switch to having a different branch for each machine (or possibly for each group of machines).
Pros:

• No clutter of configs for machines that you aren't using.
• If we want to include the BOUT-dev repo as a submodule with a known-good version for each machine config, then only one copy of BOUT-dev will exist for each branch. If we have all machines as subdirectories, then potentially each one wants a different version of BOUT-dev, so you could end up with many copies of BOUT-dev.
  Cons:
• Clutter of branches - user needs to know or be told which branch to use. Hopefully "machine name" -> "branch name" is a simple solution to this though!

Any thoughts, suggestions, other alternatives?