eessi / docs Goto Github PK
View Code? Open in Web Editor NEWDocumentation for the European Environment for Scientific Software Installations (EESSI)
Home Page: https://eessi.github.io/docs
License: GNU General Public License v2.0
Documentation for the European Environment for Scientific Software Installations (EESSI)
Home Page: https://eessi.github.io/docs
License: GNU General Public License v2.0
The documentation at the Build nodes page is out of date. Many of the installation steps have been moved to build_container.sh
and some of the hard-coded paths have changed since the documentation was written.
Since new software installation is now achieved mainly through the bot, perhaps this page could be revised with up to date information for the purpose of manually debugging cases that require a more manual inspection of each step.
The list of partners is currently not sorted alphabetically (SURF comes after the universities), this should probably be changed (unless this was done on purpose?).
Add documentation on setup of an EESSI cluster.
Add example automated setup scripts that can be used on generic clouds.
We should add a section in the documentation that explains how a Squid proxy can be set up. We could make use of the template in the filesystem layer repo; this still has to be updated for the new repo, though.
N.B.: it should also mention that if the cvmfs-config-default
is being used by clients to connect to EESSI, the proxy should allow the Cern Stratum 1 servers in their destination ACL! Otherwise the configuration repo, and hence EESSI configuration files, cannot be accessed, as we found out during the best practices tutorial ๐
Is there a way to drop the "Home" tab in the header? That introduces a lot of space that is useless unless we add more tabs (and the logo already performs the home function)
Images are loading a bit slow on my machine/connection, maybe we can improve this by optimizing the size or the resolution?
I'd like to see a chapter in the docs describing what a client needs at minimum to start making use of EESSI repository, both in a free text form and in some container definition form such as Dockerfile. Aim is to inform infrastructure architects about what they need to provide to enable adoption of EESSI.
see EESSI/software-layer#40 as starting point
next steps:
export EESSI_PILOT_VERSION=2020.12
/cvmfs/pilot.eessi-hpc.org/${EESSI_PILOT_VERSION}/compat/linux/$(uname -m)/startprefix
./EESSI-pilot-install-software.sh
Also need to document:
$EESSI_SOFTWARE_SUBDIR_OVERRIDE
to override subdirectory in which software is installed--generic
option for EESSI-pilot-install-software.sh
script/tmp
for $EESSI_TMPDIR
, you may have to bind mount that location by passing -B $EESSI_TMPDIR
to Singularity. We can probably just always use this flag in the instructions, even when /tmp
is being used.startprefix <<< buildscript.sh
)$EESSI_TMPDIR
(which can be set to something else than /tmp
), or we should bind mount a subdir of $EESSI_TMPDIR
as /tmp
in the container${EESSI_TMPDIR}/overlay-upper/${EESSI_PILOT_VERSION}/software/linux
We should document how to "onboard" new people to the project:
https://eessi.github.io/ (without docs/
) now returns a 404 error page. We probably should put something (anything) there, or redirect to https://eessi.github.io/docs/ or https://www.eessi-hpc.org/?
I think the general rule for the docs should be: Don't scare people into thinking this is too complex
As a reader with an above average attention span (there there people that are better than me but more that are worse) I would recommend to hide any complexity of EESSI in the documentation and basically have 3 main entry points:
Using EESSI by persona
1.1 Sysadmin (person with root): install packages and source the stuff in /etc/profiles.d
1.2. User: ask sysadmin to install packages and source yourself and/or put it in your profile
contributing to EESSI : https://www.eessi.io/docs/adding_software/opening_pr/
Want to know more (background info, policies, slurm clusters, etc ..... most things that people never read. Those also should be de-prioritized in the mkdocs-material searches so not confuse people
I find this quote explains it all:
"What if there was a way to avoid having to install a broad range of scientific software from scratch on every HPC cluster or cloud instance you use or maintain, without compromising on performance?"
The concept of easystack yaml files is probably a good idea but an entirely new concept to very infrequent users like me (just read about it today) . Is there not a way to autogenerate a yaml file in eb with default settings if i want to just submit a single easyconfig?
Running the install_cuda_host_injections.sh
as root doesn't work, since EasyBuild doesn't allow it:
{EESSI 2023.06}
15:03 login-gpu1.liza.surf.nl:/tmp
root# /cvmfs/software.eessi.io/versions/${EESSI_VERSION}/scripts/gpu_support/nvidia/install_cuda_host_injections.sh -c 12.3.0 --accept-cuda-eula
Attempting to load an EasyBuild module to do actual install
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/main.py", line 755, in <module>
init_session_state, eb_go, cfg_settings = prepare_main()
^^^^^^^^^^^^^^
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/main.py", line 749, in prepare_main
eb_go, cfg_settings = set_up_configuration(args=args, logfile=logfile, testing=testing)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/tools/options.py", line 1566, in set_up_configuration
check_root_usage(allow_use_as_root=options.allow_use_as_root_and_accept_consequences)
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/tools/options.py", line 1518, in check_root_usage
raise EasyBuildError("You seem to be running EasyBuild with root privileges which is not wise, "
easybuild.tools.build_log.EasyBuildError: "You seem to be running EasyBuild with root privileges which is not wise, so let's end this here."
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/main.py", line 755, in <module>
init_session_state, eb_go, cfg_settings = prepare_main()
^^^^^^^^^^^^^^
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/main.py", line 749, in prepare_main
eb_go, cfg_settings = set_up_configuration(args=args, logfile=logfile, testing=testing)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/tools/options.py", line 1566, in set_up_configuration
check_root_usage(allow_use_as_root=options.allow_use_as_root_and_accept_consequences)
File "/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/EasyBuild/4.8.2/lib/python3.11/site-packages/easybuild/tools/options.py", line 1518, in check_root_usage
raise EasyBuildError("You seem to be running EasyBuild with root privileges which is not wise, "
easybuild.tools.build_log.EasyBuildError: "You seem to be running EasyBuild with root privileges which is not wise, so let's end this here."
ERROR: The easyconfig CUDA-12.3.0.eb was not found in EasyBuild version:
This is EasyBuild 4.8.2 (framework: 4.8.2, easyblocks: 4.8.2) on host login-gpu1.liza.surf.nl.
You either need to give a different version of CUDA to install _or_
use a different version of EasyBuild for the installation.
The versions of available with the current eb command are:
Since this script is actually not unlikely to be run as root, maybe we should add the possibility of passing an --as-root
argument, that then sets the --allow-use-as-root-and-accept-consequences
argument to EasyBuild?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.