djgroen / fabsim3 Goto Github PK

in base/test_fabsim.py

• Propose a way for FabSim3 to interact with the diagnostic tool (Eye of VECMA).
• Propose a design for EoV.
• Assemble a team of rock star developers and trial users.
• Spin off EoV in a new repo, with separate issues/tasks.

• Contemplate how verification / validation and UQ fit into the tool, and formulate ideas on this.

This is a follow-up for issue #6.

What is the desired/expected behavior when using a YML like:

b: 0
c: $a
a: $b 

?

and

d:$e
e:$d

?

Please, let me know what kind of examples you would like us to add to FabSim3, to help new developers pick it up. We can add them as TODO items as follows:

• Create barebone example plugin, together with repo etc.

• Propose a way to shield domain specific variables from the global env in FabSim.
• Propose a way to limit/prevent accidental modification of core FabSim routines.
• Propose a way to structure domain-specific functionalities. Is sub dirs enough, or do we want these to be in separate repos, with FabSim included as a library?
• Contemplate a potential plug-in system for apps.
• Contemplate how verification / validation and UQ fit into the tool, and formulate ideas on this.

This task is intended to facilitate Katya's requirements. Tasks include:

• Create placeholder plugin for FabGromacs.
• Add it to the accepted plugin list for FabSim3
• Remove FabGromacs (funcs will be in FabMD).
• Add support for Gromacs executions of test problems using the plugin (by updating the gromacs template and adding test initial conditions).
• Add support for Gromacs ensemble executions using the run_ensemble() functionality (a placeholder is in place now).
• Enable @katya2828 to install, use, modify the plugin.
• Assign a maintainer for this plugin (perhaps @katya2828 and @raar1 together?)

From Vytas:

Import everything from a plugin is a bad idea.
I would suggest to change the plugin "API" so that each plugin module includes a variable called "exported_tasks" or something along those lines. Just with the names that this module exports. This way we can import only those.

Vytas raises a valid concern, and we need to trade off ease of integration against risk of intrusion into the main code. We also need to determine what is technically possible in terms of doing partial imports, and what the merits and flaws are for each approach.

Please do put your thoughts and ideas on this topic here.

Guys... I think it is time we think of designing one. Suggestions are welcome for logos of either type (I think we should have both ;)).

Initial tests have been written as part of issue #9 .

• As part of this task, make sure that at least some of the existing unit tests are working using the PyTest framework.
• Optional: make all the unit tests working.
• Optional: make the functional tests working.

Here is my simple submit script for prometheus:
prometheus.cyfronet.pl
submit.txt

I use my own modified version of lammps which is why the execute file is in one of my directories

• Document how other tools relate to FabSim, and what value they will/should add.
• Create a test integration with MUSCLE2.
• Create a test integration with QCG.

After installation of FabFlee plugin, FabFlee.py required local plugin location throughout the script. Providing local plugin location in each instance displays a long and complicated script.
For reference, lines 59-62 in FabFlee.py are long versions with local plugin location. It would be useful if FabSim3 added environment variable for local plugin location.

Define an ensemble execution mechanism using:

fab run_ensemble

This should iterate over:

• a range of input files residing in a Sweep directory.
• have a demonstrable working examples in FabDummy and FabMD.

when installing fabric3 (by setup.py) on a fresh python 3.7.0 build, it lakes the "setuptools".
(Also pip would require the setuptools when installed via python.py)
The setuptools themself require the module "_ctypes" to be installed, which is a known issue in python3.7 (Issue31652 https://bugs.python.org/issue31652 ) Depending on the operating system, this can be fixed, but maybe it's best to not use python 3.7.0 for the moment.

Formulate a very simple clay/graphene-polymer Acceptance Use Case (or Example) to be satisfied using FabSim 3.

This is because importlib in earlier versions of Python3 does not support ModuleNotFoundError.

I will now try to re-engineer this...

• Propose a first formal functional test suite for FabSim3.
• Propose a first unit test suite for FabSim3.
• Set up a continuous integration service @ UCL.

When running, e.g. "fab prometheus monitor", I keep getting the following error:

Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/fabric/main.py", line 763, in main
*args, **kwargs
File "/usr/local/lib/python3.4/dist-packages/fabric/tasks.py", line 387, in execute
multiprocessing
File "/usr/local/lib/python3.4/dist-packages/fabric/tasks.py", line 277, in _execute
return task.run(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/fabric/tasks.py", line 174, in run
return self.wrapped(*args, **kwargs)
File "/net/tesla/srv/home/robinr/FabSim3/base/manage_remote_job.py", line 18, in monitor
time.sleep(120)
NameError: name 'time' is not defined

This is easily fixed by adding "import time" to FabSim3/base/manage_remote_job.py. What I don't understand is why is this already working on other people's installs, but not on mine?

If no one has any objections, I'll make the above change to manage_remote_job.py

FabDummy testing commands

• fab localhost dummy:dummy_test
• fab localhost dummy_ensemble

show the following errors:

   Warning: Command(s) not found:
       dummy

   Warning: Command(s) not found:
       dummy_ensemble

In order to run a campaign where, at the end, we gather all the results produced by the simulation of each replica using fabsim, we need to know when the execution of all the replica is finished.

Is there some sort of 'status' command implemented in FabSim to check the status of all the jobs, or even the status of the execution of the whole campaign at once?

Formulate a very simple HMC Acceptance Use Case (or Example) to be satisfied using FabSim 3.

• yml configuration files.
• .py automation scripts.
• templates for PBS headers and code executions on machines.
• configuration directories.
• FabSim env variables

Take the deploy/NanoMD folder as a start.

Let @djgroen know when it exists, so he can integrate it.

Look for directories to run in the order
/FabSim3/plugins/plugin-name/config_files
/FabSim3/config_files

Function has derek home dir hardcoded,
When I changed it to my dir: grep gives exit code 1 if the history has no fab commands in it. In my case I think this happened because my .bash_history was tracking a different terminal or for some reason did not have recent commands in it
This has caused the first lammps example given in the install.md to break

When executing a fab command, i.e fab eagle lammps_ensemble:lammps_ensemble_execution,input_name_in_config="in.lammps", if the user is asked its password and enters it, the password is logged in an entry of the env fabric dictionnary called passwords.

The entry is such as: passwords : {login1 : password1, login2 : password2, ...}

• fab example_machine poll:lines=10,filename=
  (Continually shows the last 10 lines of a log file, updating it periodically. -1 causes it to dump all the log contents)

• fab example_machine poll_and_detect:pattern=<regex_pattern>,filename=
  (Will quit if a certain pattern is detected in the log file.)

We have converged on the fact that the replicas configurations should be provided in defined set of directory (sweep, etc...). Should we converge also on an output format for the system response quantity (SRQ) of interest for each replica?

In the ensemble_execution AUC, we suggest that each simulation prints the SRQ in a text file. Then, the SRQs for each replicas are gathered in a CSV file. The CSV file is then passed to the VVUQ toolkit for further processing (see ensemble_uq AUC).

Such input formatting could be defined as a standard for all VVUQ preprocessing toolkits output. Similarly, the outputs of the execution toolkits should be standardized to fit the input of the post-processing toolkits.

Compare Quantities of Interest. This should not be started before August.

• Develop first version of FabFlee3.
• Check correct workings through testing by cvanhille.

How should we handle invalid/missing command line arguments, It would be useful to give the user some information on the arguments commands require. We should have a standardised way to do this.

For the minute in FabMD.lammps_ensemble I have the lines
if 'input_name_in_config' not in ensemble_args: raise RuntimeError('Must declare input_name_in_config: the generic name which sweep directory files will be changed to')
We may not want the user to see the entire stack trace however.
Any other suggestions welcome

Either change functionality to make cross-platform or detail the changes needed in installation

This is currently under discussion in the UCL-CCS/EasyVVUQ repository. The outcome of that discussion is directly relevant for the work here, so should be summarized in this issue :).

This is not clearly explained at the moment, and some of the mechanisms are rather clunky and for sure poorly documented...

Preliminary:

• Test installation of FabSim3.
• Test how fab localhost install_plugin:FabDummy works.

Goal: bolster the plugin system at the very least so that:

• Plugins can be removed as well as installed.
• (if the current plugin system is retained:) Ensure that the correct imports for the plugins are maintained in fabfile.py (or perhaps an imported and untracked fabfile_user.py) whenever a new version of FabSim3 is obtained from the repository.

The file linking to the plugins is currently fabfile.py in the main directory. The code implementing the installation of plugins can be found in base/setup_fabsim.py, in the functions setup_fabsim and install_plugin.

Vytautas, if you need any help or advice, or would like to modify the underlying design, please don't hesitate to leave a comment here and add Robin and/or me to the assigned people on this ticket. One strong preference we do have is that plugins are retained in separate GitHub repositories, as that greatly eases development.

• Propose a way to shield domain specific variables from the global env in FabSim.

This is a follow-up of #6.

Currently, setting up the results directory (i.e. copying data and config files from the config directory) is done in the submission script. It might be more transparent to have this done when the fab host run_job is executed.
If all the rearranging and renaming of files was done by fab then we would have much more readable submission scripts. This has become a problem with enesemble submission where it is quite hard for a human to check the correct thing will happen. Is there any reason why these tasks should be handled by the submission script?

Currently, setting up the results directory (i.e. copying data and config files from the config directory) is done in the submission script. It might be more transparent to have this done when the fab host run_job is executed.
If all the rearranging and renaming of files was done by fab then we would have much more readable submission scripts. This has become a problem with enesemble submission where it is quite hard for a human to check the correct thing will happen.

We therefore need to move this functionality from the run script to the fabsim/base.

Provenance:

• propose simple design for how/when(!) to record past FabSim commands as part of a run.
• implement recording of FabSim commands used.
• store current github ID of FabSim code.

Ideally put it in the wiki of the repository, or as a document inside the repo.

• Automate installation of dependencies, if possible.
• Automate setup of passwordless access to localhost from localhost, if possible.

is it possible to introduce your own shell variables in the template files such as ./deploy/templates/lammps?
as I try to create my own version of that file with an internal variable to the script that I call odir for example, I get the following error at run time:

maxime@picasso:~/local_projects/FabSim3$ fab localhost lammps:lammps_tetrapox_opls_init300K,cores=1,wall_time=1:00:0
[maxime@None] Executing task 'localhost'
[maxime@localhost] Executing task 'lammps'
[maxime@localhost] Executing task 'put_configs'
[localhost] local: ssh -Y -p 22 maxime@localhost 'mkdir -p /home/maxime/FabSim3/config_files; mkdir -p /home/maxime/FabSim3/results; mkdir -p /home/maxime/FabSim3/scripts; mkdir -p /home/maxime/FabSim3/config_files/lammps_tetrapox_opls_init300K'
[maxime@localhost] rsync_project: rsync  -pthrvz  --rsh='ssh  -p 22  ' /home/maxime/local_projects/FabSim3/config_files/lammps_tetrapox_opls_init300K/ maxime@localhost:/home/maxime/FabSim3/config_files/lammps_tetrapox_opls_init300K
[localhost] local: rsync  -pthrvz  --rsh='ssh  -p 22  ' /home/maxime/local_projects/FabSim3/config_files/lammps_tetrapox_opls_init300K/ maxime@localhost:/home/maxime/FabSim3/config_files/lammps_tetrapox_opls_init300K
sending incremental file list

sent 200 bytes  received 12 bytes  424.00 bytes/sec
total size is 61.96M  speedup is 292,267.99
Traceback (most recent call last):
  File "/home/maxime/source/miniconda3/lib/python3.6/site-packages/fabric/main.py", line 763, in main
    *args, **kwargs
  File "/home/maxime/source/miniconda3/lib/python3.6/site-packages/fabric/tasks.py", line 387, in execute
    multiprocessing
  File "/home/maxime/source/miniconda3/lib/python3.6/site-packages/fabric/tasks.py", line 277, in _execute
    return task.run(*args, **kwargs)
  File "/home/maxime/source/miniconda3/lib/python3.6/site-packages/fabric/tasks.py", line 174, in run
    return self.wrapped(*args, **kwargs)
  File "/home/maxime/local_projects/FabSim3/deploy/NanoMD/fabNanoMD.py", line 29, in lammps
    job(dict(script='lammps_epoxy', wall_time='0:15:0', memory='2G'),args)
  File "/home/maxime/local_projects/FabSim3/deploy/fab.py", line 299, in job
    env.job_script = script_templates(env.batch_header, env.script)
  File "/home/maxime/local_projects/FabSim3/deploy/templates.py", line 18, in script_templates
    result= "\n".join([script_template_content(name) for name in names]+commands)
  File "/home/maxime/local_projects/FabSim3/deploy/templates.py", line 18, in <listcomp>
    result= "\n".join([script_template_content(name) for name in names]+commands)
  File "/home/maxime/local_projects/FabSim3/deploy/templates.py", line 28, in script_template_content
    return template(source.read())
  File "/home/maxime/local_projects/FabSim3/deploy/templates.py", line 58, in template
    return Template(pattern).substitute(env)
  File "/home/maxime/source/miniconda3/lib/python3.6/string.py", line 126, in substitute
    return self.pattern.sub(convert, self.template)
  File "/home/maxime/source/miniconda3/lib/python3.6/string.py", line 119, in convert
    return str(mapping[named])
KeyError: 'odir'

and in my template file ./deploy/templates/lammps_epoxy the variable odir is well defined as for instance:

odir=test_matg0-flake-tetrapox-opls_repl1_tempt300_nsst10_dldx/

At the moment it seems easy to provide the varying input files of a ensemble simulation in the following architecture:

${FabMD_DIR}/config_files/lammps_ensemble_execution/SWEEP/varying.inputfile.lammps_${replica_number}

This is OK, if only file varies per replica simulation, would there be a way to provide mulitple varying input files? In a way of that sort:

${FabMD_DIR}/config_files/lammps_ensemble_execution/SWEEP/replica_${replica_number}/varying.inputfile.*