mctools / simplebuild-dgcode Goto Github PK

sb_scanutilstests_testskeleton fails because they somehow trigger #11

A few ESS related utils exists in dgcode, and we might as well move them elsewhere.

We also need to augment the docs for ESS based users, by having a landing page in the intranet with a list of where to find and clone internal project repos, etc.

Git cloning from private repos fails in dependabot triggered workflows, as explained here:

https://docs.github.com/en/code-security/dependabot/working-with-dependabot/automating-dependabot-with-github-actions#accessing-secrets

First of all, we should add a proper check to the relevant workflows to verify that the token is nonempty.

Secondly, we can either just live with this, or we can add the same tokens also as an identically named dependabot secret.

Now simple-build-system is on conda-forge, we should add simple-build-dgcode as well.

For instance, sb_g4utils_geodisplay --g4viewer dies with:

G4Launcher:: Applying command: /vis/open OGLSXm
parameter value (OGLSXm) is not listed in the candidate List.
  Candidates are: ASCIITree ATree DAWNFILE G4HepRepFile HepRepFile RayTracer VRML2FILE gMocrenFile TOOLSSG_OFFSCREEN TSG_OFFSCREEN TOOLSSG_OFFSCREEN TSG_OFFSCREEN TSG_FILE OpenGLImmediateQt OGLIQt OGLI OpenGLStoredQt OGLSQt OGL OGLS OpenGLImmediateX OGLIX OpenGLStoredX OGLSX RayTracerX TOOLSSG_X11_GLES TSG_X11_GLES TSGX11 TOOLSSG_QT_GLES TSG_QT_GLES TSGQt TSG

Grepping reveals many in-code links to the old confluence pages.

This is needed in order to transfer the exact bits from input to output, and avoid unpacking/packing of the unit vector.

Geant4 in conda versus standalone might have issues:

diff --git a/ESSBoronTestCell/EBTCScripts/scripts/testchain.log b/ESSBoronTestCell/EBTCScripts/scripts/testchain.log
index bec1cdf98..b541c59fd 100644
--- a/ESSBoronTestCell/EBTCScripts/scripts/testchain.log
+++ b/ESSBoronTestCell/EBTCScripts/scripts/testchain.log
@@ -70,13 +70,16 @@ GriffDataReader opened file ebtc.griff
     "G4VersionNumber" : "1103"
     "GriffMode" : "REDUCED"
     "RandEngine" : "NCrystalXoroshiroEngine"
+    "datadir/G4ABLADATA" : "G4ABLA3.1"
+    "datadir/G4ENSDFSTATEDATA" : "G4ENSDFSTATE2.3"
+    "datadir/G4INCLDATA" : "G4INCL1.0"
     "datadir/G4LEDATA" : "G4EMLOW8.0"
-    "datadir/G4LEVELGAMMADATA" : "PhotonEvaporation5.7"
+    "datadir/G4LEVELGAMMADATA" : "G4PhotonEvaporation5.7"
     "datadir/G4NEUTRONHPDATA" : "G4NDL4.6"
-    "datadir/G4NEUTRONXSDATA" : ""
+    "datadir/G4PARTICLEXSDATA" : "G4PARTICLEXS4.0"
     "datadir/G4PIIDATA" : "G4PII1.3"
-    "datadir/G4RADIOACTIVEDATA" : "RadioactiveDecay5.6"
-    "datadir/G4REALSURFACEDATA" : "RealSurface2.2"
+    "datadir/G4RADIOACTIVEDATA" : "G4RadioactiveDecay5.6"
+    "datadir/G4REALSURFACEDATA" : "G4RealSurface2.2"
     "datadir/G4SAIDXSDATA" : "G4SAIDDATA2.0"
     "genName" : "G4StdGenerators/ProfiledBeamGen"
     "geoName" : "G4GeoEBTC/GeoEBTC"
diff --git a/SingleCell/G4SimSingleCell/scripts/testchain.log b/SingleCell/G4SimSingleCell/scripts/testchain.log
index db71c2795..fba059199 100644
--- a/SingleCell/G4SimSingleCell/scripts/testchain.log
+++ b/SingleCell/G4SimSingleCell/scripts/testchain.log
@@ -88,13 +88,16 @@ GriffDataReader opened file sc.griff
     "G4VersionNumber" : "1103"
     "GriffMode" : "REDUCED"
     "RandEngine" : "NCrystalXoroshiroEngine"
+    "datadir/G4ABLADATA" : "G4ABLA3.1"
+    "datadir/G4ENSDFSTATEDATA" : "G4ENSDFSTATE2.3"
+    "datadir/G4INCLDATA" : "G4INCL1.0"
     "datadir/G4LEDATA" : "G4EMLOW8.0"
-    "datadir/G4LEVELGAMMADATA" : "PhotonEvaporation5.7"
+    "datadir/G4LEVELGAMMADATA" : "G4PhotonEvaporation5.7"
     "datadir/G4NEUTRONHPDATA" : "G4NDL4.6"
-    "datadir/G4NEUTRONXSDATA" : ""
+    "datadir/G4PARTICLEXSDATA" : "G4PARTICLEXS4.0"
     "datadir/G4PIIDATA" : "G4PII1.3"
-    "datadir/G4RADIOACTIVEDATA" : "RadioactiveDecay5.6"
-    "datadir/G4REALSURFACEDATA" : "RealSurface2.2"
+    "datadir/G4RADIOACTIVEDATA" : "G4RadioactiveDecay5.6"
+    "datadir/G4REALSURFACEDATA" : "G4RealSurface2.2"
     "datadir/G4SAIDXSDATA" : "G4SAIDDATA2.0"
     "genName" : "G4StdGenerators/FlexGen"
     "geoName" : "G4GeoSingleCell/GeoB10SingleCell"

(old issue from 2016, transferred from old system @ ESS, not confirmed in new system)

First expand a volume and then one of several sub-volumes to show both some daughters and grand-daughters at the same time. Now, trigger contraction of the original volume by shift-middle-clicking on a daughter does hide the daughters, but leaves grand-daughters visible.

NamedMaterialMgr should return const material pointers because otherwise the internal database might get corrupted.

If running without matplotlib, a few handful(s) of tests fail. This should be handled properly with a dedicated extdep.

We should double-check that everything works nicely on osx-arm64.

With C++11/pybind11 we should be able to use std::shared_ptr instead.

After upgrading to XCode 14.3 (requires macOS Ventura 13.0 or later), I get a lot of compillation errors (warnings treated as errors) saying "A function declaration without a prototype is deprecated in all versions of C".
This is because "C2x will be removing support for functions with identifier lists, and will be changing the behavior of prototypeless functions with empty parentheses to match the behavior of C++" (from link)

A lot of functions in MiniZLib use the now unsupported identifier lists (a part of K&R-style function definition syntax). The errors/warnings could be silenced by using DG_EXTRA_CFLAGS=-Wno-strict-prototypes compillation flag (as suggested in madler/zlib#633 ), but for us it would be better to change the function definitions:

int foo(s, f)
  char* s;
  float f;
{
  return 5;
}

to

int foo(char* s, float f)
{
  return 5;
}

Also, for similar reason, empty parameter lists should be given an explicit parameter of void:

int foo()
{
  return 5;
}

to

int foo(void)
{
  return 5;
}

Once docs are "final" we should make should to add make linkcheck and nitpicky checks in CI (cf. mctools/simplebuild#11).

We should make sure to once in a while exercise our code on the large amount of packages in the internal ESS repos.

As evidenced in various warnings and this local uncommitted hack, the PyAna/extras tex font settings might need updating:

--- a/packages/Framework/PyAnalysis/PyAna/python/extras.py
+++ b/packages/Framework/PyAnalysis/PyAna/python/extras.py
@@ -59,10 +59,25 @@ def floatToScientificLatex(x):
 
 def enable_tex_fonts():
     import matplotlib
-    matplotlib.rcParams.update({'text.usetex':1,
-                                'font.family':'serif',
-                                'font.serif':['cmr10','cm'],
-                                'text.latex.preamble' : r'\usepackage{amsmath}'})#LaTeX fonts
+
+    matplotlib.rcParams.update({
+        # Use mathtext, not LaTeX
+        'text.usetex': False,
+        # Use the Computer modern font
+        'font.family': 'serif',
+        'font.serif': 'cmr10',
+        'mathtext.fontset': 'cm',
+        # Use ASCII minus
+        'axes.unicode_minus': False,
+        #cmr10 needs axis formatter to use mathtext:
+        'axes.formatter.use_mathtext': True,
+    })
+    #
+    #matplotlib.rcParams.update({'text.usetex':1,
+    #                            'font.family':'serif',
+    #                            'font.serif':['cmr10','cm'],
+    #                            'text.latex.preamble' : r'\usepackage{amsmath}'})#LaTeX fonts
+

We should cross-check for our dependencies that all of them (+cif2hkl to be nice) are enabled for M1 in this file:

https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/migrations/osx_arm64.txt

If not, we need a PR.

This should not give an error: "weight > 0 and (uz>0 or uy>0)". But it does: "unknown function "and". Only an issue when using python-like boolean operators, fine with "&&" and "||".

Long ago a lot of std::exit calls were used, which should mostly be replaced with proper exceptions.

MCPL files containing more that 2^31 events are difficult to processes due to the skip_events parameter of the MCPL generator being an integer.

The issue occurs when I run a McStas simulation to produce an MCPL file - containing neutrons scattered on a sample - that I want to use a source of particles for the subsequent Geant4 simulation of the detection process. I use the ess_dmscutils_submit tool to submit a lot of simulation jobs with 1e5 events in each, using the skip_events parameter in accordance with the job id, but that stops working when it reaches the 2^31 integer limit.

My current solution is splitting the large MCPL files, and processing them separately, but it would be nice to avoid that extra manual step (which is also a potential source of errors).

• 1) intro: “Most importantly, dgcode includes functionality for easy seting up and launching of Geant4 simulation jobs based on configurable geometry and generator modules(...)”

• Could we use “Particle generators” instead of “generator modules” here? (unless generator module is a Geant4 term that we expect the readers to understand even in the introduction.)

• The same for “Generators are likewise (...)“, in Features

• 2) newsimproject: “An analysis program which can be invoked via the command sb_tricorder_ana and which is used to run on the Griff files and which will result in a SimpleHists file being created with analysis histograms.”

• The ’and which will results in a (...)” part confusing refers to the ’run on Griff files', while the first ‘and which’ refers to the ana file itself.

• “being created with analysis histograms” does this mean “consisting of analysis histograms”?

• 3) griff: Extend the “How to enable Griff output” section with info about the multiple griff files created when using -jN (mystuff.0.griff to mystuff.[N-1].griff). (The tip in Analysing Griff files / Basic approach assumes this knowledge later on “Multiple input files can be chained together by specifying them with a wildcard, e.g. mysim.*.griff” ). Maybe put it after “Which changes the default to be to write Griff output in REDUCED mode to a file named mystuff.griff.”

• 4) scans: In the example, sample_radius_mm is changed, but in the text it says “The first plot will vary a detector thickness” (it is a bit confusing for me, even if the sample is the detector in this context(?))

• 5) simplehists: The ‘Example: Python analysis of histograms’ script at the bottom of the page is supposed to be a "small example of how one can get data out in formats ready to input to the various pylab plotting routines"; which is probably true, but I don't see the need for the “import pylab as pl” line.

• 6) mcpl: “The sb_mcplextra_filterview command can be used to produce a new MCPL file from an existing one, containing only a subset of the original particles. (...)“.

• The filterview script is introduced instead of sb_mcplextra_filterfile, that is actually used in the following examples.

• This is the only mentioning of the filterview script/command, so I guess a paragraph dedicated to it is missing.

• 7) In newsimproject, it is stated that “For the sake of the following discussions, we will assume you are creating a completely fresh working directory in which you wish to work (this might of course not be the case).” but a little bit later the user is reassured that “Of course, if you are working with code in a shared repository (e.g. at GitHub), nothing has been added to the remote repository at this point. (...)” It is a bit controversial for me. I think this should be restructured a bit to separate the part of creating a bundle (unless working in an existing one), and creating a new package. Restructure part between:
  “For the sake of the following discussions” AND
  “Rather, you are now expected to modify the files as appropriate for your actual project before finally git add’ing and committing and pushing them to the central repository server.”

• 8 ) particlegenerators: “The is used to select which histogram from the file to use, and the optional part can be specified either as a name or a number.” For the ‘number’ option, you have to know the default Geant4 unit and provide the conversion factor? (Should we point at some sort of default Geant4 units list?)

• 9) particlegenerators: “Note that in addition to the actual bin-contents of the histogram, the sampler also considers the min/max statistics as well as any underflow/overflow content.” How does it consider the underflow/overflow content?

• 10) sbparticlegen.rst “As important for a Geant4 simulation as geometry, materials and physics list, is the initial set of particles to be “let loose” in the simulation each event” This might be only confusing for my ear, as I’m not used to the Geant4 meaning of an “event”, can we maybe put in an extra word somewhere? (it is probalbly not important)

The followings are suggested additions:

• 11) install: Maybe add a sentence after “conda env create -f conda_dgcode.yml”, that depending on the internet connection, it might take several minutes and the resulting conda environment will take up ~3.8G(?) disk space.

• 12) newsimproject: simplebuild.cfg file is first mentioned here as “As written, new packages and files have been created for you in your local directory (defaulting to the directory of your master simplebuild.cfg file)“.
  We might want to link to simplebuild documentation (sbdotcfg or automatic-creation) a little bit earlier when asking the user to invoke sb --init dgcode, mentioning that this command will create the (master) simplebuild.cfg file (just to avoid confusion).

• 13) particlegenerators: At the end of “Generating from Griff files” (after the part about “(...)Griff file was created with a certain filter to select the particles (...)“) Maybe I’d add that together with the visualisation tool (--aim / --data), it can be a very powerful tool for debugging / understanding unexpected events, e.g. by finding unexpected particle paths.

It should somehow be possible to register custom materials with the NamedMaterials mgr.

For historical reasons, a lot of the docs still refer to "pylab", which is not really appropriate, since most of the time we simply mean either numpy or matplotlib.

use g4exprparser to add convenient --print and --plot options, for instance:

sb_xxx_sim --print="step.pre.ekin/meV:step,trk.is_primary where trk.is_neutron and step.volname=='Sample'"

To get on-the-fly lines printed from the simulation for neutrons stepping through volumes named Sample of two variables: kinetic energy in meV and a number indication that the particle is primary (1) or secondary (0).

We could also extract .shist files in this manner!