xenonnt / fuse Goto Github PK

We have to check the strax time information of the refactored code. At the moment the Geant4 events are separated by 1s plus an additional shift for the chunk boundaries. This large spacing allows us to keep the raw_records inside a predefined chunk. We need to do this part more carefully and add back some flexibility like variable event rates etc.

I needed to come up with a lot of names for plugins, data kinds and so on. We should rename some of them to follow python programming recommendations and make it easier to read and understand.

By convention, we use snake_case for config names and CamelCase for class names only.

We should add the functionality to set the random seed manually.

The Geant4 file to read is given in the strax config. The strax runname has no relation to the used Geant4 file at the moment. We should come up with a clever naming convention.

fuse plugins have an if case to handle empty chunks and return empty output data. There is an strax function to basically the same and we can use it to improve the code: https://github.com/AxFoundation/strax/blob/bf0c16b8e89f14b1bd7ee482a20b5e3c7decd20a/strax/plugins/plugin.py#L310

There is some duplicate code, e.g the make_map function. We should collect them in a single .py file and import where needed.

After-pulses need to be added to XeSim. As the overall (linear) structure of XeSim is quite different from the loop structure in WFSim we need to come up with a good solution.

We should provide some usage examples as Jupyter notebooks. These notebooks should outline the basic usage and explain the concepts of fuse simulations.

Add the S2 pattern map modification to fuse.

We have the option in strax to track certain config options in the lineage. For now, (almost?) all of them are not tracked. We need to evaluate which ones are so important that they need to be tracked.

We should aim for a good documentation of fuse both for the code itself and setting up a Read the Docs site.

As pointed out by @HenningSE, some of the unused plugin dependencies in the WFSim->fuse transition went under the radar in the review of #72. Before the next release, make sure to clean up the code of these dependencies.

Related to this: let's also clean up the deprecated options in the config *json files.

Some plugins use straxen to handle maps. I think we should try to remove the dependency on straxen so that fuse can be used on local machines without a complete straxen installation.

See #41, where @HenningSE propagated an epix tag to filter only certain events (clean-ish NR energy deposits). We might want to give this a proper structure for other cases (multi-site calibrations, er_only, etc.).

Neutron Veto simulations are not included in the refactored code yet.

We should use a binomial process to determine how many electrons reach the liquid gas interface instead of just a simple multiplication.

Not very important but the name "fuse" will become an issue if you ever want to make the package public.
FUSE is a linux module for loading user space filesystems. So any search will bring that up way before your package both on readthedocs and pypi.

Right now event times are set by fuse based on a source rate. We can add the option for csv input simulation to use event and interaction times from the csv file. This way the user has some more flexibility with their simulations.

@HenningSE enabled the fast-sim skeleton in #35, now it's time to add the actual physical plugins (work in progress in XENONnT/epix#27) for the effective simulation of S1s and S2s.

(For neutron veto we will default to hitlet simulator, hence no full waveform handling. Issue is alive: #2)

We need to make sure that some plugins are not only separated by individual physical processes, but also by complexity, so that a subset of the total plugins can as "effective simulator" without the need of producing waveforms.

The strax framework and plugin handling makes large parts of our existing mc_chain code obsolete. We need to change these scripts to be able to run fuse on OSG.

Not directly a fuse issue, but currently we have this piece of code to prevent crashes:

Strax has a build in plugin timeout of 80 seconds. If no data arrives in this time, it crashes. If we just ask for raw_records the simulation of a single chunk up to raw_records can easily take longer than 80 seconds. We need to increase the timeout for our plugins.

With nestpy version 2.0.0 event_quanta = nc.GetQuanta(y) can not be used in a numpy vectorized function. There is no error message but the function runs basically forever. When going back to nestpy 1.4.3 everything works fine.

I left out certain parts of the code that are not frequently used (e.g. luminescence_timings_garfield). We need to add these functions to the refactored code.

We currently do not set the first interaction in LXe to t=0, such that matching, e.g., TPC and nVeto entries can be done directly when importing Geant4 files. However, for some isotopes, such as U238, we may be approaching the np.int64 maximum of ~9e18 when handling the time of interaction in Geant4.

One can use progress_bar = True in st.make() to get a visualization how long the simulation step runs. As far as I can tell, it uses the time of the events and simulations with delayed interactions can lead to a greatly overestimated simulation time. We can try to adapt it to fuse or come up with a better progress visualization.

We need a context to register all the plugins. We need to set up a mc_chain context.

I tried to move all needed configuration variables (e.g. electron lifetime,...) into the strax config options. The default values are then loaded from a hardcoded fax-config file. We need to resolve this.

Most of the XeSim code is copy and paste from epix and WFSim with changes applied only where necessary to run in the plugin structure of strax. There are for sure many places where the code can be optimized exploiting the new structure.

We should fix this issue from epix in fuse: XENONnT/epix#69

We need to add automatic tests for each PR, which will also serve to check the resource handling (see #6) with dummy inputs.

For simulations of really small signals (going towards single photons) it could be nice to have dark counts in the simulation. @HenningSE told me they are not considered yet.

At the moment the output_plugin translates the micro_physics simulation (epix) output to the required format by WFSim. As we are now refactoring both packages I think it makes sense to replace the output_plugin by a strax merger plugin and adapt the S1PhotonPropagation, ElectronDrift and ElectronExtraction to the new format. There is no more reason to have photons and electrons in separate rows and can just use columns.

Original problem reported in wfsim at here. However, such problem is reproducible also in fuse.

We should evaluate if we can speedup the code removing unnecessary loops or using numba. Especially the pmt_response_and_daq and S2_photon_distributions_and_timing plugins could greatly benefit from higher speeds.

In WFSim instructions are grouped based on time. These groups are then simulated at once. In fuse this is not easily possible. Here the S1 and S2 photons are grouped and then pulses are computed. As this logic differs from WFSim we need to check if it makes sense.

Some functions are imported directly from WFSim and epix. We need to move these functions into XeSim.

strax gives us the option to decide which plugin-output we want to save. The four possible cases are given here. As some plugins are fast to calculate and steps can be easily repeated with the deterministic random seed we do not need to save the output of all plugins.

We need to figure out for each plugin if we need to save the output or can calculate it on the fly.

epix wfsim have the functionality to simulate with csv files as input. We need to add plugins to bring this to XeSim.

The fix from XENONnT/epix#75 may also need to be put into fuse.

for modern python packages it is convention to have the dependencies tracked in one of these formats.

tag_cluster_by is set as a boolean in the fuse config. The if case below will be false all the time.

Truth data that is produced by WFSim is not included in fuse. We need to evaluate which truth information we need as we are able to access intermediate results due to the plugin structure.

The current state of XeSim needs to have the Private XENONnT Auxiliary Files repo to load all needed maps etc. We should find a better way to handle it.

We should add comments to the provided columns of fuse plugins. This would improve the user friendliness of fuse.