amrex-astro / maestroex Goto Github PK
View Code? Open in Web Editor NEWA C++ low Mach number stellar hydrodynamics code
Home Page: https://amrex-astro.github.io/MAESTROeX/
License: BSD 3-Clause "New" or "Revised" License
A C++ low Mach number stellar hydrodynamics code
Home Page: https://amrex-astro.github.io/MAESTROeX/
License: BSD 3-Clause "New" or "Revised" License
Currently wdconvect with inputs_3d_regression fails during the first MAC projection (too many multigrid V-cycles). (pure MPI)
If I back out amrex to 953c10b3c07a914973aea6e8c6bad69b9cbcd88d the wdconvect example runs correctly. No further time to debug it at the moment.
The advective reflux is currently disabled (so do_reflux=F by default and all inputs files). Instead, there is an AverageDownEdges call on the species/enthalpy fluxes. This should give the same result as refluxing (to machine precision), however it is not as efficient since averageDownFaces operates over all components and all faces (instead of flux registers which only update information ON C-F boundaries)
The docs have not been updated much to reflect the changes in MAESTROeX vs MAESTRO, so there are a few sections where they should be updated
There should be a routine like the writeSmallPlotFile
routine in Castro that only outputs a small subset of the variables
It would be useful to have a chk_deltat
parameter (similar to plot_deltat
) so checkpoints could be output at fixed simulation time intervals
Support for 1-d problems was added to original Maestro to allow us to model smallscale flames. In the original Bell et al. 2004 low Mach algorithm we used for flames, we never had 1-d -- we just ran flames in long, narrow 2-d domains if we were just interested in the laminar properties.
1-d doesn't really make sense for atmospheric flows.
So I would propose that we remove 1-d support from MAESTROeX. We can still do smallscale flames in 2-d if needed (and this is not any user's science priority at the moment).
This crashes with a backtrace in screening.
Augment each MFIter loop to include tilling. Castro and IAMR have this implemented already as a guide.
In wdconvect, inputs_3d_C.128.2levels generates gibberish after ~50 time steps. Compare this to original MAESTRO/wdconvect inputs_3d_C.128.2levels.fixed, which has the same fixed grid structure and works fine. I notice that during the first nodal projection, when looking at multigrid verbosity, the answers diverge.
When compiling/running on summit with pgi, an error message appears
ERROR: problem in the namelist
FORTRAN STOP
You can compile/run with gnu and there's no error. Or you can comment out the exit command and use pgi and it seems to run ok. We suspect this is an issue due to combining the inputs file and probin (namelist) into 1 file.
This is the script that writes the generates code in the header files and routine that describes the runtime parameters in _cpp_parameters
. Currently, it is having issues with small_plot_vars
, where the default value is set to a list of space-separated values. E.g. rather than
std::string Maestro::small_plot_vars = "rho p0 magvel";
it is generating
#ifdef magvel"
std::string Maestro::small_plot_vars = "rho;
#endif
Look at IAMR development branch commit a3f9f6d096992dd63cf4733632b8afafba5083e2. There is a bug in the 3d corner coupling terms for conservative differencing. This manifested itself in IAMR test cases as a constant density field not remaining constant under divergence-free flow. This needs to be fixed in MAESTRO and MAESTROeX.
We need reacting_bubble_2d/3d working with AMR. Must of the support is missing, such as regridding the base state and modifying tagging so it spans the domain.
With the 19.02 releases of AMReX, Microphysics, and MAESTROeX, compilation of extern.F90
in MAESTROeX/Exec/SCIENCE/wdconvect
using the Intel v19.0.0.117 compilers fails as follows:
extern.F90(153): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [VELPERT_AMPLITUDE]
namelist /probin/ velpert_amplitude
--------------------^
extern.F90(154): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [VELPERT_RADIUS]
namelist /probin/ velpert_radius
--------------------^
extern.F90(155): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [VELPERT_SCALE]
namelist /probin/ velpert_scale
--------------------^
extern.F90(156): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [VELPERT_STEEP]
namelist /probin/ velpert_steep
--------------------^
extern.F90(157): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [TAG_DENSITY_1]
namelist /probin/ tag_density_1
--------------------^
extern.F90(158): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [TAG_DENSITY_2]
namelist /probin/ tag_density_2
--------------------^
extern.F90(159): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [TAG_DENSITY_3]
namelist /probin/ tag_density_3
--------------------^
extern.F90(160): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [PARTICLE_TEMP_CUTOFF]
namelist /probin/ particle_temp_cutoff
--------------------^
extern.F90(161): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [PARTICLE_TPERT_THRESHOLD]
namelist /probin/ particle_tpert_threshold
--------------------^
extern.F90(162): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [USE_EOS_COULOMB]
namelist /probin/ use_eos_coulomb
--------------------^
extern.F90(163): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [EOS_INPUT_IS_CONSTANT]
namelist /probin/ eos_input_is_constant
--------------------^
extern.F90(164): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [EOS_TTOL]
namelist /probin/ eos_ttol
--------------------^
extern.F90(165): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [EOS_DTOL]
namelist /probin/ eos_dtol
--------------------^
extern.F90(166): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [SMALL_X]
namelist /probin/ small_x
--------------------^
extern.F90(167): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [USE_TABLES]
namelist /probin/ use_tables
--------------------^
extern.F90(168): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [USE_C12AG_DEBOER17]
namelist /probin/ use_c12ag_deboer17
--------------------^
extern.F90(169): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [ODE_MAX_STEPS]
namelist /probin/ ode_max_steps
--------------------^
extern.F90(170): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [SCALING_METHOD]
namelist /probin/ scaling_method
--------------------^
extern.F90(171): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [USE_TIMESTEP_ESTIMATOR]
namelist /probin/ use_timestep_estimator
--------------------^
extern.F90(172): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [ODE_SCALE_FLOOR]
namelist /probin/ ode_scale_floor
--------------------^
extern.F90(173): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [ODE_METHOD]
namelist /probin/ ode_method
--------------------^
extern.F90(174): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [SAFETY_FACTOR]
namelist /probin/ safety_factor
--------------------^
extern.F90(175): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [DO_CONSTANT_VOLUME_BURN]
namelist /probin/ do_constant_volume_burn
--------------------^
extern.F90(176): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [CALL_EOS_IN_RHS]
namelist /probin/ call_eos_in_rhs
--------------------^
extern.F90(177): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [DT_CRIT]
namelist /probin/ dT_crit
--------------------^
extern.F90(178): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [BURNING_MODE]
namelist /probin/ burning_mode
--------------------^
extern.F90(179): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [BURNING_MODE_FACTOR]
namelist /probin/ burning_mode_factor
--------------------^
extern.F90(180): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [INTEGRATE_TEMPERATURE]
namelist /probin/ integrate_temperature
--------------------^
extern.F90(181): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [INTEGRATE_ENERGY]
namelist /probin/ integrate_energy
--------------------^
extern.F90(182): error #6576: If a namelist-group-name has the PUBLIC attribute, no item in the namelist-group-object-list may have the PRIVATE attribute. [JACOBIAN]
namelist /probin/ jacobian
--------------------^
/tmp/ifortO5xWJ0.i90(527): catastrophic error: Too many errors, exiting
This error is new with Intel v19; it does not occur for any release of the v18 compilers.
It might make some sciencing easier if we had a proper 2-d axisymmetric solver.
The job_info
file is missing some information. In particular:
this can pretty much all be directly copied from Castro
Currently, for lots of the hydro routines (ppm
, mkutrans
, velpred
, slope
, make_edge_scal
) there are multiple versions for 1, 2 and 3 dimensions. It is likely that many of these can be combined into single, dimensionally agnostic routines, reducing the amount of repeated code and making them easier to maintain.
It might be useful to add a do_smallscale
feature and associated test problems that is analogous to the feature in the old MAESTRO code.
(Namely, setting rho_0 = rhoh_0 = 0, not evolving the background, and using div U = S constraint.)
Since it seems like an approachable first issue, I'll attempt this and then issue a pull request once it is ready.
Are there any thoughts on moving the current version of MaestroeX to take advantage of the Amr/AmrLevel classes (like IAMR) rather than the current implementation using AmrCore? Is there any obvious advantage to doing this?
Issue can be reproduced using inputs_3d_C.128.avgbase.dr
for uneven base state spacing, and inputs_3d_C.128.avgbase.dx
for even base state spacing. Both tests fail after approx. 20 time steps, near the base_cutoff_radius. Backtrace files show that error is due to either a failure to converge during MAC projection, or eos failure to compute enthalpy outside base_cutoff_radius in UpdateScal().
It would be useful for the plotfiles to include the Mach number
The plot variables in MAESTROeX should match those in MAESTRO as much as possible. Quite a few variables are missing. There are also "plot_X" logicals/integers in the _parameters that control whether things get plotted (and are not implemented).
We can use the new amrex functionality to get rid of the probin file completely, and instead have it just in the inputs file.
See https://ccse.lbl.gov/pub/RegressionTesting/MAESTROeX/
After merging "tiling" into "development", if you build with DEBUG=TRUE and run, these tests crash. If you build with DEBUG=FALSE they run to completion, so we may not have noticed this before.
We should do some scaling with MPI / MPI+OMP
Instead of manually adding problem-specific parametrers in a probdata.f90
, and then setting the initial value in probdata_init
, we should consider adding the parameters in an _parameters
file, that is automatically captured by the build system. This makes it easier to maintain and document.
This would require using the AMReX tensor solve since div(u) != 0.
We should have a routine in MaestroPlot.cpp
that outputs the gravitational acceleration
We have no examples with inflow / user-specified boundary conditions. We should port the mach_jet problem over to demonstrate / implement this functionality.
We should follow the lead of Castro and offload the reactions to GPUs
The docs for managing jobs (from the original Maestro) point to the old Titan job scripts path and don't clarify that the restart flag is now maestro.restart_file=[checkpoint file]
.
We should update this as well as add MAESTROeX specific job scripts.
The HPSS archiving script should also use the new C++ ftime in AMReX.
I'm testing these changes and will update the docs when I confirm they work.
I've been trying to run a two-level 3D calc with planar geometry but currently tells me that init_multilevel has not been implemented for this yet. Is there a work around for this (e.g. single chunk at finest level)?
unless there are any objections, we should enable ppm_trace_forces
by default
Currently, during initialization the plotfiles plt_InitData
, plt_after_DivuIter
and plt_after_InitProj
are output to the directory where the simulation is being run. It would make more sense for these plots to have the prefix maestro.plot_base_name
(rather than plt_
) so that e.g. they are saved to the same directory as the other plotfiles (if that is not the runtime directory).
Hello,
Im looking to develop some capability to model the dynamics of hot bubble motion in a stratified atmosphere, where the scale height is an important consideration, and the densities/temperatures (though not the pressures) are very different to the background. I suspect that MAESTRO has the capability to do so and so i intend to modify the code accordingly. My question is should i be looking at using the Fortran MAESTRO or the c++/Fortran MAESTROeX to do this? I note that not all tests work properly with MAESTROeX which used to work with MAESTRO. Are both codes in active development or is MAESTRO likely to be deprecated? Many thanks for your time.
The master branch still seems to have a dependency on some things from F_BaseLib, so when I try compiling one of the TEST_PROBLEMS, I get this error. It seems fine on the development branch
make: *** No rule to make target '/home/jt/data/amrex/Src/F_BaseLib/FParallelMG.mak'. Stop.
The original MAESTRO had unit tests under the Exec/
directory. These had their own drivers. We should port them to MAESTROeX
It is useful for scaling studies to have the tile size in the job_info file along with the number of MPI tasks and OMP threads.
we should do some profiling to find "hot spots" for single processor
In Maestro_F.H there are several instances of NULL arguments (for the mask) to fortran. This does not work with all fortran compilers according to Weiqun; I'm not sure of a good solution right now.
Note that this also breaks compilation when you do "make typecheck" to check for inconsistent argument lists between Maestro_F.H and the corresponding F90 subroutines.
We need to add a few tests to the regression suite for OpenMP/Tiling.
This would make the installation process easier for new users.
The default value of burning_cutoff_density
is 3.e6 (g/cc). This reflects Maestro's origin as a code for modeling burning in WDs, but for other applications users may be unaware of this cutoff. We should have this off by default (set it to 0.0 as in Castro). This point of this parameter is optimization, just to skip the burn in regions where it shouldn't matter.
One thing we talked about in the past was whether we should evolve a momentum equation in Maestro instead of a velocity equation. Maybe this is something to think about as we rethink the algorithm.
we should store the contents of the &extern
namelist in the job_info file.
We should port the ppm_trace_forces
functionality from Maestro and turn it on be default. This seems to help some problems a lot, like test_stability
Add a regression test on battra for double_bubble, incomp_shear_jet, rt, test_convect in 2d (and 3d if available)
Similar to what we do in the checkpoint file, we should write out the base state to the plotfile.
I just realised that MAESTROeX has no license currently. Can we just copy from the license file in Castro?
We might use deltagamma for a model for slow rotation, so we should recover that from old Maestro
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.