[INSTALL] MAPL 2.34.0 about hpc-stack HOT 21 CLOSED

@natalie-perlin We need to update following libraries along with MAPL 2.34.0.

ESMF: 8.4.1b07
HDF5: 1.14.0
netcdf: 4.9.1

would you please let us know when you have those libraries available on NOAA R&D platform (e.g. hera) for us to test?

from hpc-stack.

from hpc-stack.

@junwang-noaa I can quickly add an installation together with ESMF/8.4.0, and waiting for UFS test to get together into the code delivery. what do you think?

from hpc-stack.

Excellent! Also let me know if you have an installation on R&D platform, I can do some testing

from hpc-stack.

from hpc-stack.

@Hang-Lei-NOAA I asked our NASA collaborators, they said the dependent libraries are specified in the release note. Please see the notes below to turn off all the extra libraries. We only need gFTL, gFTL_shared and ESMF 8.4.0. Thanks

https://github.com/GEOS-ESM/MAPL/releases/tag/v2.34.0

The libraries this version of MAPL is currently tested with are below.

Baselibs 7.7.0
netcdf-c 4.9.0 (Needed for quantize support, but not required for all of MAPL)
netcdf-fortran 4.6.0 (Needed for quantize support, but not required for all of MAPL)
ESMF 8.4.0 (Required)
GFE 1.8.0
gFTL 1.8.1
gFTL-shared 1.5.0
pFUnit 4.6.1 (optional)
fArgParse 1.4.1 (if -DBUILD_WITH_FARGPARSE=YES, default=YES)
pFlogger 1.9.1 (if -DBUILD_WITH_PFLOGGER=YES, default=YES)
yaFyaml 1.0.4 (if -DUSE_EXTDATA2G=YES, default=YES)
FLAP geos/v1.10.0 (if -DBUILD_WITH_FLAP=YES, default=YES)

from hpc-stack.

from hpc-stack.

@natalie-perlin follow up of mapl-2.34.0 is on our to-do list. I will check on gaea.

from hpc-stack.

@junwang-noaa , @jkbk2004 -
I could start installing.
We have the following dependencies for MAPL that are currently specified explicitly during building of hpc-stacks, as shown below. The versions that are currently available are given in a regular font, or bolditalic if they need to be installed

MAPL/2.34.0
esmf/8.3.0b09 esmf/8.5.0 esmf/8.4.0
sqlite/3.36.0
esma_cmake/v3.4.3
cmakemodules/v1.2.0
ecbuild/ecmwf-3.6.1
gftl-shared/v1.5.0
yafyaml/v0.5.1 yafyaml/v1.0.4
hdf5/1.10.6
netcdf/4.7.4
netcdf-c/4.9.0
netcdf-fortran/4.6.0
pFlogger 1.9.1
gFTL 1.8.1
GFE 1.8.0
fArgParse 1.4.1
pFUnit 4.6.1

We also have geos/3.8.1 (from https://download.osgeo.org/geos/), but it might be different geos from the list (FLAP geos?)

Please let me know of any specifics with these packages.

from hpc-stack.

@Hang-Lei-NOAA @junwang-noaa @jkbk2004 -
Some updates so far:

• netcdf/4.9.0 with hdf5/1.14.0 builds successfully and is available.
• netcdf/4.9.1 is not building right away (some compilation errors) with neither intel nor gnu compilers; will need further debugging. Please let me know if netcdf/4.9.0 is sufficient for now!
• esmf/8.4.1b07 with netcdf/4.9.0 + hdf5/1.14.0 is built
• mapl/2.34.0 is not building, missing packages reported during configuration. Pending further debugging and updates.

These builds (hdf5/1.14.0, netcdf-c/4.9.0, netcdf-fortran/4.6.0, esmf/8.4.1b07) are done for the following platforms and compilers:

Hera:
/scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/intel-2022.1.2
/scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-9.2/
/scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-10.2/

Orion: (currently building) /work/noaa/epic-ps/role-epic-ps/hpc-stack/libs/intel-2022.1.2

from hpc-stack.

@natalie-perlin @Hang-Lei-NOAA FYI. GOCART developers are working on MAPL 2.35 release, we need to go to that version in order to get GOCART to work in UFS WM.

from hpc-stack.

from hpc-stack.

@junwang-noaa @Hang-Lei-NOAA - thank you for updated Re: netcdf/4.9.1 and MAPL/2.35 needed!
I will continue with installation of netcdf/4.9.1 and use Hang's hpc-stack installation on Hera as an example.

from hpc-stack.

@Hang-Lei-NOAA -
I was able to locate only 4.7.4 and 4.9.0 versions of netcdf modules, but not 4.9.1, in /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack-gfsv16 that you mentioned. Please let me know if you have netcdf/4.9.1 elsewhere by any chance.
I'll attempt to compile with "--disable-libxml2 --disable-byterange" for the meanwhile.
Thank you!

from hpc-stack.

@natalie-perlin Is it possible that we can have a MAPL 2.23.1. installed with
ESMF: 8.4.1b07
HDF5: 1.14.0
netcdf: 4.9.1
on hera so that we can test those version before MAPL 2.35 comes out? It helps us verify ESMF 8.4.1b07 for the ESMF 8.4.1 release.

from hpc-stack.

from hpc-stack.

@Hang-Lei-NOAA Thanks for testing, the 2.34.3 is an old version that does not have the changes yet.

from hpc-stack.

So far, the following combination is currently available on Hera:
hdf1.14.0-netcdf-c-4.9.1-netcdf-fortran-4.6.0-esmf-8.4.1b07-mapl-2.23.1
for Intel-2022.1.2 and gnu-9.2 compilers (so far):
/scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/intel-2022.1.2
/scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-9.2

UPDATE:
Also on Hera: /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-10.2
Orion: /work/noaa/epic-ps/role-epic-ps/hpc-stack/libs/intel-2022.1.2

Please note that these modules are within the default hpc-stack locations, so you when loading, explicit versions of each package needs to be provided, in order to fulfill the dependencies, as in the chain shown above.

from hpc-stack.

@junwang-noaa @Hang-Lei-NOAA
Updates for Hera installation with intel/2022.1.2 compiler.

Installed the target configuration:
hdf/1.14.0, netcdf-c/4.9.1, netcdf-fortran/4.6.0, fms/2022.04, pio/2.5.10, esmf-8.4.1, mapl/2.35.2

Additional updates and modules that [mostly] help to resolve mapl dependencies:
yafyaml/1.0.4, fargparse/1.4.1, pflogger/1.9.1, pfunit/4.6.1, gftl/1.8.1, gftl-shared/v1.5.1
esma_cmake/v3.24.0

cmake options used for configuring MAPL (if relevant):
-DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_FLAP=OFF -DBUILD_WITH_PFLOGGER=ON -DESMA_USE_GFE_NAMESPACE=ON -DBUILD_SHARED_MAPL=OFF -DUSE_EXTDATA2G=ON

New stack location is used because of conflicting dependencies with the currently used versions (e.g., fms/2022.04 is built off hdf/1.10.6 and netcdf/4.7.4), in

/scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/intel-2022.1.2_ncdf49

module use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/intel-2022.1.2_ncdf49/modulefiles/stack/
module load intel/2022.1.2
module load impi/2022.1.2
module load hpc/1.2.0
module load hpc-intel
module load hpc-impi
module load hdf5/1.14.0
module load netcdf/4.9.1
module load fms/2022.04
module load esmf/8.4.1
module load pio/2.5.10
module load gftl/1.8.1
module load gftl-shared/v1.5.1
module load yafyaml/1.0.4
module load fargparse/1.4.1
module load pflogger/1.9.1
module load pfunit/4.6.1
module load mapl/2.35.2-esmf-8.4.1

Please let me know if this configuration works successfully, especially mapl/2.35.2 and dependencies built. It is possible MAPL could be installed with fewer additional modules, but it requires knowledge what features are needed.

from hpc-stack.

Has this been resolved?
If so, please close?

from hpc-stack.

This task has been changed due to new mapl bugfix version release.
The replacement task have been included in UFS ticket ufs-community/ufs-weather-model#1621
This ticket can be closed.

from hpc-stack.