wrf-cmake / wrf Goto Github PK

The JOSS review asks reviewers to verify:

Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).

While the primary purpose of this work is to extend the build system and enable and validate portability and build system simplification, which it certainly has done, it would be good to include on the readme additional things, even though they may be more the responsibility of the upstream project. (Of course you can quote with attribution, where appropriate.)

I think at a minimum, there are two pieces of information that need to be included in---or linked to from---the readme:

1. How to run a very basic simulation once the software has been built, preferably with parallelism enabled.
2. How to run any existing unit/assessment/integration tests manually.

It's great that the CI pipeline does some assessment & acceptance testing automatically, but it would be beneficial for any human who wishes to verify that their installation is working to run some automated tests themselves, and setup and run a very simple example. Currently, there are no instructions on how to do this.

https://github.com/WRF-CMake/WRF/blob/wrf-cmake/doc/cmake/LIBS.md has broken links back to https://github.com/WRF-CMake/WRF/blob/wrf-cmake/doc/cmake/INSTALL.md

Also, consider using common square bracket links to to make maintenance easier. The awesome_bot project can be helpful for validating links in markdown and avoiding duplication.

e.g.:

You can go back to [Build and Install WRF-CMake].
You can even [show it as different text][Build and Install WRF-CMake].
You don't have to keep writing out the link to [Build and Install WRF-CMake]. If it breaks, you just need to fix it once!

[Build and Install WRF-CMake]:README_CMAKE_INSTALL.md#build-and-install-wrf-cmake

@letmaik this is really a non-issue but to make it consistent with other (current and future) repositories we have in the org, I think we should make the name for WRF and WPS repository lowercase. Note this this change will not impact the make version / CI as GitHub will keep the link to the uppercase repositories.

Differently then it is the case for releases, the assets section is not automatically expanded for pre-releases. This may be confusing to users who may stumble on the page looking for binaries and they cannot see them. I would suggest to either document this a bit more at the beginning of the description section or convert the pre-release to a release.

The simplest way to run WRF on Windows is to download the experimental prebuilt binary releases as described in the project's README or to install GIS4WRF on QGIS -- the latter may be particularly useful if you are looking to also pre- and post-process your input/output data.

Update WRF model ref to https://opensky.ucar.edu/islandora/object/opensky:2898

seems that processes does not communicate with each other and the calculation times are the same weather for 1, 6, 8 or more processes.

wrf.exe job finishes ok, but with exactly same timerun independent on core numbers
using the wrf-cmake-4.0-dmpar-basic-release-linux

Describe the bug
geogrid.exe is looking for orography data in DATA\geog/orogwd_1deg/con/index. However, windows does not allow any directory to be named "con". When geog data are unpacked, this directory gets named "_con", and cannot be renamed to "con". Hence, geogrid.exe fails.

To Reproduce
Steps to reproduce the behavior:
Run geogrid.exe following standard test case on https://gis4wrf.github.io/tutorials/wrf-cmake/simulate-the-2018-european-heat-wave-with-wrf-cmake/#download-geographical-data

Expected behavior
Code is actually behaving as expected. The problem is the path geogrid.exe uses to search for orography data, and the inability of Windows to name a directory "con".

System information (please complete the following information):

• Windows 10 pro
• Using downloaded binary wps-cmake-4.1.0-basic_nesting-serial-x64-windows-release.zip

Additional context
The code runs, so it's not really a bug, but geogrid.exe won't work without the orog data, so it's not really a feature request either...

To be consistent with other repos/projects, we should modify the licence text to:

# WRF-CMake (https://github.com/WRF-CMake/WRF).
# Copyright 2018-2019 M. Riechert and D. Meyer. Licensed under the MIT License.

Hi,
I just tried to merge WRF 4.1 into WRF-CMake but got a lot of errors.
Question : Will WRF-CMake support 4.1 and new version officially soon ?
Tx

when i run : ### Cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/pnt/Downloads/bufr/my_bouild_dir

i get this probleme :

CMake Deprecation Warning at CMakeLists.txt:3 (CMAKE_MINIMUM_REQUIRED):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

-- Found NetCDF: /usr/include (found version "4.8.1")
-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
NETCDF_LIBRARIES_F90 (ADVANCED)
linked by target "netcdf2littler" in directory /home/pnt/Downloads/bufr/src/netcdf2littler

-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

can you help me to build this programme (netcdflittleR)

#24 (comment)

In some situations, typically HPC environments, this would avoid having to set -DNETCDF_DIR=.. / -DNETCDF_FORTRAN_DIR=...

Note that if netcdf-fortran is built with CMake, then nf-config --prefix just prints "not supported in CMake builds" or something similar.

Hi. I'm new to WRF and CMake and this pre-built binaries really helps me a lot.
My problem is, when I try to refine the grid as in the tutorial (https://www2.mmm.ucar.edu/wrf/OnLineTutorial/CASES/NestRuns/ndown.php) , I find there is no ndown.exe in the main folder. So I wonder whether ndown in supported in this version of WRF, and are there alternative methods to complete this task?
I use the latest version of the program built for Windows: "wrf-cmake-4.2.2-basic_nesting-serial-x64-windows-release"

When working with WRF, sometimes one needs to re-compile the code (e.g., after changing some things in the source code). When changing the Registry, even a full re-compile is necessary (e.g., ./clean -a && ./compile em_real && ./compile wrf).

From the compilation instructions here, it is unclear to me how this relates to the WRF-CMAKE build system.

Do I always follow the cmake ... && make install ? Or do I have to clean somehow when changing the registry?

This should be added to the documentation.

WRF version 4.1.1 and 4.1.2 have been released, do we want to merge these into wrf-cmake branch or wait for 4.2.0... If we merge them in and create the releases, it may be a bit confuses if the prebuilt binaries are not releases.

Hi,

I want to compile WRF on windows and I was following this wiki ( https://github.com/WRF-CMake/wrf/blob/wrf-cmake/doc/cmake/LIBS.md#windows ) for installing required libraries on a clean installation of MSYS2

I got a first error compiling netcdf-c but it was easy to solve using :

pacman -S m4

But when I compile netcdf-fortran-4.4.4

I got this error I really do not find a solution

$ make
[  2%] Building Fortran object fortran/CMakeFiles/netcdff.dir/netcdf4.f90.obj
netcdf4_func.f90:708:75:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(4)/REAL(8)).
netcdf4_func.f90:698:75:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/REAL(8)).
netcdf4_func.f90:688:74:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(4)/REAL(8)).
netcdf4_func.f90:678:73:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(2)/REAL(8)).
netcdf4_func.f90:668:73:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(1)/REAL(8)).
netcdf4_func.f90:648:75:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(4)/REAL(8)).
netcdf4_func.f90:638:75:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/REAL(8)).
netcdf4_func.f90:628:74:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(4)/REAL(8)).
netcdf4_func.f90:618:73:

......
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(2)/REAL(8)).
netcdf4_func.f90:608:73:

Any help is appreciated.

Regards

For Windows builds with MPI, we currently require version 9 of MS-MPI due to microsoft/Microsoft-MPI#7. There is a work-around available but the long-term solution is that MS-MPI is released in two variants (with/without control flow guard compiler support) and we should wait until this happens as then we don't need to apply any changes ourselves.

• Wait until MS-MPI release without control flow guard
• Update download URL in README.md
• Update download URL in doc/cmake/LIBS.md

I have fixed the commit for now so that we can build on Windows (see 110d2fb). However this is not ideal and we need to find a better solution for the long-er term.

For reference, the netcdf-c build log can be viewed here: https://ci.appveyor.com/project/WRF-CMake/releases/build/1.0.1/job/91g2eytq43r08hi3#L600

It would be awesome to see WRF make it into the homebrew package manager system in the future. This would lower the barrier to entry for researchers and tinkerers. But there are some potential issues that might prevent that. The purpose of this issue is to discuss those potential issues, and further improving ease of use on macOS.

1. In general Homebrew doesn't like things built with brewed GCC as the C and C++ compiler. Whenever possible formulae should use native Apple clang tooling. The cmake build system throws a fatal error when the user attempts to build with the default Apple clang compiler. Is there a good reason for this? Are there known bugs, limitations, issues when building with Apple clang?
2. Homebrew, as is the case with many package managers, picks a default MPI implementation. The implementation chosen was OpenMPI, mimicking other Linux package managers that had to pick a version. As such, asking the users to install with MPICH can cause confusion and errors if they have other scientific/numerical software installed on their systems. Changing the install instructions for macOS to instruct users to use OpenMPI rather than MPICH will result in a smoother user experience and better compatibility with extant software on the target system. I have tested with both MPICH and OpenMPI, and, AFAICT, I did not encounter any issues utilizing OpenMPI rather than MPICH. Is there a good reason for telling users to use MPICH? Could the instructions be updated to recommend installing OpenMPI on macOS?
3. Consider adding a Brewfile to the repository, or instructions on what to put in it. From the JOSS guidelines:

Good: A package management file such as a Gemfile or package.json or equivalent
A Brewfile would be the Homebrew equivalent of this, allowing users to install all specified dependencies and execute commands in an isolated environment.

This issue will likely conclude my comments about installation on macOS.

Here are some example Brewfiles that you may wish to use.

MPICH (less prefered)

# macOS Homebrew Brewfile to install dependencies for WRF
# Usage: `brew bundle install --file=./Brewfile`
tap "homebrew/core"
# Distributed revision control system
brew "git"
# Cross-platform make
brew "cmake"
# GNU compiler collection
brew "gcc@8"
# Libraries and data formats for array-oriented scientific data
brew "netcdf"
# Library for manipulating JPEG-2000 images
brew "jasper"
# Implementation of the MPI Message Passing Interface standard
brew "mpich"

OpenMPI

# macOS Homebrew Brewfile to install dependencies for WRF
# Usage: `brew bundle install --file=./Brewfile`
tap "homebrew/core"
# Distributed revision control system
brew "git"
# Cross-platform make
brew "cmake"
# GNU compiler collection
brew "gcc@8"
# Libraries and data formats for array-oriented scientific data
brew "netcdf"
# Library for manipulating JPEG-2000 images
brew "jasper"
# High performance message passing library
brew "open-mpi"

Describe the bug
I have tested WRF-CMake-4.1.5 binary release:
wrf-cmake-4.1.5-basic_nesting-serial-x64-linux-release.tar.xz
and the RPATH in the binary executables are pointing to: $ORIGIN/../app.libs
but no app.libs directory it's bundle in the .tar.xz file.

To Reproduce
Steps to reproduce the behavior:

1. Just download the above release.
2. Execute a binary (i.e. main/wrf.exe)
3. See error about shared object not found

Expected behavior
The dependencies and RPATH should point to $ORIGIN/.libs.
Release wrf-cmake-4.1.4, works!!.

System information (please complete the following information):

• OS name and version: Ubuntu 18.04
• Compiler name and version: N/A

Before publishing we should update the Figure 1 in JOSS paper with the new plots created by WATS. Conclusions will not change but the plots will be slightly different due to changes in 4.1.0.

As far as I can tell, when you create a tag through the GitHub UI, it creates a dangling commit object. This means that git describe doesn't include it when searching through the tags for the current branch or commit.

For example, I have checked out the WRF-CMake-4.0.3 tag, but git describe on d98d1d4 says: v4.0.3-39-gd98d1d47d

Hello,

Just testing the WRFv4.0 binaries (serial) on my Windows 10. It seems like the ideal simulations cannot be started due to an error in ideal.exe.

\wrf-cmake-4.0-serial-basic-release-windows\test\em_hill2d_x
also same error in em_les

$ ..\..\main\ideal.exe
 Ntasks in X            1 , ntasks in Y            1
IDEAL V4.0 PREPROCESSOR
 *************************************
 Parent domain
 ids,ide,jds,jde            1         202           1           3
 ims,ime,jms,jme           -4         207          -4           8
 ips,ipe,jps,jpe            1         202           1           3
 *************************************
DYNAMICS OPTION: Eulerian Mass Coordinate
   alloc_space_field: domain            1 ,              65282320  bytes allocated
  pi is    3.14159274
At line 19288 of file C:/projects/releases/WRF/build/inc/nl_config.inc
Fortran runtime error: Actual string length is shorter than the declared one for dummy argument 'mminlu' (4/256)

Error termination. Backtrace:

Could not print backtrace: libbacktrace could not find executable to open
#0  0xffffffff
...

Could you take a look at this problem? real.exe and wrf.exe are running just fine.

Thanks for your efforts.

Part of the JOSS review asks about:

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

I'm a bit confused as to how to assess this. I know the eventual plan is, possibly, to integrate this back upstream. But I have yet to pour over upstream's readme document.

I think, at a minimum, you need to include a CONTRIBUTING.md file, with a link from the readme that addresses some of this. You could put all of it in the README, of course, but point 1, "contributing to the software", typically has enough text to warrant its own CONTRIBUTING.md file. For reporting issues or problems, you could add an issue template or templates, and address this in the README. For 3, "seek support", you could delineate which types of issues should be reported upstream through the usual mechanism, whatever that is, and include any details of how to seek support for the CMake/build system portions, even if that is "no support offered, beyond bug reporting and pull requests".

In Linux, when using Environment Modules, CMake cannot find the NetCDF Fortran interface when NetCDF-Fortran has been installed under a different path to NetCDF-C.

For example, with the following configuration:

$ nc-config --all

This netCDF 4.4.1.1 has been built with the following features:

  --cc        -> mpiicc
  --cflags    -> -I/external/netcdf/4.4.1-c/include -I/external/hdf5/1.10.1//include
  --libs      -> -L/external/netcdf/4.4.1-c/lib -lnetcdf

  --has-c++   -> no
  --cxx       ->

  --has-c++4  -> no
  --cxx4      ->

  --has-fortran-> yes
  --fc        -> mpiifort
  --fflags    -> -I/external/netcdf/4.4.4-fortran/include
  --flibs     -> -L/external/netcdf/4.4.4-fortran/lib -lnetcdff -L/external/netcdf/4.4.1-c/lib -lnetcdf -lnetcdf
  --has-f90   -> no
  --has-f03   -> yes

  --has-dap   -> yes
  --has-nc2   -> yes
  --has-nc4   -> yes
  --has-hdf5  -> yes
  --has-hdf4  -> no
  --has-logging-> no
  --has-pnetcdf-> no
  --has-szlib ->

  --prefix    -> /external/netcdf/4.4.1-c
  --includedir-> /external/netcdf/4.4.1-c/include
  --libdir    -> /external/netcdf/4.4.1-c/lib
  --version   -> netCDF 4.4.1.1

The configuration fails with the following error:

$ cmake -DNETCDF_DIR=/external/netcdf/4.4.1-c ..
-- Using /home/WRF/cmake/arch/default/Intel_UNIX.cmake (C)
-- Using /home/WRF/cmake/arch/default/Intel_UNIX.cmake (Fortran)
-- Using /home/WRF/cmake/arch/default/Intel_Fortran_UNIX.cmake (Fortran)
-- Using /home/WRF/cmake/arch/default/Intel_UNIX.cmake (linker)
-- Failed to find NetCDF interface for F77
-- Failed to find NetCDF interface for F90
CMake Error at /external/cmake/3.14.0/share/cmake-3.14/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find NetCDF (missing: NETCDF_HAS_INTERFACES)
Call Stack (most recent call first):
  /external/cmake/3.14.0/share/cmake-3.14/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindNetCDF.cmake:150 (find_package_handle_standard_args)
  CMakeLists.txt:264 (find_package)

Currently, the documentation for building from source says to run make install after CMake. So far (build in progress) it seems that you can build in parallel. It would be nice to explicitly address this issue.

Placeholder issue to remember to update installation docs using Ninja.

I haven't tried make install on linux yet, but on macOS it puts executables into an abnormal directory structure: <prefix>/{main,test}.

Also, .exe suffix is not commonly used outside of windows. CMake should handle that automatically for you.

As for the directory structure, I would recommend using GNUInstallDirs on macOS and Linux and putting binaries into ${CMAKE_INSTALL_BINDIR} and the test/examples directory can be installed into ${CMAKE_INSTALL_DATADIR}/wrf.

@letmaik this is currently not a priority or a strong requirement but it would be good to understand the type and amount of work required to add this option to our CMake support. Are you happy to have a look at this so we can judge the feasibility of such enhancement? This option would apply to WRF only -- i.e. it would not apply to WPS.

After running brew update && brew install open-mpi , what do I need to run to be able to start using wrf?