Troubles to build about luma HOT 17 CLOSED

Hi, thanks for opening the issue. You are really supposed to install the parallel version of HDF5 as per the installation instructions. Typically your linking will fail if you try to link against the serial version. However, at the moment your compilation is failing so let's start with that. Can you confirm that the file hdf5.h is indeed present at the path: /usr/include/hdf5/include which is where the makefile is looking for it?

from luma.

Thanks for the reply @aharwood2 . I am trying to install the parallel version of hdf5....

I have one question: Which version have you used for hdf5? I noticed in the hdf5 website there is no longer maintenance... I am getting errors in the parallel installation. This is a excerpt of my installation guide:

2.1. Know parallel compilers
----------------------------
HDF5 knows several parallel compilers: mpicc, hcc, mpcc, mpcc_r. To build
parallel HDF5 with one of the above, just set CC as it and configure.

$ CC=/usr/local/mpi/bin/mpicc ./configure --enable-parallel --prefix=<install-directory>
$ make # build the library
$ make check # verify the correctness
# Read the Details section about parallel tests.
$ make install

Just one comment, for example inside the /usr/include/mpi folder I have the next files and folders:
mpi-ext.h mpif-config.h mpif-externals.h mpif.h mpif-io-constants.h mpif-sentinels.h mpi.h mpp openshmem pshmem.h shmem-compat.h shmem.h mpif-c-constants-decl.h mpif-constants.h mpif-ext.h mpif-handles.h mpif-io-handles.h mpif-sizeof.h mpi_portable_platform.h openmpi pmix pshmemx.h shmem.fh shmemx.h

I tried to configure the installation with the following call ./configure --prefix=/usr/local/hdf5 --enable-parallel.... I even enable fortran but it didn't passed the make check call.

from luma.

Are you following our installation guide? It states quite clearly what version we are using and how to install it on Ubuntu. If this is out of date, please let us know and I can update the docs.

from luma.

Dear @aharwood2 I have downloaded the version HDF5 1.8.17 version, I assume its the same as in the installation guide link you shared before.

I have done the following steps

1. sudo apt-get install mpich regarding MPI let me share this message from terminal libopenmpi-dev is already the newest version (2.1.1-8), this came after calling sudo apt-get install libopenmpi-dev.
2. Following link to install paralell hdf5 I wrote in the terminal the following command CC=/usr/lib/mpich/include.
3. After I executed the configuration step ./configure --enable-parallel --enable-fortran --prefix=/usr/local/hdf5
4. later make (no problems)
5. After make check and here comes the problem I reported before. Let me share and excerpt of the error:
   Makefile:1306: recipe for target 't_pflush1.chkexe_' failed
make[4]: *** [t_pflush1.chkexe_] Error 1
make[4]: Leaving directory '/home/valdez/CMake-hdf5-1.8.17/hdf5-1.8.17/testpar'
Makefile:1381: recipe for target 'build-check-p' failed
make[3]: *** [build-check-p] Error 1
make[3]: Leaving directory '/home/valdez/CMake-hdf5-1.8.17/hdf5-1.8.17/testpar'
Makefile:1286: recipe for target 'test' failed
make[2]: *** [test] Error 2
make[2]: Leaving directory '/home/valdez/CMake-hdf5-1.8.17/hdf5-1.8.17/testpar'
Makefile:1089: recipe for target 'check-am' failed
make[1]: *** [check-am] Error 2
make[1]: Leaving directory '/home/valdez/CMake-hdf5-1.8.17/hdf5-1.8.17/testpar'
Makefile:576: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1

Can you tell me how to fix the make check step? OR should I ignore it and make install?

Thanks.-

from luma.

I'm still not sure why you are doing steps 2-4. What Linux distro are you using?

On our Ubuntu systems we just type: sudo apt-get install mpich followed by sudo apt-get install libhdf5-mpich-dev as it says in the installation guide to install the packages. Why are you trying to build something with make? We don't tell you to do we?

Why do you call sudo apt-get install libopenmpi-dev? This installs OpenMPI. You already have MPICH installed. If you install both side-by-side I guarantee you are causing yourself a world of pain. We don't tell you to install OpenMPI in the instructions so why have you?

My suggestion is to purge your packages and follow the installation instructions to the letter.

from luma.

Dear @aharwood2 , I did steps 2-4, because I thought that LUMA needed hdf5 library installed..... I will just then do this steps:
sudo apt-get install mpich,
sudo apt-get install libhdf5-mpich-dev,

and after go back to LUMA's installation, is this ok?

from luma.

Line 2 of what you typed installs HDF5 -- that is why the package name has hdf5 in it..

If you stick with the instructions you shouldn't run into any more problems.

from luma.

Dear @aharwood2, before closing, I have one last comment. Inside the path /usr/include/hdf5/mpich contains the hdf5.h file. I export the path for LUMA's buildingexport HDF5_HOME=/usr/include/hdf5/mpich and get the following error after make command:
src/../inc/hdf5luma.h:31:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h" // Load C API
^~~~~~~~
compilation terminated.
makefile:37: recipe for target 'obj/GridObj_ops_io.o' failed
make: *** [obj/GridObj_ops_io.o] Error 1

Thanks for the help.

from luma.

Please can you tell us

1. the operating system you are running on
2. the instructions you are following

from luma.

dear @ianhinder I am working on ubuntu 18.04.4 LTS. I have executed the following steps:

export HDF5_HOME=/usr/include/hdf5/mpich

make in the directory where the makefile is located....

from luma.

Thanks. Please can you answer no 2 - which instructions are you following? Is it https://github.com/aharwood2/LUMA/wiki/Installing-LUMA, and then https://github.com/aharwood2/LUMA/wiki/Building-Dependencies? I just want to make sure we are all talking about the same instructions.

from luma.

dear @ianhinder I understood your question. Let me explain.

Firstly from here I executed the terminal commands sudo apt-get install mpich and sudo apt-get install libhdf5-mpich-dev in that order. Everything was fine.

And secondly from here I imported the path:
export HDF5_HOME=/usr/include/hdf5/mpich the path is correct and exists and contains the header file hdf5.h. Later I gave make inside the LUMA/LUMA folder and the error came out.

from luma.

I think I see the problem - the instructions say to use

export HDF5_HOME=/usr/lib/x86_64-linux-gnu/hdf5/mpich

but you have set

export HDF5_HOME=/usr/include/hdf5/mpich

Can you try with the first one? Make sure to run "make clean" before "make" to remove the results of the previous incorrect compilation.

from luma.

dear @ianhinder problem solved. Thank you very much... I compiled and obtain the executable LUMA.....
During the execution of mpirun -np 6 ./LUMA I get the following message (many times, like once per iteration)
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 0:
#000: ../../../src/H5Dio.c line 225 in H5Dwrite(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type

Is this ok?

from luma.

Apologies for the delay in replying.

1. Please can you send (as a file attachment to a comment) the output of "dpkg -l"? This will list all the packages installed on your system. I'm thinking you might have a mixture of different MPI and HDF5 versions.
2. Please can you also send the rank 0 logfile? This will be in the output directory output_<date>/log_rank0.log.

from luma.

Dear @ianhinder please find attached the requested files.
dpkg_output.txt
log_rank0.log

from luma.

You have both openmpi and mpich libraries installed. I'm not sure if this is causing the problem, but it can sometimes be an issue. Can you try removing all the openmpi related ones with

apt remove libhdf5-openmpi-100 libopenmpi-dev libopenmpi2 openmpi-bin openmpi-common

then "make clean" and "make" again in the LUMA directory, and try the simulation again.

The error message is actually saying that LUMA failed to write to the HDF5 files, which is a bit strange. Is it possible that you don't have enough disk space?

from luma.