hpc4cmb / libmadam Goto Github PK

When running configure on my laptop (a 64-bit PC running Linux Mint 17.2), the following error happens:

**** Enabling FFTW support.
checking fftw3.f03 fortran 90 header... find: `NONE': No such file or directory
not found

To install the FFTW library file, I used apt-get to install the package in Lint's repositories, which is in the standard location /usr/lib/x86_64-linux-gnu/libfftw3.*. The problem can be fixed by passing --with-fftw=/usr to configure:

./configure --with-fftw=/usr

However, it would be nice if the configure script would work it out for itself.

Things don't go smooth with --with-fftw, however, as configure now fails with another error, which I report on this page because I suspect it shares the same cause with the issue above:

checking fftw3.f03 fortran 90 header... not found
configure: error: in `/home/tomasi/tmp/libmadam/build2':
configure: error: fftw3.f03 is required.

I am not a gfortran expert, so I do not know why configure is not able to find this header in such a standard location as /usr/include. I brutally fixed this problem by copying fftw3.f03 manually into libmadam's source directory:

cp /usr/include/fftw3.f03 ./src

With this trick (and --with-fftw) I have been able to configure and compile libmadam correctly on my Linux laptop.

Is it possible to setup MADAM to use pixelation schemes other than HEALPix? E.g., some arbitrary WCS pixelation scheme on a cartesian grid?

I am trying to make the executables for libmadam. My configuration has successfully completed. however, i am unable to make it. It crashes with following errors:

kumasura@cori07:~/libmadam> make
make all-recursive
make[1]: Entering directory '/global/u2/k/kumasura/libmadam'
Making all in src
make[2]: Entering directory '/global/u2/k/kumasura/libmadam/src'
/bin/sh ../libtool --tag=CC --mode=link mpicc -O3 -fopenmp -fopenmp -o test_libmadam test_libmadam.o libmadam.la -L/usr/common/software/fftw/3.3.4/hsw/intel//lib -lfftw3 -lm -L/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_cdft_core.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_gf_ilp64.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_gnu_thread.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_blacs_openmpi_ilp64.a /usr/common/software/cfitsio/3.370-reentrant/hsw/intel/lib/libcfitsio.a /usr/common/software/fftw/3.3.4/hsw/intel/lib/libfftw3.a -Wl,--end-group -lgomp -lpthread -lm -ldl
libtool: link: mpicc -O3 -fopenmp -fopenmp -o .libs/test_libmadam test_libmadam.o -Wl,--start-group -Wl,--end-group ./.libs/libmadam.so -L/usr/common/software/fftw/3.3.4/hsw/intel//lib -L/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_scalapack_ilp64.a -liomp5 /usr/common/software/fftw/3.3.4/hsw/intel/lib/libfftw3.so /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_cdft_core.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_gf_ilp64.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_gnu_thread.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_blacs_openmpi_ilp64.a /usr/common/software/cfitsio/3.370-reentrant/hsw/intel/lib/libcfitsio.a /usr/common/software/fftw/3.3.4/hsw/intel/lib/libfftw3.a /opt/gcc/7.3.0/snos/lib/../lib64/libgomp.so -lpthread -lm -ldl -fopenmp -Wl,-rpath -Wl,/usr/common/software/fftw/3.3.4/hsw/intel/lib -Wl,-rpath -Wl,/opt/gcc/7.3.0/snos/lib/../lib64
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(dsbrdb.o): in function mkl_lapack_dsbrdb': dsbrdb_gen.f:(.text+0xdf): undefined reference to mkl_lapack_dsbrdbn'
/usr/bin/ld: dsbrdb_gen.f:(.text+0x1de7): undefined reference to mkl_lapack_dlaeh2' /usr/bin/ld: dsbrdb_gen.f:(.text+0x2f57): undefined reference to mkl_lapack_dlaeh2'
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(_avx_d__trmm_rut.o): in function mkl_blas_avx_dtrmm_rut': _trmm_rut.f:(.text+0x818f): undefined reference to mkl_blas_daxpy'
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(_avx_d__trmm_run.o): in function mkl_blas_avx_dtrmm_run': _trmm_run.f:(.text+0x8140): undefined reference to mkl_blas_daxpy'
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(_avx_d__trmm_rlt.o): in function mkl_blas_avx_dtrmm_rlt': _trmm_rlt.f:(.text+0x84e1): undefined reference to mkl_blas_daxpy'
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(_avx_d__trmm_rln.o): in function mkl_blas_avx_dtrmm_rln': _trmm_rln.f:(.text+0x81c2): undefined reference to mkl_blas_daxpy'
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(_mc3_d__trmm_rut.o): in function mkl_blas_mc3_dtrmm_rut': _trmm_rut.f:(.text+0x7bfb): undefined reference to mkl_blas_daxpy'
/usr/bin/ld: /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a(_mc3_d__trmm_run.o):_trmm_run.f:(.text+0x7961): more undefined references to mkl_blas_daxpy' follow /usr/bin/ld: link errors found, deleting executable .libs/test_libmadam'
Makefile:684: recipe for target 'test_libmadam' failed
make[2]: *** [test_libmadam] Error 1
make[2]: Leaving directory '/global/u2/k/kumasura/libmadam/src'
Makefile:414: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/global/u2/k/kumasura/libmadam'
Makefile:345: recipe for target 'all' failed
make: *** [all] Error 2

The current loaded module in the environment are:

kumasura@cori07:~/libmadam> module list
Currently Loaded Modulefiles:

1. modules/3.2.10.6 13) ugni/6.0.14.0-6.0.7.1_3.13__gea11d3d.ari
2. nsg/1.2.0 14) pmi/5.0.14
3. craype-haswell 15) dmapp/7.1.1-6.0.7.1_5.45__g5a674e0.ari
4. craype-network-aries 16) gni-headers/5.0.12.0-6.0.7.1_3.11__g3b1768f.ari
5. craype/2.5.15 17) xpmem/2.2.15-6.0.7.1_5.11__g7549d06.ari
6. cray-mpich/7.7.3 18) job/2.2.3-6.0.7.1_5.43__g6c4e934.ari
7. openmpi/3.1.3 19) dvs/2.7_2.2.118-6.0.7.1_10.1__g58b37a2
8. gcc/7.3.0 20) alps/6.6.43-6.0.7.1_5.45__ga796da32.ari
9. altd/2.0 21) rca/2.2.18-6.0.7.1_5.47__g2aa4f39.ari
10. darshan/3.1.4 22) atp/2.1.3
11. cray-libsci/18.07.1 23) PrgEnv-gnu/6.0.4
12. udreg/2.3.2-6.0.7.1_5.13__g5196236.ari 24) intel/18.0.1.163

Please help me resolve the problem?

Running the script toast_satellite_sim.py with option --madam causes the following error:

Traceback (most recent call last): File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/bin/toast_satellite_sim.py", line 4, in <module> __import__('pkg_resources').run_script('toast==2.0', 'toast_satellite_sim.py') File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 719, in run_script self.require(requires)[0].run_script(script_name, ns) File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1504, in run_script exec(code, namespace, namespace) File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/EGG-INFO/scripts/toast_satellite_sim.py", line 357, in <module> main() File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/EGG-INFO/scripts/toast_satellite_sim.py", line 345, in main madam.exec(data) File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/toast/map/madam.py", line 146, in exec raise RuntimeError("Cannot find libmadam") RuntimeError: Cannot find libmadam

The actual error is hidden by the try-except statement in the file /homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/toast/map/madam.py (lines 26-34). Adding the line libmadam = ct.CDLL('libmadam.so') before the try-except statement reveals the real error:

Traceback (most recent call last): File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/bin/toast_satellite_sim.py", line 4, in <module> __import__('pkg_resources').run_script('toast==2.0', 'toast_satellite_sim.py') File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 719, in run_script self.require(requires)[0].run_script(script_name, ns) File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1504, in run_script exec(code, namespace, namespace) File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/EGG-INFO/scripts/toast_satellite_sim.py", line 20, in <module> import toast.map as tm File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/toast/map/__init__.py", line 8, in <module> from .madam import OpMadam File "/homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/python3.5/site-packages/toast-2.0-py3.5-linux-x86_64.egg/toast/map/madam.py", line 25, in <module> libmadam = ct.CDLL('libmadam.so') File "/appl/opt/python/3.5.1-gnu520/lib/python3.5/ctypes/__init__.py", line 347, in __init__ self._handle = _dlopen(self._name, mode) OSError: /homeappl/home/ovlindho/appl_sisu/Virtualenvs/hpc4cmb/lib/libmadam.so: undefined symbol: omp_get_num_threads_

The system I'm compiling on is called "Sisu". The operating system is described as follows:

"Sisu is a Linux cluster. The login nodes of Sisu are based on the SUSE Linux Enterprise Server 11.3 (SLES11.3) distribution. The computing nodes use a run-time environment provided by Cray, called Compute Node Linux (CNL). Compute Node Linux, is a run-time environment based on SLES as well, but it includes only drivers, libraries and compilers required to run high-performance computing applications and a minimal amount of user-level commands (see http://www.busybox.net/)."

The compiler I'm using is GNU Fortran (GCC) 5.3.0, apparently with some modifications by Cray. I'm compiling the code on the login nodes and running it on the computing nodes.

Hi,

I would greatly appreciate some help on setting up libmadam please?
It seems the configure command doesn't find a few modules.

Here is the output of the configure command:

"checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for mpif95... no
checking for mpxlf95_r... no
checking for mpxlf95... no
checking for ftn... no
checking for mpif90... mpif90
checking whether the Fortran compiler works... yes
checking for Fortran compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU Fortran compiler... yes
checking whether mpif90 accepts -g... yes
checking for function MPI_INIT... yes
checking for mpif.h... yes
checking for Fortran flag to compile .f90 files... none
checking whether make supports the include directive... yes (GNU style)
checking for mpicc... mpicc
checking for gcc... mpicc
checking whether we are using the GNU C compiler... yes
checking whether mpicc accepts -g... yes
checking for mpicc option to accept ISO C89... none needed
checking whether mpicc understands -c and -o together... yes
checking dependency style of mpicc... gcc3
checking for function MPI_Init... yes
checking for mpi.h... yes
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by mpicc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from mpicc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... mpicc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if mpicc supports -fno-rtti -fno-exceptions... no
checking for mpicc option to produce PIC... -fPIC -DPIC
checking if mpicc PIC flag -fPIC -DPIC works... yes
checking if mpicc static flag -static works... yes
checking if mpicc supports -c -o file.o... yes
checking if mpicc supports -c -o file.o... (cached) yes
checking whether the mpicc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for mpif90 option to produce PIC... -fPIC
checking if mpif90 PIC flag -fPIC works... yes
checking if mpif90 static flag -static works... yes
checking if mpif90 supports -c -o file.o... yes
checking if mpif90 supports -c -o file.o... (cached) yes
checking whether the mpif90 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for mpif90 option to support OpenMP... -fopenmp
checking for mpicc option to support OpenMP... -fopenmp
checking for sgemm_... no
checking for ATL_xerbla in -latlas... no
checking for sgemm_ in -lmkl... no
checking for sgemm_... no
checking for sgemm_ in -lblas... yes
checking for dgemm_ in -ldgemm... no
checking for sgemm_ in -lcxml... no
checking for sgemm_ in -ldxml... no
checking for sgemm_ in -lscs... no
checking for sgemm_ in -lcomplib.sgimath... no
checking for sgemm_ in -lblas... (cached) yes
checking for sgemm_ in -lessl... no
checking for sgemm_ in -lblas... (cached) yes
checking for cheev_... no
checking for cheev_ in -llapack... yes
checking fitsio.h usability... no
checking fitsio.h presence... no
checking for fitsio.h... no
checking for ffopen in user specified location... no
checking for ffopen in default location... no
configure: error: Could not find the CFITSIO library!"

I did install fitsio via pip3 and CFITSIO via configure/make.
Specifying the CFITSIO path using the --with-cfitsio option ends up with the same issue.

Any idea what the matter could be?

Compiling the code with make -j N in order to spread the compilation over N jobs makes compilation fail:

(snip)
/bin/bash ../libtool  --tag=FC   --mode=compile mpif90  -I/usr/include -I/usr/include -O3 -fopenmp -c -o simulation.lo  ../../src/simulation.f90
../../src/satellite.f90:14.7:

   use healpix_routines
       1
Fatal Error: Can't open module file 'healpix_routines.mod' for reading at (1): No such file or directory

Avoiding -j N (or using -j 1) makes the code compile perfectly.

For me there is no need to try to fix this issue: I am just reporting this so that others who get struck with this are able to figure out what's happening. (I'm thinking of those who have a line like export MAKEFLAGS=j4 buried in their .bashrc.)

After a successful configuration & build of libmadam, make check fails with the following error:

(snip)
FAIL: test_libmadam
FAIL: test_libmadam.py
(snip)

The first test (test_libmadam) fails with the following error:

(snip)
 Iteration done
    16 iteration steps

 Finalization begins

 Clock =     0.566 s
 Writing pixel matrix...
 FITSIO Error Status =         105 : couldn't create the named file
 failed to create new file (already exists?):                                    
 ./maps/madam_wcov_inv.fits                                                      
 FITS error
 program exits with exit code            1

I looked for a file named madam_wcov_inv.fits, but found no matches:

$ find ./ -name "madam*fits"
$

I have no idea why it fails here.

The second test (test_libmadam.py) fails with the following error message:

('Failed to load libmadam at ', '../../src/.libs/libmadam.so')
Traceback (most recent call last):
  File "../../src/test_libmadam.py", line 14, in <module>
    _madam = ctypes.CDLL( path.replace('.so','.dylib') )
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: ../../src/.libs/libmadam.dylib: cannot open shared object file: No such file or directory

Here the problem is clearer, the test script assumes an OS X architecture (where dynamic libraries have extension .dylib). I would have happily submitted a PR, but I am not sure how to fix this (perhaps by a call to find_library?).

Dear all, when i try to run the python test i get the following output (after an apparently successful installation):

sudo python3 setup.py test
running test
tests.test_libmadam (unittest.loader._FailedTest) ... ERROR
tests.test_libmadam_mc (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: tests.test_libmadam (unittest.loader._FailedTest)

ImportError: Failed to import test module: tests.test_libmadam
Traceback (most recent call last):
File "/home/esteban/libmadam-master/python/libmadam_wrapper/init.py", line 28, in
_madam = ct.CDLL("libmadam.so")
File "/usr/lib/python3.6/ctypes/init.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: libmadam.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
import(name)
File "/home/esteban/libmadam-master/python/tests/test_libmadam.py", line 19, in
import libmadam_wrapper as madam
File "/home/esteban/libmadam-master/python/libmadam_wrapper/init.py", line 32, in
_madam = ct.CDLL(path)
File "/usr/lib/python3.6/ctypes/init.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: libmadam.so.0: cannot open shared object file: No such file or directory

======================================================================
ERROR: tests.test_libmadam_mc (unittest.loader._FailedTest)

ImportError: Failed to import test module: tests.test_libmadam_mc
Traceback (most recent call last):
File "/home/esteban/libmadam-master/python/libmadam_wrapper/init.py", line 28, in
_madam = ct.CDLL("libmadam.so")
File "/usr/lib/python3.6/ctypes/init.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: libmadam.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
import(name)
File "/home/esteban/libmadam-master/python/tests/test_libmadam_mc.py", line 19, in
import libmadam_wrapper as madam
File "/home/esteban/libmadam-master/python/libmadam_wrapper/init.py", line 32, in
_madam = ct.CDLL(path)
File "/usr/lib/python3.6/ctypes/init.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: libmadam.so.0: cannot open shared object file: No such file or directory

Ran 2 tests in 0.000s

FAILED (errors=2)

How do i fix this?
Thank you in advance.

When using toast with the --madam flag, libmadam crashes with the following error message, caused by a missing function zhhev (part of LAPACK):

$ ./toast_satellite_sim.py --madam
Traceback (most recent call last):
  File "/home/tomasi/work/CORE/toast-master/venv/lib/python3.4/site-packages/toast-2.0-py3.4-linux-x86_64.egg/toast/map/madam.py", line 27, in <module>
    libmadam = ct.CDLL('libmadam.so')
  File "/usr/lib/python3.4/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/tomasi/work/CORE/toast-master/venv/lib/libmadam.so: undefined symbol: zheev_

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./toast_satellite_sim.py", line 4, in <module>
    __import__('pkg_resources').run_script('toast==2.0', 'toast_satellite_sim.py')
  File "/home/tomasi/work/CORE/toast-master/venv/lib/python3.4/site-packages/pkg_resources/__init__.py", line 719, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/tomasi/work/CORE/toast-master/venv/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1504, in run_script
    exec(code, namespace, namespace)
  File "/home/tomasi/work/CORE/toast-master/venv/lib/python3.4/site-packages/toast-2.0-py3.4-linux-x86_64.egg/EGG-INFO/scripts/toast_satellite_sim.py", line 20, in <module>
    import toast.map as tm
  File "/home/tomasi/work/CORE/toast-master/venv/lib/python3.4/site-packages/toast-2.0-py3.4-linux-x86_64.egg/toast/map/__init__.py", line 8, in <module>
    from .madam import OpMadam
  File "/home/tomasi/work/CORE/toast-master/venv/lib/python3.4/site-packages/toast-2.0-py3.4-linux-x86_64.egg/toast/map/madam.py", line 31, in <module>
    libmadam = ct.CDLL(path)
  File "/usr/lib/python3.4/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/tomasi/work/CORE/toast-master/venv/lib/libmadam.so.0: undefined symbol: zheev_

Running ldd on libmadam.so produces the following output:

$ ldd src/.libs/libmadam.so                                                                                                    [2:28:02]
    linux-vdso.so.1 =>  (0x00007ffdbc32f000)
    libcfitsio.so.3 => /usr/lib/x86_64-linux-gnu/libcfitsio.so.3 (0x00007f171ab00000)
    libfftw3.so.3 => /usr/lib/x86_64-linux-gnu/libfftw3.so.3 (0x00007f171a708000)
    libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f171a3ed000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f171a0e7000)
    libmpi_f90.so.1 => /usr/lib/libmpi_f90.so.1 (0x00007f1719ee2000)
    libmpi_f77.so.1 => /usr/lib/libmpi_f77.so.1 (0x00007f1719cad000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f1719a9e000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1719888000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f17194c2000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f17192a4000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f171908b000)
    libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f1718e4e000)
    libmpi.so.1 => /usr/lib/libmpi.so.1 (0x00007f1718acd000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f172508e000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f17188c9000)
    libhwloc.so.5 => /usr/lib/x86_64-linux-gnu/libhwloc.so.5 (0x00007f1718689000)
    libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f171847f000)
    libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f1718273000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f171806f000)

My Linux version is Mint 17.2, with gfortran 4.8.4 and LAPACK 3.5.0. LAPACK has been installed using apt-get, so the library is in a standard directory (/usr/lib/liblapack.so).

Madam uses a boolean table to keep track of which processes have data for which submaps. The size of the table is proportional to both the number of MPI processes and the number of submaps.

We have seen cases where the boolean table seemingly allocates successfully (no error is returned) but the code abruptly terminates trying to initialize it. Reducing the number of submaps via nside_submap works around the problem.