MIDAS (MIDAS Midas is Data Analysis Software) is a Python package primarily for processing gridded data stored in CF-compliant NetCDF/HDF5 format (http://cfconventions.org).
- spatial interpolation between quadrilateral meshes
- temporal interpolation between calendar dates (datetime)
- conservative re-mapping in the vertical dimension (MOM6/ALE)
- spatial integration/averaging with generalized horizontal/vertical coordinates
- temporal averaging (Datetime).
MIDAS was first developed by Matt Harrison as an employee of NOAA/GFDL and has been used for the generation of realistic global model configurations and post-run analysis scripts (https://github.com/NOAA-GFDL/MOM6-examples.git)
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.
If you just want the basics, and are not planning to use the remapping features, then simply type (bash)
INSTALL_PATH='/your/python/module/path'
python setup_nofort.py install --prefix=$INSTALL_PATH
If you have root access and want to use default installation paths, then omit the prefix option. If you are using conda, then the default location is specific to your current environment.
- Download and install Anaconda
(wget https://repo.anaconda.com/archive/Anaconda3-5.1.0-Linux-x86_64.sh;./Anaconda3-5.1.0-Linux-x86_64.sh)
- Update your existing shell to add conda to your path
source ~/.bashrc
- Now update conda
conda update conda
conda install conda-build
- Install gfortran development libraries (if needed)
sudo apt-get install libgfortran-6-dev
optionally install mpich2 and associated libraries
(sudo apt-get install libmpich2-dev)
Or else if you do not have root privileges
(. activate;wget http://www.mpich.org/static/downloads/3.2.1/mpich-3.2.1.tar.gz;cd Downloads;tar xvf mpich-3.2.1.tar.gz;cd mpich-?3.2.1;./configure --enable-sha\
red --prefix=$CONDA_PREFIX;make; make install)
On linux platforms, simply type
make
This will take some time, since the Makefile will be downloading and compiling several large packages, like hdf5 and netcdf. Why are we compiling everything when there are pre-compiled binaries avaialble from the Anaconda cloud? Problems arise when linking c and c++ libraries to fortran APIs. There are often strict requirements for matching versions and glibc compatability which make using pre-compiled code unfeasible.
For platforms with multiple users, it is recommended that the compiled libraries and Python packages be made available through a local channel.
For best results, build the following libraries yourself - conda does not handle dependencies for linking c and c++ libraries to fortran APIs - consider yourself lucky if you can work with pre-compiled packages and associated libraries
(conda create --name MIDAS)
git clone [email protected]:MJHarrison-GFDL/conda-recipes.git
(. activate MIDAS; cd conda-recipes/zlib;conda build .;conda install --use-local zlib)
(. activate MIDAS; cd conda-recipes/hdf5;conda build .;conda install --use-local hdf5)
(. activate MIDAS; cd conda-recipes/libnetcdf;conda build .;conda install --use-local libnetcdf)
(. activate MIDAS; cd conda-recipes/libnetcdff;conda build .;conda install --use-local libnetcdff)
Install the netCDF4 python API
(. activate MIDAS;pip install netCDF4)
Install MIDAS
git clone [email protected]:mjharriso/MIDAS.git
(. activate MIDAS; cd MIDAS;git checkout master;. build.sh)
TROUBLESHOOTING
if you have a problem with libmkl missing:
(. activate MIDAS; conda install nomkl numpy scipy scikit-learn numexpr)
(. activate MIDAS; conda remove mkl mkl-service)
missing libcomm_err.so.3 at runtime?
(ln -s $CONDA_PREFIX/pkgs/krb5-1.14.6-0/lib/libcom_err.so.3 $CONDA_PREFIX/lib/.)
EXAMPLES
cd examples
source activate MIDAS
python contour_example.py # Fetches OpenDAP URL from NODC and plots with Matplotlib
python hinterp_example.py # fms/horiz_interp does bi-linear interpolation from the original
# 1-deg to a 5-deg grid with masking
python hist.py # volume-weighted histogram of salinity in the Indian Ocean
python subtile.py # use subtiling algorithm to calculate un-weighted
# cell average bathymetry and roughness. Example along the Eastern US
source deactivate
HOW TO OBTAIN DOCUMENTATION
ipython
import midas.rectgrid as rectgrid
rectgrid.[Tab] # complete listing of methods
rectgrid.quadmesh # a generic rectangular grid description
# Can be read from a file or provided as
# 2-d lat/lon position arrays.
rectgrid.state? # Description of state instance
rectgrid.state.state.[Tab] # available methods for state objects
rectgrid.state.volume_integral? # integrate scalars over the domain
# in 'X','Y','Z','XY' or 'XYZ'
rectgrid.state?? # View the code
MORE INFORMATION
type
(. activate MIDAS;conda list)
this returns your current environment which should look something like this:
# Name Version Build Channel
ca-certificates 2018.4.16 0 conda-forge
curl 7.60.0 0 conda-forge
hdf5 1.8.20 0 local
krb5 1.14.6 0 conda-forge
libnetcdf 4.4.1 0 local
libnetcdff 4.4.4 0 local
libssh2 1.8.0 2 conda-forge
openssl 1.0.2o 0 conda-forge
zlib 1.2.11 1 local
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent