Comments (13)
zxdawn
commented on May 26, 2024
1
Here's the sample extracted from original wrfout* file.
from xesmf.
JiaweiZhuang
commented on May 26, 2024
conservative works on intensive properties (density-like fields). If you have extensive properties (mass-like), divide by cell area first.
from xesmf.
JiaweiZhuang
commented on May 26, 2024
Alternatively, nearest_d2s conserves extensive properties directly.
from xesmf.
zxdawn
commented on May 26, 2024
Thank you for your advice! nearest_d2s works fine.
By the way, have you tried to calculate the cell area? I can't find package in python to calculate it directly. I used to calculate it by cal_area.m with pymatbridge.
from xesmf.
JiaweiZhuang
commented on May 26, 2024
nearest_d2sworks fine.
That's good to know! I would appreciate a reproducible example (code + sample data). nearest_d2s is so rarely used so it is nice to show a use case.
have you tried to calculate the cell area?
See this gist for now: https://gist.github.com/JiaweiZhuang/fc2f860133174523051260817e75a99d
from xesmf.
zxdawn
commented on May 26, 2024
I've rewritten the regridding prat and you can check it.
It looks like nearest_d2s and conservative get the similar result, while bilinear keeps the original distribution.
from xesmf.
JiaweiZhuang
commented on May 26, 2024
Thanks! Is the data file also available?
from xesmf.
zxdawn
commented on May 26, 2024
Maybe we can compare the result with 'True' value directly.
Since all methods are almost same for 0.1° * 0.1°.
We can regird data to 0.1° * 0.1° first and use binned_statistic_2d to sum for 1° * 1°.
Finish. You can check here.
from xesmf.
zxdawn
commented on May 26, 2024
nearest_d2sworks fine.That's good to know! I would appreciate a reproducible example (code + sample data).
nearest_d2sis so rarely used so it is nice to show a use case.have you tried to calculate the cell area?
See this gist for now: https://gist.github.com/JiaweiZhuang/fc2f860133174523051260817e75a99d
I encountered some problems when installing xesmf from area branch:
It's successful when xesmf is installed on my machine:
pip install --upgrade git+ssh://[email protected]/JiaweiZhuang/xESMF.git@area
Branch area set up to track remote branch area from origin.
Switched to a new branch 'area'
Requirement already satisfied, skipping upgrade: esmpy in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (7.1.0.dev0)
Requirement already satisfied, skipping upgrade: xarray in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (0.11.0)
Requirement already satisfied, skipping upgrade: numpy in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (1.15.4)
Requirement already satisfied, skipping upgrade: scipy in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (1.1.0)
Requirement already satisfied, skipping upgrade: pandas>=0.19.2 in ./Software/anaconda3/lib/python3.6/site-packages (from xarray->xesmf==0.1.1) (0.23.4)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in ./Software/anaconda3/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2.6.1)
Requirement already satisfied, skipping upgrade: pytz>=2011k in ./Software/anaconda3/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2017.3)
Requirement already satisfied, skipping upgrade: six>=1.5 in ./Software/anaconda3/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas>=0.19.2->xarray->xesmf==0.1.1) (1.11.0)
Building wheels for collected packages: xesmf
Running setup.py bdist_wheel for xesmf ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-asry4otw/wheels/86/a4/73/9f5efeec79d3e63db519ba88566cc5b5f469dae1469a2d7e9b
Successfully built xesmf
Installing collected packages: xesmf
Successfully installed xesmf-0.1.1
However, I got error when installing it on hpc:
pip install --upgrade git+ssh://[email protected]/JiaweiZhuang/xESMF.git@area
Collecting git+ssh://[email protected]/JiaweiZhuang/xESMF.git@area
Cloning ssh://[email protected]/JiaweiZhuang/xESMF.git (to revision area) to /tmp/pip-req-build-k16i3mzq
Branch 'area' set up to track remote branch 'area' from 'origin'.
Switched to a new branch 'area'
Collecting esmpy (from xesmf==0.1.1)
Could not find a version that satisfies the requirement esmpy (from xesmf==0.1.1) (from versions: )
No matching distribution found for esmpy (from xesmf==0.1.1)
I checked the installed list by conda list on hpc and found xesmf:
xesmf 0.1.1 <pip>
But, I couldn't remove it by conda remove --force xesmf or pip uninstall xesmf:
Solving environment: done
PackagesNotFoundError: The following packages are missing from the target environment:
- xesmf
Skipping xesmf as it is not installed.
from xesmf.
JiaweiZhuang
commented on May 26, 2024
@zxdawn Install ESMPy with conda conda install -c conda-forge esmpy and then install xESMF with pip. ESMPy is only installable from conda so pip cannot resolve the dependency.
from xesmf.
zxdawn
commented on May 26, 2024
conda install -c conda-forge esmpy
Actually, it has been installed:
Solving environment: done
# All requested packages already installed.
However, I tried to install xesmf again, it worked:
Collecting git+ssh://[email protected]/JiaweiZhuang/xESMF.git@area
Cloning ssh://[email protected]/JiaweiZhuang/xESMF.git (to revision area) to /tmp/pip-req-build-ufistfyw
Branch 'area' set up to track remote branch 'area' from 'origin'.
Switched to a new branch 'area'
Requirement already satisfied, skipping upgrade: esmpy in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (7.1.0.dev0)
Requirement already satisfied, skipping upgrade: xarray in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (0.11.0)
Requirement already satisfied, skipping upgrade: numpy in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (1.15.4)
Requirement already satisfied, skipping upgrade: scipy in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (1.2.0)
Requirement already satisfied, skipping upgrade: pandas>=0.19.2 in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xarray->xesmf==0.1.1) (0.24.0)
Requirement already satisfied, skipping upgrade: pytz>=2011k in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2018.9)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2.7.5)
Requirement already satisfied, skipping upgrade: six>=1.5 in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas>=0.19.2->xarray->xesmf==0.1.1) (1.12.0)
Building wheels for collected packages: xesmf
Running setup.py bdist_wheel for xesmf ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-37s9eb5s/wheels/86/a4/73/9f5efeec79d3e63db519ba88566cc5b5f469dae1469a2d7e9b
Successfully built xesmf
Installing collected packages: xesmf
Found existing installation: xesmf 0.1.1
Uninstalling xesmf-0.1.1:
Successfully uninstalled xesmf-0.1.1
Successfully installed xesmf-0.1.1
When I import xesmf in Python, I got this error:
Traceback (most recent call last):
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/loadESMF.py", line 118, in <module>
mode=ct.RTLD_GLOBAL)
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/libesmf_fullylinked.so: undefined symbol: __netcdf_MOD_nf90_get_var_eightbytereal
Traceback (most recent call last):
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/loadESMF.py", line 118, in <module>
mode=ct.RTLD_GLOBAL)
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/libesmf_fullylinked.so: undefined symbol: __netcdf_MOD_nf90_get_var_eightbytereal
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/xesmf/__init__.py", line 1, in <module>
from . import util
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/xesmf/util.py", line 4, in <module>
import ESMF
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/__init__.py", line 70, in <module>
from ESMF.api.esmpymanager import *
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/api/esmpymanager.py", line 11, in <module>
from ESMF.interface.cbindings import *
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/cbindings.py", line 13, in <module>
from ESMF.interface.loadESMF import _ESMF
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/loadESMF.py", line 121, in <module>
raise ImportError('The ESMF shared library did not load properly.')
ImportError: The ESMF shared library did not load properly.
After export LD_LIBRARY_PATH='', it works now.
So, I think it's related to library again ....
Thank you for your help :)
from xesmf.
JiaweiZhuang
commented on May 26, 2024
Yes ImportError: The ESMF shared library did not load properly. is an ESMPy problem. Your HPC system probably has a separate ESMF library installed and dynamically linked, so ESMPy finds the incorrect installation of ESMF.
from xesmf.
zxdawn
commented on May 26, 2024
@JiaweiZhuang How about the unit of xe.util.cell_area? I can't find it in the definition of get_area() and area().
I see ... It's 4*pi*(1^2)
Lines 104 to 107 in 3e46108
from xesmf.
Related Issues (20)
- Can I regrid between rectilinear grid and curvilinear grid?
- Install instructions to avoid ImportError failing? HOT 2
- regridder with dask is failing when multi-variables with differing dimensions HOT 1
- xesmf.Regridder(,,method='bilinear') issue
- After conda update, problem with cf_xarray: "Receive multiple variables for key 'longitude': ['i', 'lon']" HOT 4
- installation impossible with anaconda HOT 3
- MOAB support in ESMF 8.1.0 HOT 1
- repo not maintained see https://github.com/pangeo-data/xESMF
- ValueError: Dimensions {'bounds'} do not exist. Expected one or more of ('lon', 'bnds') HOT 1
- Example of interpolating 6 tiles cube-sphere grid to lat-lon grid HOT 2
- Error writing log file HOT 1
- xESMF regridding showing repeated values at latitude poles? HOT 2
- xESMF error after 12+ hour script run in downscaling grid/upsampling data HOT 2
- Misrepresentation of the number of precipitation events HOT 1
- ImportError: The ESMF shared library did not load properly. HOT 1
- How does xesmf treats grid cell with Nan values? HOT 1
- Hi, want to regrid wrfout (rainc+rainnc) data HOT 1
- ModuleNotFoundError: No module named 'ESMF' HOT 6
- ValueError: {'longitude', 'latitude'} are not variables in the underlying object. {'longitude', 'latitude'} are dimensions with no index.
- Using xesmf to efficiently regrid data to another resolution including vertical dimension
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xesmf.