Comments (12)
FabianHofmann
commented on July 21, 2024
1
Thanks for reporting @Tasqu, I will look into it.
from atlite.
Tasqu
commented on July 21, 2024
I briefly tested the historic-comparison-germany.ipynb example as well just to be safe, and it seems to suffer from the same problem. At least on my machine.
from atlite.
FabianHofmann
commented on July 21, 2024
@Tasqu I cannot reproduce the bug. All works fine for me, having xarray 2023.09.0 and dask 2023.10.0 installed. I am seeing that you have mixed installations from pypi and conda-forge. I would recommend to create a new environment with conda/mamba and retest.
from atlite.
Tasqu
commented on July 21, 2024
@FabianHofmann Ok, good to know. I'll have to try different environment setups, hopefully that solves this. I'll try to get back to you later today.
from atlite.
Tasqu
commented on July 21, 2024
@FabianHofmann meanwhile, I can't seem to get rid of this error 😅 I tried creating new conda environments using solely conda to install the packages for both Python 3.11 and 3.9, but I still keep running in (almost) the same errors. Here's the latest conda environment I used for testing this out:
# packages in environment at [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite):
#
# Name Version Build Channel
affine 2.4.0 pyhd8ed1ab_0 conda-forge
asttokens 2.4.0 pyhd8ed1ab_0 conda-forge
atlite 0.2.11 pyhd8ed1ab_0 conda-forge
attrs 23.1.0 pyh71513ae_1 conda-forge
aws-c-auth 0.7.4 hd571b1d_2 conda-forge
aws-c-cal 0.6.2 hd5965a7_2 conda-forge
aws-c-common 0.9.3 hcfcfb64_0 conda-forge
aws-c-compression 0.2.17 hd5965a7_3 conda-forge
aws-c-event-stream 0.3.2 hef061cf_2 conda-forge
aws-c-http 0.7.13 h064cb6d_2 conda-forge
aws-c-io 0.13.33 ha16e049_0 conda-forge
aws-c-mqtt 0.9.7 h37bb463_0 conda-forge
aws-c-s3 0.3.18 h263813f_0 conda-forge
aws-c-sdkutils 0.1.12 hd5965a7_2 conda-forge
aws-checksums 0.1.17 hd5965a7_2 conda-forge
aws-crt-cpp 0.24.3 hd745c5a_1 conda-forge
aws-sdk-cpp 1.11.156 hc1d519e_5 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge
blas 1.0 mkl
blosc 1.21.5 hdccc3a2_0 conda-forge
bokeh 3.3.0 pyhd8ed1ab_0 conda-forge
bottleneck 1.3.7 py39hd88c2e4_1 conda-forge
branca 0.6.0 pyhd8ed1ab_0 conda-forge
brotli 1.1.0 hcfcfb64_1 conda-forge
brotli-bin 1.1.0 hcfcfb64_1 conda-forge
brotli-python 1.1.0 py39h99910a6_1 conda-forge
bzip2 1.0.8 h8ffe710_4 conda-forge
c-ares 1.20.1 hcfcfb64_0 conda-forge
ca-certificates 2023.7.22 h56e8100_0 conda-forge
cairo 1.18.0 h1fef639_0 conda-forge
cdsapi 0.6.1 pyhd8ed1ab_0 conda-forge
certifi 2023.7.22 pyhd8ed1ab_0 conda-forge
cfitsio 4.3.0 h9b0cee5_0 conda-forge
cftime 1.6.2 py39hd88c2e4_2 conda-forge
charset-normalizer 3.3.0 pyhd8ed1ab_0 conda-forge
click 8.1.7 win_pyh7428d3b_0 conda-forge
click-plugins 1.1.1 py_0 conda-forge
cligj 0.7.2 pyhd8ed1ab_1 conda-forge
cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.6 pyhd8ed1ab_0 conda-forge
comm 0.1.4 pyhd8ed1ab_0 conda-forge
contourpy 1.1.1 py39h1f6ef14_1 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
cytoolz 0.12.2 py39ha55989b_1 conda-forge
dask 2023.10.0 pyhd8ed1ab_0 conda-forge
dask-core 2023.10.0 pyhd8ed1ab_0 conda-forge
debugpy 1.8.0 py39h99910a6_1 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
distributed 2023.10.0 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.1.3 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
expat 2.5.0 h63175ca_1 conda-forge
fiona 1.9.5 py39h48fce2b_0 conda-forge
folium 0.14.0 pyhd8ed1ab_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 hbde0cde_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.43.1 py39ha55989b_0 conda-forge
freetype 2.12.1 hdaf720e_2 conda-forge
freexl 2.0.0 h8276f4a_0 conda-forge
fsspec 2023.9.2 pyh1a96a4e_0 conda-forge
gdal 3.7.2 py39hbe60bc6_7 conda-forge
geopandas 0.14.0 pyhd8ed1ab_1 conda-forge
geopandas-base 0.14.0 pyha770c72_1 conda-forge
geos 3.12.0 h1537add_0 conda-forge
geotiff 1.7.1 hcf4a93f_14 conda-forge
gettext 0.21.1 h5728263_0 conda-forge
glib 2.78.0 h12be248_0 conda-forge
glib-tools 2.78.0 h12be248_0 conda-forge
gst-plugins-base 1.22.6 h001b923_2 conda-forge
gstreamer 1.22.6 hb4038d2_2 conda-forge
hdf4 4.2.15 h5557f11_7 conda-forge
hdf5 1.14.2 nompi_h73e8ff5_100 conda-forge
icu 73.2 h63175ca_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.8.0 pyha770c72_0 conda-forge
importlib-resources 6.1.0 pyhd8ed1ab_0 conda-forge
importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge
importlib_resources 6.1.0 pyhd8ed1ab_0 conda-forge
iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2023.2.0 h57928b3_50496 conda-forge
ipykernel 6.25.2 pyh60829e3_0 conda-forge
ipython 8.16.1 pyh5737063_0 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
joblib 1.3.2 pyhd8ed1ab_0 conda-forge
jupyter_client 8.4.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.4.0 py39hcbf5309_0 conda-forge
kealib 1.5.2 ha10e780_1 conda-forge
kiwisolver 1.4.5 py39h1f6ef14_1 conda-forge
krb5 1.21.2 heb0366b_0 conda-forge
lcms2 2.15 h67d730c_3 conda-forge
lerc 4.0.0 h63175ca_0 conda-forge
libabseil 20230802.1 cxx17_h63175ca_0 conda-forge
libaec 1.1.2 h63175ca_1 conda-forge
libarchive 3.7.2 h6f8411a_0 conda-forge
libarrow 13.0.0 h64a251c_10_cpu conda-forge
libblas 3.9.0 19_win64_mkl conda-forge
libboost-headers 1.82.0 h57928b3_6 conda-forge
libbrotlicommon 1.1.0 hcfcfb64_1 conda-forge
libbrotlidec 1.1.0 hcfcfb64_1 conda-forge
libbrotlienc 1.1.0 hcfcfb64_1 conda-forge
libcblas 3.9.0 19_win64_mkl conda-forge
libclang 15.0.7 default_h77d9078_3 conda-forge
libclang13 15.0.7 default_h77d9078_3 conda-forge
libcrc32c 1.1.2 h0e60522_0 conda-forge
libcurl 8.4.0 hd5e4a3a_0 conda-forge
libdeflate 1.19 hcfcfb64_0 conda-forge
libevent 2.1.12 h3671451_1 conda-forge
libexpat 2.5.0 h63175ca_1 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
libgdal 3.7.2 h3217549_7 conda-forge
libglib 2.78.0 he8f3873_0 conda-forge
libgoogle-cloud 2.12.0 ha74b051_3 conda-forge
libgrpc 1.58.1 h2a9c87f_2 conda-forge
libhwloc 2.9.3 default_haede6df_1009 conda-forge
libiconv 1.17 h8ffe710_0 conda-forge
libjpeg-turbo 3.0.0 hcfcfb64_1 conda-forge
libkml 1.3.0 haf3e7a6_1018 conda-forge
liblapack 3.9.0 19_win64_mkl conda-forge
libnetcdf 4.9.2 nompi_h8284064_112 conda-forge
libogg 1.3.4 h8ffe710_1 conda-forge
libpng 1.6.39 h19919ed_0 conda-forge
libpq 16.0 h43585b0_1 conda-forge
libprotobuf 4.24.3 hb8276f3_1 conda-forge
libre2-11 2023.06.02 h8c5ae5e_0 conda-forge
librttopo 1.1.0 h92c5fdb_14 conda-forge
libsodium 1.0.18 h8d14728_1 conda-forge
libspatialindex 1.9.3 h39d44d4_4 conda-forge
libspatialite 5.1.0 hbf340bc_0 conda-forge
libsqlite 3.43.2 hcfcfb64_0 conda-forge
libssh2 1.11.0 h7dfc565_0 conda-forge
libthrift 0.19.0 ha2b3283_1 conda-forge
libtiff 4.6.0 h6e2ebb7_2 conda-forge
libutf8proc 2.8.0 h82a8f57_0 conda-forge
libvorbis 1.3.7 h0e60522_0 conda-forge
libwebp-base 1.3.2 hcfcfb64_0 conda-forge
libxcb 1.15 hcd874cb_0 conda-forge
libxml2 2.11.5 hc3477c8_1 conda-forge
libzip 1.10.1 h1d365fa_3 conda-forge
libzlib 1.2.13 hcfcfb64_5 conda-forge
locket 1.0.0 pyhd8ed1ab_0 conda-forge
lz4 4.3.2 py39hf617134_1 conda-forge
lz4-c 1.9.4 hcfcfb64_0 conda-forge
lzo 2.10 he774522_1000 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
mapclassify 2.6.1 pyhd8ed1ab_0 conda-forge
markupsafe 2.1.3 py39ha55989b_1 conda-forge
matplotlib 3.8.0 py39hcbf5309_2 conda-forge
matplotlib-base 3.8.0 py39hf19769e_2 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
minizip 4.0.1 h5bed578_5 conda-forge
mkl 2023.2.0 h6a75c08_50496 conda-forge
mkl-service 2.4.0 py39h2bbff1b_1
msgpack-python 1.0.6 py39h1f6ef14_0 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
munch 4.0.0 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nest-asyncio 1.5.8 pyhd8ed1ab_0 conda-forge
netcdf4 1.6.4 nompi_py39h9a3bb69_103 conda-forge
networkx 3.1 pyhd8ed1ab_0 conda-forge
numexpr 2.8.7 py39h2cd9be0_0
numpy 1.26.0 py39hddb5d58_0 conda-forge
openjpeg 2.5.0 h3d672ee_3 conda-forge
openssl 3.1.3 hcfcfb64_0 conda-forge
orc 1.9.0 hd95f75e_3 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandas 2.1.1 py39h32e6231_1 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.4.1 pyhd8ed1ab_0 conda-forge
pcre2 10.40 h17e33f8_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.1.0 py39h368b509_0 conda-forge
pip 23.3 pyhd8ed1ab_0 conda-forge
pixman 0.42.2 h63175ca_0 conda-forge
platformdirs 3.11.0 pyhd8ed1ab_0 conda-forge
pluggy 1.3.0 pyhd8ed1ab_0 conda-forge
ply 3.11 py_1 conda-forge
poppler 23.10.0 hc2f3c52_0 conda-forge
poppler-data 0.4.12 hd8ed1ab_0 conda-forge
postgresql 16.0 hc80876b_1 conda-forge
progressbar2 4.2.0 pyhd8ed1ab_0 conda-forge
proj 9.3.0 he13c7e8_1 conda-forge
prompt-toolkit 3.0.39 pyha770c72_0 conda-forge
prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge
psutil 5.9.5 py39ha55989b_1 conda-forge
pthread-stubs 0.4 hcd874cb_1001 conda-forge
pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pyarrow 13.0.0 py39h0a09291_10_cpu conda-forge
pygments 2.16.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pyproj 3.6.1 py39h9e31499_2 conda-forge
pyqt 5.15.9 py39hb77abff_5 conda-forge
pyqt5-sip 12.12.2 py39h99910a6_5 conda-forge
pysocks 1.7.1 pyh0701188_6 conda-forge
pytest 7.4.2 pyhd8ed1ab_0 conda-forge
python 3.9.18 h4de0772_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge
python-utils 3.8.1 pyhd8ed1ab_0 conda-forge
python_abi 3.9 4_cp39 conda-forge
pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge
pywin32 306 py39h99910a6_2 conda-forge
pyyaml 6.0.1 py39ha55989b_1 conda-forge
pyzmq 25.1.1 py39hea35a22_1 conda-forge
qt-main 5.15.8 h9e85ed6_17 conda-forge
rasterio 1.3.8 py39hdc4e632_4 conda-forge
re2 2023.06.02 hcbb65ff_0 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
rioxarray 0.15.0 pyhd8ed1ab_0 conda-forge
rtree 1.1.0 py39h09fdee3_0 conda-forge
scikit-learn 1.3.1 py39h7c199eb_1 conda-forge
scipy 1.11.3 py39hddb5d58_1 conda-forge
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
shapely 2.0.2 py39hacc7841_0 conda-forge
sip 6.7.12 py39h99910a6_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.10 hfb803bf_0 conda-forge
snuggs 1.4.7 py_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
sqlite 3.43.2 hcfcfb64_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
tbb 2021.10.0 h91493d7_1 conda-forge
tblib 2.0.0 pyhd8ed1ab_0 conda-forge
threadpoolctl 3.2.0 pyha21a80b_0 conda-forge
tiledb 2.16.3 h1ffc264_3 conda-forge
tk 8.6.13 hcfcfb64_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
toolz 0.12.0 pyhd8ed1ab_0 conda-forge
tornado 6.3.3 py39ha55989b_1 conda-forge
tqdm 4.66.1 pyhd8ed1ab_0 conda-forge
traitlets 5.11.2 pyhd8ed1ab_0 conda-forge
typing-extensions 4.8.0 hd8ed1ab_0 conda-forge
typing_extensions 4.8.0 pyha770c72_0 conda-forge
tzdata 2023c h71feb2d_0 conda-forge
ucrt 10.0.22621.0 h57928b3_0 conda-forge
unicodedata2 15.1.0 py39ha55989b_0 conda-forge
uriparser 0.9.7 h1537add_1 conda-forge
urllib3 2.0.6 pyhd8ed1ab_0 conda-forge
vc 14.3 h64f974e_17 conda-forge
vc14_runtime 14.36.32532 hdcecf7f_17 conda-forge
vs2015_runtime 14.36.32532 h05e6639_17 conda-forge
wcwidth 0.2.8 pyhd8ed1ab_0 conda-forge
wheel 0.41.2 pyhd8ed1ab_0 conda-forge
win_inet_pton 1.1.0 pyhd8ed1ab_6 conda-forge
xarray 2023.9.0 pyhd8ed1ab_0 conda-forge
xerces-c 3.2.4 h63175ca_3 conda-forge
xorg-libxau 1.0.11 hcd874cb_0 conda-forge
xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge
xyzservices 2023.10.0 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h8d14728_0 conda-forge
yaml 0.2.5 h8ffe710_2 conda-forge
zeromq 4.3.4 h0e60522_1 conda-forge
zict 3.0.0 pyhd8ed1ab_0 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 hcfcfb64_5 conda-forge
zstd 1.5.5 h12be248_0 conda-forgeWhen trying to run building_stock_weather_aggregation.ipynb with the following cutout (spans 2 months)
cutout = atlite.Cutout(
path="finland-2011-0607.nc",
module="era5",
x=slice(x1 - 0.2, x2 + 0.2),
y=slice(y1 - 0.2, y2 + 0.2),
time=slice("2011-06","2011-07"),
)I get a new error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
[c:\atlite\examples\building_stock_weather_aggregation.ipynb](file:///C:/atlite/examples/building_stock_weather_aggregation.ipynb) Cell 18 line 1
[13](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=12) dirs = {"north": 0.0, "east": 90.0, "south": 180.0, "west": 270.0}
[14](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=13) for name, lout in layouts.items():
---> [15](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=14) irr_total[name] = {
[16](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=15) d: cutout.irradiation(
[17](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=16) orientation={"slope": 90.0, "azimuth": az}, layout=lout.fillna(0.0)
[18](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=17) )
[19](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=18) .squeeze()
[20](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=19) .to_series()
[21](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=20) for d, az in dirs.items()
[22](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=21) }
[23](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=22) irr_direct[name] = {
[24](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=23) d: cutout.irradiation(
[25](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=24) orientation={"slope": 90.0, "azimuth": az},
(...)
[31](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=30) for d, az in dirs.items()
[32](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=31) }
[33](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=32) irr_diffuse[name] = {
[34](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=33) d: cutout.irradiation(
[35](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=34) orientation={"slope": 90.0, "azimuth": az},
(...)
[41](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=40) for d, az in dirs.items()
[42](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=41) }
[c:\atlite\examples\building_stock_weather_aggregation.ipynb](file:///C:/atlite/examples/building_stock_weather_aggregation.ipynb) Cell 18 line 1
[13](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=12) dirs = {"north": 0.0, "east": 90.0, "south": 180.0, "west": 270.0}
[14](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=13) for name, lout in layouts.items():
[15](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=14) irr_total[name] = {
---> [16](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=15) d: cutout.irradiation(
[17](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=16) orientation={"slope": 90.0, "azimuth": az}, layout=lout.fillna(0.0)
[18](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=17) )
[19](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=18) .squeeze()
[20](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=19) .to_series()
[21](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=20) for d, az in dirs.items()
[22](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=21) }
[23](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=22) irr_direct[name] = {
[24](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=23) d: cutout.irradiation(
[25](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=24) orientation={"slope": 90.0, "azimuth": az},
(...)
[31](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=30) for d, az in dirs.items()
[32](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=31) }
[33](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=32) irr_diffuse[name] = {
[34](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=33) d: cutout.irradiation(
[35](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=34) orientation={"slope": 90.0, "azimuth": az},
(...)
[41](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=40) for d, az in dirs.items()
[42](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=41) }
AttributeError: 'Cutout' object has no attribute 'irradiation'As if the irradiation attribute is missing? However, when trying to run a 3-month cutout:
cutout = atlite.Cutout(
path="finland-2011-0608.nc",
module="era5",
x=slice(x1 - 0.2, x2 + 0.2),
y=slice(y1 - 0.2, y2 + 0.2),
time=slice("2011-06","2011-08"),
)I still get the same error as before:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
[c:\atlite\examples\building_stock_weather_aggregation.ipynb](file:///C:/atlite/examples/building_stock_weather_aggregation.ipynb) Cell 18 line 6
[3](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=2) layouts = {"Weighted": layout, "Uniform": uniform, "Jyväskylä": jyvaskyla}
[4](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=3) for name, lout in layouts.items():
[5](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=4) temperature_dict[name] = (
----> [6](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=5) cutout.temperature(layout=lout.fillna(0.0)).squeeze().to_series()
[7](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=6) )
[9](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=8) # Calculate radiation data into dictionaries
[10](vscode-notebook-cell:/c%3A/atlite/examples/building_stock_weather_aggregation.ipynb#X23sZmlsZQ%3D%3D?line=9) irr_total = {}
File [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite\lib\site-packages\atlite\convert.py:216](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite/lib/site-packages/atlite/convert.py:216), in temperature(cutout, **params)
215 def temperature(cutout, **params):
--> 216 return cutout.convert_and_aggregate(convert_func=convert_temperature, **params)
File [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite\lib\site-packages\atlite\convert.py:174](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite/lib/site-packages/atlite/convert.py:174), in convert_and_aggregate(cutout, convert_func, matrix, index, layout, shapes, shapes_crs, per_unit, return_capacity, capacity_factor, show_progress, dask_kwargs, **convert_kwds)
171 if index is None:
172 index = pd.RangeIndex(matrix.shape[0])
--> 174 results = aggregate_matrix(da, matrix=matrix, index=index)
176 if per_unit or return_capacity:
177 caps = matrix.sum(-1)
File [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite\lib\site-packages\atlite\aggregate.py:19](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite/lib/site-packages/atlite/aggregate.py:19), in aggregate_matrix(da, matrix, index)
17 if isinstance(da.data, dask.array.core.Array):
18 da = da.stack(spatial=("y", "x"))
---> 19 return xr.apply_ufunc(
20 lambda da: da * matrix.T,
21 da,
22 input_core_dims=[["spatial"]],
23 output_core_dims=[[index.name]],
24 dask="parallelized",
25 output_dtypes=[da.dtype],
26 dask_gufunc_kwargs=dict(output_sizes={index.name: index.size}),
27 ).assign_coords(**{index.name: index})
28 else:
29 da = da.stack(spatial=("y", "x")).transpose("spatial", "time")
File [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite\lib\site-packages\xarray\core\computation.py:1249](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite/lib/site-packages/xarray/core/computation.py:1249), in apply_ufunc(func, input_core_dims, output_core_dims, exclude_dims, vectorize, join, dataset_join, dataset_fill_value, keep_attrs, kwargs, dask, output_dtypes, output_sizes, meta, dask_gufunc_kwargs, on_missing_core_dim, *args)
1247 # feed DataArray apply_variable_ufunc through apply_dataarray_vfunc
1248 elif any(isinstance(a, DataArray) for a in args):
-> 1249 return apply_dataarray_vfunc(
1250 variables_vfunc,
1251 *args,
1252 signature=signature,
1253 join=join,
1254 exclude_dims=exclude_dims,
1255 keep_attrs=keep_attrs,
1256 )
1257 # feed Variables directly through apply_variable_ufunc
1258 elif any(isinstance(a, Variable) for a in args):
File [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite\lib\site-packages\xarray\core\computation.py:308](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite/lib/site-packages/xarray/core/computation.py:308), in apply_dataarray_vfunc(func, signature, join, exclude_dims, keep_attrs, *args)
303 result_coords, result_indexes = build_output_coords_and_indexes(
304 args, signature, exclude_dims, combine_attrs=keep_attrs
305 )
307 data_vars = [getattr(a, "variable", a) for a in args]
--> 308 result_var = func(*data_vars)
310 out: tuple[DataArray, ...] | DataArray
311 if signature.num_outputs > 1:
File [c:\Users\trtopi\AppData\Local\miniconda3\envs\atlite\lib\site-packages\xarray\core\computation.py:754](file:///C:/Users/trtopi/AppData/Local/miniconda3/envs/atlite/lib/site-packages/xarray/core/computation.py:754), in apply_variable_ufunc(func, signature, exclude_dims, dask, output_dtypes, vectorize, keep_attrs, dask_gufunc_kwargs, *args)
752 for axis, dim in enumerate(core_dims, start=-len(core_dims)):
753 if len(data.chunks[axis]) != 1:
--> 754 raise ValueError(
755 f"dimension {dim} on {n}th function argument to "
756 "apply_ufunc with dask='parallelized' consists of "
757 "multiple chunks, but is also a core dimension. To "
758 "fix, either rechunk into a single array chunk along "
759 f"this dimension, i.e., ``.chunk(dict({dim}=-1))``, or "
760 "pass ``allow_rechunk=True`` in ``dask_gufunc_kwargs`` "
761 "but beware that this may significantly increase memory usage."
762 )
763 dask_gufunc_kwargs["allow_rechunk"] = True
765 output_sizes = dask_gufunc_kwargs.pop("output_sizes", {})
ValueError: dimension spatial on 0th function argument to apply_ufunc with dask='parallelized' consists of multiple chunks, but is also a core dimension. To fix, either rechunk into a single array chunk along this dimension, i.e., ``.chunk(dict(spatial=-1))``, or pass ``allow_rechunk=True`` in ``dask_gufunc_kwargs`` but beware that this may significantly increase memory usage.I deleted and re-downloaded the weather data .nc files just in case, but that didn't seem to make a difference. (Not sure why it would, but hey, worth a shot?)
I'll probably keep trying to mess with my environments to see if this goes away somehow...
from atlite.
davide-f
commented on July 21, 2024
To add on this issue, I've encountered this issue as well, which may occur with large cutouts.
This is linked to pypsa-meets-earth/pypsa-earth#900 as well.
from atlite.
Tasqu
commented on July 21, 2024
@davide-f thanks for the tip! Seems like downgrading to xarray=2023.8.0 also avoids this error, but as far as I can tell it results in similar performance than passing allow_rechunk=True in dask_gufunc_kwargs (as suggested by the error messages) for the aggregate_matrix function.
Both solutions allow me to bypass this issue, but the performance seems worse than I remember it being last time I was dealing with country-wide year-long weather data aggregation. It seems to me that every time the aggregate_matrix function is called, it does some re-chunking of the underlying data, losing time with repeated cutout.temperature(...) etc. calls.
from atlite.
FabianHofmann
commented on July 21, 2024
@Tasqu could you do us a favour and test the implementation in #326?
from atlite.
Tasqu
commented on July 21, 2024
@FabianHofmann sure thing, I'll give it a go.
from atlite.
Tasqu
commented on July 21, 2024
@FabianHofmann yeah, #326 seems to work. As far as I can tell, it's working the same way as downgrading to xarray=2023.8.0 or passing the allow_rechunk=True. The performance for repeated cutout.temperature etc. still isn't what I remember it being for long timespans, but that might be related to #252.
Or maybe I'm just imagining things 😅 it has been several months since I last did full-year weather data aggregation over entire countries.
from atlite.
coroa
commented on July 21, 2024
Completely different thing you can try with those dask heavy internals is to run with the dask.distributed TaskScheduler instead of the default dask local, which has slightly different characteristics, which makes it a bit more robust.
Concretely, this means in your script or notebook that you add:
from dask.distributed import Client
client = Client() # set up local cluster on your laptopas described at
https://distributed.dask.org/en/stable/quickstart.html#setup-dask-distributed-the-easy-way
before you open your cutout object.
from atlite.
Tasqu
commented on July 21, 2024
@coroa Thanks for the tip! In my use case, this cuts the weather aggregation time roughly in half. Preparing the layout seems to take a little longer, but still seems like a worthwhile gain overall.
distributed throws a ton of garbage collection and worker unresponsiveness related warnings, but still worth investigating further 😅
from atlite.
Related Issues (20)
- hydro not compatible with normalize_using_yearly option HOT 2
- Add heuristic for ERA5 download chunk sizes HOT 3
- Incorrect unit conversion in hydro inflow shift
- ERA5 Solar Position Time Shift Broken for Certain Time Spans HOT 5
- Cutout AttributeError: "EntryPoints" object has no attribute get in Colab HOT 2
- Question about runoff conversion HOT 3
- Include xarray-spatial
- pad_extent leads to rasterio error for global scape
- Licence description on PyPI incorrect HOT 1
- Error cannot convert float nan to int HOT 10
- Issue with build_cutout using alite HOT 5
- reanalysis-era5-single-levels HOT 3
- Atlite errors with ESRI:540060 reprojections and Fiji HOT 7
- Read from url for `excluder.add_raster` and `excluder.add_geometry` HOT 2
- PV conversion: New model based on Bloomfield et al. (2019)
- Wind Conversion: potential bug when power curve does not end with zero after cutout speed HOT 1
- Weather/climate data variable descriptions (for alternate model data use) HOT 2
- Data type error when building cutout with SARAH v3 HOT 4
- Merging cutouts / Integrate downloaded SARAH data into existing ERA-5 cutout HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from atlite.