Giter Club home page Giter Club logo

pyclim-sdm's Introduction

pyClim-SDM: Statistical Downscaling for Climate Change Projections with a Graphical User Interface





Institution: Spanish Meteorological Agency (AEMET)

License: GNU General Public License v3.0

Citation: Hernanz, A., Correa, C., García-Valero, J. A., Domínguez, M., Rodríguez-Guisado, E., & Rodríguez-Camino, E. (2023). pyClim-SDM: Service for generation of statistical downscaled climate change projections supporting national adaptation strategies. Climate Services. https://doi.org/10.1016/j.cliser.2023.100408

pyClim-SDM is a software for statistical downscaling of climate change projections for the following daily surface variables: mean, maximum and minimum temperature, precipitation, zonal and meridional wind components, relative and specific humidity, cloud cover, surface downwelling shortwave and longwave radiation, evaporation, potential evaporation, sea level pressure, surface pressure, total runoff and soil water content. Additionally, it is prepared for downscaling any other user defined variable. pyClim-SDM incorporates the following utilities:

  • evaluation of Global Climate Models (GCMs).
  • downscaling of both reanalysis (for evaluation) and GCMs.
  • bias correction of downscaled climate projections.
  • post-processing: ETCCDI extreme climate indices (Karl et al., 1999; https://www.climdex.org/learn/indices/).
  • visualization of downscaled projections and evaluation metrics.

Methods

Raw:

  • RAW: no downscaling (nearest grid point).
  • RAW-BIL: no downscaling (bilinear interpolation).

Model Output Statistics:

  • QM: Empirical Quantile Mapping (Themeßl et al., 2011).
  • DQM: Detrended Quantile Mapping (Cannon et al., 2015). Quantile adjustment over detrended series.
  • QDM: Quantile Delta Mapping in (Cannon et al., 2015). Delta change over quantiles.
  • PSDM: (Parametric) Scaled Distribution Mapping (Switanek et al., 2017).

Analogs / Weather Typing:

  • ANA-SYN: Analog based on synoptic analogy. 1NN: Nearest analog, kNN: k-nearest analogs, rand: random analog from Probability Density Function. See Hernanz et al. (2021).
  • ANA-LOC: Same as ANA-SYN but using synoptic+local analogy. See Petisco de Lara, (2008a), Amblar-Francés et al. (2017) and Hernanz et al. (2021).
  • ANA-VAR: Same as ANA-SYN but using the spatial pattern of the target variable itself.

Linear:

  • MLR: multiple linear regression. See Amblar-Francés et al., (2017) and Hernanz et al. (2021). Based on SDSM (Wilby et al., 2002).
  • MLR-ANA: multiple linear regression based on analogs. See Petisco de Lara (2008b), Amblar-Francés et al. (2017) and Hernanz et al. (2021).
  • MLR-WT: multiple linear regression based on weather types. Similar to ANA-MLR but using precalibrated relationships for each weather type.
  • GLM: Generalized Linear Model. Logistic + MLR (LIN), or over transformed data (EXP for exponential and CUB for cubic regression). See Amblar-Francés et al. (2017) and Hernanz et al. (2021). Based on SDSM (Wilby et al., 2002).

Machine Learning:

  • SVM: Support Vector Machine. Non-linear machine learning classification/regression. See Hernanz et al. (2021).
  • LS-SVM: Least Square Support Vector Machine. Non-linear machine learning classification/regression. See Hernanz et al. (2021).
  • RF: Random Forest. Non-linear machine learning classification/regression. This method is combined with a MLR to extrapolate to values out of the observed range (configurable).
  • XGB: eXtreme Gradient Boost. Non-linear machine learning classification/regression. This method is combined with a MLR to extrapolate to values out of the observed range (configurable).
  • ANN: Artificial Neural Networks. Non-linear machine learning classification/regression. See García-Valero (2021) and Hernanz et al. (2021).
  • CNN: Convolutional Neural Networks. Non-linear machine learning classification/regression.

Weather Generators:

  • WG-PDF: Downscaling parameters of the distributions instead of downscaling daily data. See Erlandsen et al. (2020) and Benestad (2021).
  • WG-NMM: Non-homogeneous Markov Model. Non-parametric Weather Generator based on a first-order two-state (wet/dry) Markov chain. Both the transition probabilities and the empirical distributions used for the intensity are conditioned on the precipitation given by the reanalysis/models. See Richardson (1981).

Installation

pyClim-SDM has been originally designed for Linux and might present problems over a different OS.

In order to use pyClim-SDM, python3 is required. pyClim-SDM makes use of the python libaries listed at requirements.txt. You can install them by following these steps:

  • Install Miniconda 3 (6Gb aprox. needed): https://docs.conda.io/en/latest/miniconda.html
  • Create a virtual environment: conda create -n env_pyClim-SDM -y -c conda-forge absl-py=1.2.0 numpy=1.23.1 pandas=1.4.3 geopandas=0.7.0 geopy=2.2.0 matplotlib=3.5.2 mpi4py=3.1.3 netCDF4=1.6.0 scikit-learn=1.1.2 scipy=1.9.0 seaborn=0.11.2 Shapely=1.8.2 statsmodels=0.13.2 tensorflow=2.7.0 xarray=0.17.0 xgboost=1.5.1 basemap=1.3.3 xarray
  • Activate your environment: conda activate env_pyClim-SDM

References

  • Amblar-Francés, P., Casado-Calle, M.J., Pastor-Saavedra, M.A., Ramos-Calzado, P., and Rodríguez-Camino, E. (2017). Guía de escenarios regionalizados de cambio climático sobre España a partir de los resultados del IPCC-AR5. Available at: https://www.aemet.es/documentos/es/conocermas/recursos_en_linea/publicaciones_y_estudios/publicaciones/Guia_escenarios_AR5/Guia_escenarios_AR5.pdf

  • Benestad, R.E. (2021) A Norwegian approach to downscaling. Geoscientific Model Development Discussion (in review). https://doi.org/10.5194/gmd-2021-176.

  • Cannon, A.J., S.R. Sobie, and T.Q. Murdock (2015). Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes?. J. Climate, 28, 6938–6959, https://doi.org/10.1175/JCLI-D-14-00754.1

  • Erlandsen, H.B., Parding, K.M., Benestad, R., Mezghani, A. and Pontoppidan, M. (2020). A hybrid downscaling approach for future temperature and precipitation change. Journal of Applied Meteorology and Climatology, 59(11), 1793–1807. https://doi.org/10.1175/JAMC-D-20-0013.1

  • García-Valero, J.A. (2021). Redes neuronales artificiales. Aplicación a la regionalización de la precipitación y temperaturas diarias. In: AEMET Nota Técnica, Área de Evaluación y Modelización del Cambio Climático. Spain: AEMET. https://www.aemet.es/documentos/es/conocermas/recursos_en_linea/publicaciones_y_estudios/publicaciones/NT_34_Redes_neuronales_artificiales/NT_34_Redes_neuronales_artificiales.pdf

  • Hernanz, A., García-Valero, J. A., Domínguez, M., Ramos-Calzado, P., Pastor-Saavedra, M. A. and Rodríguez-Camino, E. (2021). Evaluation of statistical downscaling methods for climate change projections over Spain: present conditions with perfect predictors. International Journal of Climatology, 42( 2), 762– 776. https://doi.org/10.1002/joc.7271

  • Karl, T.R., Nicholls, N., Ghazi, A. (1999). CLIVAR/GCOS/WMO workshop on indices and indicators for climate extremes. Workshop summary. Climatic Change 42, 3–7. https://doi.org/10.1007/978-94-015-9265-9_2

  • Petisco de Lara, S.E. (2008a). Método de regionalización de precipitación basado en análogos. Explicación y Validación. In: AEMET Nota Técnica 3A, Área de Evaluación y Modelización del Cambio Climático. Spain: AEMET.

  • Petisco de Lara, S.E. (2008b). Método de regionalización de temperatura basado en análogos. Explicación y Validación. In: AEMET Nota Técnica 3B, Área de Evaluación y Modelización del Cambio Climático. Spain: AEMET.

  • Richardson, C. W. (1981), Stochastic simulation of daily precipitation, temperature, and solar radiation, Water Resour. Res., 17( 1), 182– 190, https://doi.org/10.1029/WR017i001p0018

  • Switanek, M.B., Troch, P.A., Castro, C.L., Leuprecht, A., Chang, H.-I., Mukherjee, R., and Demaria, E.M.C. (2017). Scaled distribution mapping: a bias correction method that preserves raw climate model projected changes, Hydrol. Earth Syst. Sci., 21, 2649–2666, https://doi.org/10.5194/hess-21-2649-2017

  • Themeßl, M.J., Gobiet, A. and Leuprecht, A. (2011). Empirical-statistical downscaling and error correction of daily precipitation from regional climate models. Int. J. Climatol., 31: 1530-1544. https://doi.org/10.1002/joc.2168

  • Wilby, R., Dawson, C. and Barrow, E.M. (2002). SDSM—a decision support tool for the assessment of regional climate change impacts. Environmental Modelling & Software, 17, 145–157. https://doi.org/10.1016/S1364-8152(01)00060-3

pyclim-sdm's People

Contributors

ahernanzl avatar carloscorreag avatar

Stargazers

Andrew Clelland avatar LMXB avatar avatar Hadri_Anas avatar avatar LE Xinlong avatar avatar Kostas Kougioumoutzis avatar Nolan Townsend avatar Bastian Morales avatar lixin avatar avatar Benjamin avatar Trevor James Smith avatar Zhihao LIU avatar Crhistian Cornejo avatar avatar avatar avatar Tobias Augspurger avatar Ekaterina avatar Brayan Torres Z avatar avatar Javed Ali avatar Amides Figueroa avatar

Watchers

Amides Figueroa avatar avatar

Forkers

weilin2018 l5d1l5 carloscorreag shiph0326 aydinbakhtar zejiang-unsw minchiangrunner yangtao4753 chandra04 khalil628 jimjoker

pyclim-sdm's Issues

Pyclim-SDM:variable h in p_hres_metadata and t_hres_metadata

Hi,
I have few queries in using the PyClim-SDM model. The Input_data_template contains two files p_hres_metadata and t_hres_metadata with the variables id,lon,lat and h. May I know what h stands for and what is its unit . Also, p_hres_metadata has data for the variable 'h' and its not given in t_hres_metadata. So, is it mandatory to include data for h in these metadata files. Kindly clarify in this regard.

Error in down_scene_RAW.py

I am trying to downscale ERA5 data to particular stations but I am encountering an error. May I request your guidance on resolving it?

--------------------------------------
collect chunks 1
tasmax RAW TESTING reanalysis
--------------------
ichunk:	 0 / 1
downscaling tasmax RAW TESTING reanalysis 0.0 %
Traceback (most recent call last):
  File "/home/ltonga/workdir/pyClim-SDM/src/.tmp_main.py", line 15, in <module>
    main()
  File "/home/ltonga/workdir/pyClim-SDM/src/.tmp_main.py", line 11, in main
    process.downscale()
  File "/home/ltonga/workdir/pyClim-SDM/src/../lib/process.py", line 182, in downscale
    reanalisys(method_dict, scene, model)
  File "/home/ltonga/workdir/pyClim-SDM/src/../lib/process.py", line 56, in reanalisys
    down_scene_RAW.downscale_chunk(targetVar, methodName, family, mode, fields, scene, model)
  File "/home/ltonga/workdir/pyClim-SDM/src/../lib/down_scene_RAW.py", line 126, in downscale_chunk
    est[:, ipoint_local_index] = 100 * X_test[:, 0] # Factor 100 is for coherency with other methods
    ~~~^^^^^^^^^^^^^^^^^^^^^^^
ValueError: could not broadcast input array from shape (1096,) into shape (7305,)

Thank you very much. Please let me know if additional information about my input files are needed.

How WG treats missing hres data

Hello,

When I use the WG-NMM method to downscale precipitation, I noticed that when loading the hres data and and saving it as an npy file, the missing data which are initially -999 become 42949672.95. This messes up with the resulting ECDFs and I would like some guidance how I can properly handle missing data.

For illustration, below is the actual CDF of the observed station data.
image
But when it is processed through the WG_lib and saved as obs_0.npy, this occurs.
image
This results to very weird downscaled values for precipitation.

Any guidance will be much appreciated. Thank you very much for your time.

Sincerely,
Leia Pauline

Error while running the model

Hi Sir,
I have used pyClim SDM for running Preprocess of the model. I am getting the following error:

grids.association() starts tasmax grids.association nearest 0.0 % tasmax grids.association nearest 54.6448087431694 % grids.association() starts tasmax grids.association bilinear 0.0 % TypeError: only size-1 arrays can be converted to Python scalars The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/mnt/d/pyClim-SDM-master/src/.tmp_main.py", line 13, in main() File "/mnt/d/pyClim-SDM-master/src/.tmp_main.py", line 9, in main preprocess.preprocess() File "/mnt/d/pyClim-SDM-master/src/../lib/preprocess.py", line 43, in preprocess common() File "/mnt/d/pyClim-SDM-master/src/../lib/preprocess.py", line 59, in common grids.association(interp_mode, targetVar) File "/mnt/d/pyClim-SDM-master/src/../lib/grids.py", line 169, in association df_association['i1'].iloc[ipoint] = ilatUp File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/indexing.py", line 716, in setitem iloc._setitem_with_indexer(indexer, value, self.name) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/indexing.py", line 1690, in _setitem_with_indexer self._setitem_single_block(indexer, value, name) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/indexing.py", line 1938, in _setitem_single_block self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 337, in setitem return self.apply("setitem", indexer=indexer, value=value) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 304, in apply applied = getattr(b, f)(**kwargs) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/internals/blocks.py", line 955, in setitem values[indexer] = value ValueError: setting an array element with a sequence.

Please help me to sort it out

Run GUI mode

Hi Dear
when I run gui_mode.py with python .\gui_mode.py, I get an error.
'cp' is not recognized as an internal or external command, operable program, or batch file.
What should I do to fix this issue?
Thank you

software

Thanks for your help, how can I get the software which it was mentioned as "pyClim-SDM" in the package?

Issue regarding spatial downscaling

Buenos días,

Estoy intentando generar un downscaling de distintos modelos climáticos utilizando el reanálisis ERA5 de forma espacial. He intentado generarlo con los datasets de prueba que hay, pero continuamente obtengo resultados de forma puntual. ¿Cómo sería posible generar un downscaling de forma espacial, es decir, en una malla?

Gracias y un saludo,

Issue regarding to dates in NETCDF

Hello !

I´ve been modifying the input data, but I still have the same problem during missing data check:

image

I tryed to understad source scripts, but i'm not sure if problem is regarding to model or reanalysis data.

testing error

I am testing this useful package on mac but even though numpy installed I am getting below error:

python gui_mode.py
Traceback (most recent call last):
File "/Users/sarr/pyClim-SDM/src/gui_mode.py", line 36, in
from imports import *
File "/Users/sarr/pyClim-SDM/src/../config/imports.py", line 9, in
import numpy as np
ModuleNotFoundError: No module named 'numpy'

Any hint?

Adding direct predictors

Hello, for some derived predictors (e.g. K index and Total totals index), I already have the actual values downloaded from ERA5. I want to use them directly instead of deriving them, but I can't figure out how to do so on my own. May I request your guidance on this matter? Thank you very much.

Sincerely,
Leia Pauline

Error in linux subsystem for windows

Can we run it in linux subsystem for windows ? because I am getting these errors

"python gui_mode.py

ERROR 1: PROJ: proj_create_from_database: Open of /home/rashid/miniconda3/envs/env/share/proj failed
Synoptic domain incompatible with coordinates in input files. saf_lat_up forced from 49.0 to 46.0
Synoptic domain incompatible with coordinates in input files. saf_lat_down forced from 29.5 to 34.0
Synoptic domain incompatible with coordinates in input files. saf_lon_left forced from -18.0 to -15.0
Synoptic domain incompatible with coordinates in input files. saf_lon_right forced from 12.0 to 9.0
Traceback (most recent call last):
File "/mnt/d/pyClim-SDM-master/pyClim-SDM-master/src/gui_mode.py", line 3441, in
main()
File "/mnt/d/pyClim-SDM-master/pyClim-SDM-master/src/gui_mode.py", line 3438, in main
selectionWindow()
File "/mnt/d/pyClim-SDM-master/pyClim-SDM-master/src/gui_mode.py", line 2917, in init
run, self.showWelcomeMessage = welcomeMessage().get()
File "/mnt/d/pyClim-SDM-master/pyClim-SDM-master/src/gui_mode.py", line 175, in init
root = tk.Tk()
File "/home/rashid/miniconda3/envs/env/lib/python3.9/tkinter/init.py", line 2270, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.