The nwpeval from debasish-mahapatra

NWPeval

NWPeval is a Python package designed to facilitate the evaluation and analysis of numerical weather prediction (NWP) models. It provides a comprehensive set of metrics and tools to assess the performance of NWP models by comparing their output with observed weather data.

Features

Supports a wide range of evaluation metrics, including:
- Mean Absolute Error (MAE)
- Root Mean Square Error (RMSE)
- Anomaly Correlation Coefficient (ACC)
- Fractions Skill Score (FSS)
- Equitable Threat Score (ETS)
- Probability of Detection (POD)
- False Alarm Ratio (FAR)
- Critical Success Index (CSI)
- Brier Skill Score (BSS)
- Heidke Skill Score (HSS)
- Peirce Skill Score (PSS)
- Symmetric Extreme Dependency Score (SEDS)
- Frequency Bias (FB)
- Gilbert Skill Score (GSS)
- Hanssen-Kuipers Discriminant (H-KD)
- Odds Ratio Skill Score (ORSS)
- Extreme Dependency Score (EDS)
- Symmetric Extremal Dependence Index (SEDI)
- Ranked Probability Skill Score (RPSS)
- Total Squared Error (TSE)
- Explained Variance Score (EVS)
- Normalized Mean Squared Error (NMSE)
- Fractional Variance (FV)
- Pearson Correlation Coefficient (PCC)
- Standard Deviation Ratio (SDR)
- Variance Inflation Factor (VIF)
- Median Absolute Deviation (MAD)
- Interquartile Range (IQR)
- Coefficient of Determination (R^2)
- Normalized Absolute Error (NAE)
- Relative Mean Bias (RMB)
- Mean Absolute Percentage Error (MAPE)
- Weighted Mean Absolute Error (WMAE)
- Absolute Skill Score (ASS)
- Relative Skill Score (RSS)
- Quadratic Skill Score (QSS)
- Normalized Root Mean Squared Error (NRMSE)
- Logarithmic Mean Bias Error (LMBE)
- Scaled Mean Squared Error (SMSE)
- Mean Bias Deviation (MBD)
- Geometric Mean Bias (GMB)
- Symmetric Brier Score (SBS)
- Adjusted Explained Variance (AEV)
- Cosine Similarity
- F1 Score
- Matthews Correlation Coefficient (MCC)
- Balanced Accuracy (BA)
- Negative Predictive Value (NPV)
- Jaccard Similarity Coefficient
- Gain
- Lift
- Mean Kullback-Leibler Divergence (MKLDIV)
- Jensen-Shannon Divergence (JSDIV)
- Hellinger Distance
- Wasserstein Distance
- Total Variation Distance
- Chi-Square Distance
- Intersection
- Bhattacharyya Distance
- Harmonic Mean
- Geometric Mean
- Lehmer Mean
- Chernoff Distance
- Rényi Divergence
- Tsallis Divergence
Flexible computation of metrics along specified dimensions or over the entire dataset.
Support for threshold-based metrics with customizable threshold values.
Integration with xarray and NumPy for efficient computation and data handling.
Compatibility with both time series and spatial data, supporting 2D, 3D, and 4D datasets.
Detailed Examples to guide users in utilizing the package effectively.

Installation

You can install NWPeval using pip:

pip install nwpeval

Usage

Here are a few examples of how to use the package

Example 1: Computing basic metrics

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Compute basic metrics
metrics = ['MAE', 'RMSE', 'ACC']
results = nwp_stats.compute_metrics(metrics)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

Example 2: Computing metrics with thresholds

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Define thresholds for specific metrics
thresholds = {
    'FSS': 0.6,
    'FSS_neighborhood': 5,
    'ETS': 0.7,
    'POD': 0.5
}

# Compute metrics with thresholds
metrics = ['FSS', 'ETS', 'POD']
results = nwp_stats.compute_metrics(metrics, thresholds=thresholds)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

Example 3: Computing metrics along specific dimensions

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Compute metrics along specific dimensions
metrics = ['MAE', 'RMSE', 'ACC']
dimensions = ['lat', 'lon']
results = nwp_stats.compute_metrics(metrics, dim=dimensions)

# Print the results
for metric, value in results.items():
    print(f"{metric}:")
    print(value)

Example 4: Computing probabilistic metrics with custom thresholds

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Define custom thresholds for probabilistic metrics
thresholds = {
    'SEDS': 0.6,
    'SEDI': 0.7,
    'RPSS': 0.8
}

# Compute probabilistic metrics with custom thresholds
metrics = ['SEDS', 'SEDI', 'RPSS']
results = nwp_stats.compute_metrics(metrics, thresholds=thresholds)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

Example 5: Computing weighted metrics

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Define weights for weighted metrics
weights = xr.DataArray(...)

# Compute weighted metrics
metrics = ['WMAE']
thresholds = {'WMAE_weights': weights}
results = nwp_stats.compute_metrics(metrics, thresholds=thresholds)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

Example 6: Computing distribution comparison metrics

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Compute distribution comparison metrics
metrics = ['MKLDIV', 'JSDIV', 'Hellinger', 'Wasserstein']
results = nwp_stats.compute_metrics(metrics)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

Example 7: Computing reference-based metrics

import xarray as xr
from nwpeval import NWP_Stats

# Load observed, modeled, and reference data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)
reference_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Define thresholds for reference-based metrics
thresholds = {
    'ASS_reference_error': reference_data,
    'RSS_reference_skill': 0.6,
    'QSS_reference_forecast': reference_data
}

# Compute reference-based metrics
metrics = ['ASS', 'RSS', 'QSS']
results = nwp_stats.compute_metrics(metrics, thresholds=thresholds)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

Example 8: Computing metrics with custom parameters

import xarray as xr
from nwpeval import NWP_Stats

# Load observed and modeled data as xarray DataArrays
obs_data = xr.DataArray(...)
model_data = xr.DataArray(...)

# Create an instance of NWP_Stats
nwp_stats = NWP_Stats(obs_data, model_data)

# Define custom parameters for metrics
thresholds = {
    'LehmerMean_p': 3,
    'Chernoff_alpha': 0.7,
    'Renyi_alpha': 0.8,
    'Tsallis_alpha': 0.9
}

# Compute metrics with custom parameters
metrics = ['LehmerMean', 'Chernoff', 'Renyi', 'Tsallis']
results = nwp_stats.compute_metrics(metrics, thresholds=thresholds)

# Print the results
for metric, value in results.items():
    print(f"{metric}: {value}")

These examples demonstrate the versatility of the package in handling different requirements and scenarios. They showcase the computation of probabilistic metrics with custom thresholds, weighted metrics, distribution comparison metrics, reference-based metrics, and metrics with custom parameters.

By providing a wide range of metrics and the flexibility to customize their computation, the package enables users to perform comprehensive evaluations of NWP models based on their specific needs and requirements.

Help

The help() function provides a convenient way to explore the available methods in the NWP_Stats class and view their descriptions and usage instructions.

Example usage:

from nwpeval import NWP_Stats

# Display help for a specific method
help(NWP_Stats.compute_fss)

# Display help for all available methods
help(NWP_Stats)

For more detailed usage instructions and Documentation and examples, please refer to the Examples directory.

NEXT UPDATE

Instability (Thunderstorm) indices,

Total Totals Index (TT)
Delta-T Index (DTI)
K Index (KI)
Lifted Index (LI)
Showalter Index (SI)
Deep Convective Index (DCI)
Severe Weather Threat Index (SWEAT)
Convective Available Potential Energy (CAPE) and More !!!

Contributing

Contributions to NWPeval are welcome! If you encounter any issues, have suggestions for improvements, or would like to contribute new features, please open an issue or submit a pull request on the GitHub repository.

License

NWPeval is licensed under the MIT License.

Acknowledgments

I would like to express my gratitude to the developers and contributors of the libraries and tools used in building NWPeval, including NumPy, xarray, and SciPy.

Contact

For any questions, feedback, or inquiries, please contact the maintainer:

Name: Debasish Mahapatra
Email: [email protected] | [email protected]

I hope you find NWPeval useful in evaluating and analyzing your numerical weather prediction models!

compute_rpss is not working on generated lightning data

I tried with examples given, the function to calculate rpss is giving errors about the absence of dim_0.

version: 1.5.0
python: 3.10.2

code:

import nwpeval as nw

obs, model = lightning_data()
metrics_obj = nw.NWP_Stats(obs.lightning_density, model.lightning_density)
thresholds = {
    'SEDS': 0.6,
    'SEDI': 0.7,
    'RPSS': 0.8
}

# Compute probabilistic metrics with custom thresholds
metrics = ['SEDS', 'SEDI', 'RPSS']
metric_values = metrics_obj.compute_metrics(metrics, thresholds=thresholds)

Error:

[682](file:///C:/Users/kizhu001/AppData/Local/miniforge3/envs/nbase-windows/lib/site-packages/xarray/namedarray/core.py:682) except ValueError:
--> [683](file:///C:/Users/kizhu001/AppData/Local/miniforge3/envs/nbase-windows/lib/site-packages/xarray/namedarray/core.py:683)     raise ValueError(f"{dim!r} not found in array dimensions {self.dims!r}")

ValueError: 'dim_0' not found in array dimensions ('time', 'lat', 'lon')```

When I tracked the error, it came from the compute_rpss function which expects dim_0 as a native dimension in input. I am not sure if this will be the case in every dataset that we provide.

Possible solution
Standardise the output and input structure of data by enforcing attributes and checking routines.

Please let me know what you think.

debasish-mahapatra / nwpeval Goto Github PK

nwpeval's Introduction

NWPeval

Features

Installation

Usage

Help

NEXT UPDATE

Contributing

License

Acknowledgments

Contact

nwpeval's People

Contributors

Stargazers

Watchers

Forkers

nwpeval's Issues

Recommend Projects

Recommend Topics

Recommend Org