Giter Club home page Giter Club logo

getgfs's Introduction

getgfs

TestsBlack LintingDOI

Logo - Cloud with sun emerging from behind with text 'GetGFS' underneath

getgfs extracts weather forecast variables from the NOAA GFS forecast in a pure python, no obscure dependencies way. Currently you can:

  • "Connect" to a forecast
  • Search the variables
  • Download variables for any time range, longitude, latitude and altitude
  • Download "wind profiles" where you get an interpolation object for the u and v wind components by altitude

For full documentation please see the docs

Installing

Installation is simple with PyPi:

pip install getgfs

Requirements

The required libraries (installed by PyPi) are:

scipy, requests, fuzzywuzzy, numpy, python_dateutil, regex

I have tried to ensure that these are well maintained and work across platforms (as this was the motive for writing this library).

About

The incentive to write this library was that the current method to get any variable was to download and extract information from a grib file. This requires you to use the ECMWF's ecCodes which doesn't work on Windows. To get around this the OpenDAP version of the forecast is used and a custom decoder reads the downloaded files.

Previous Python projects that attempted this do not fulfil all the requirements, mainly being an importable library. Acknowledgment must be made to albertotb's project get-gfs for providing the first foothold along the way.

Usage

The library is straight forward to use. To get started create a Forecast object by:

>>>import getgfs
>>>f=getgfs.Forecast("0p25")

You can choose the resolution to be 0p25, 0p50 or 1p00 and for the 0p25 forecast you can optional specify a shorter time step by adding 1hr after.

First to find what variable you are looking for use the search function, for example if I want the wind speed I could search for "wind":

>>>f.search("wind")
[('gustsfc', '** surface wind speed (gust) [m/s] ', 100), ('ugrdprs', '** (1000 975 950 925 900.. 7 5 3 2 1) u-component of wind [m/s] ', 125), ('ugrd_1829m', '** 1829 m above mean sea level u-component of wind [m/s] ', 125), ...

So now I can see I might want "gustsfc". Now if I want the wind speed at N70.1 W94.7 at 5:30 on the 27th of February (only forecasts going back around a week are available and future times available depend on the forecast - look for f.times) I could do:

>>>res=f.get(["gustsfc"],"20210227 5:30", 70.1,-94.7)
>>>res.variables["gustsfc"].data
array([[[18.808477]]])

You can get more information (e.g. what is the units of this) by exploring the variables information

{'_FillValue': 9.999e+20, 'missing_value': 9.999e+20, 'long_name': '** surface wind speed (gust) [m/s] ', 'level_dependent': False}

You can also get multiple variables by including more names in the list or a range of positions by using "'[min_lat:max_lat]'" type strings in place of the position parameters.

Contributing

Please see contributing for more information.

Todo

getgfs's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

getgfs's Issues

Question about Future Forecasts

While trying to query future forecasts, I encounter an error. I do not understand where else I could manipulate the variable or date time to get a value for a future date other than in the get request. I tried to go to the discussions as referenced in the documentation but encountered a 404 not found. I cannot find anything in the documentation that explains how to achieve this. Attached below is the error encountered. Any help would be greatly appreciated! Thank you.

image

Documentation: Longitude is [0,360] and doesn't accept [-180,+180]

This is just a note on documentation you might want to add somewhere as it tripped me up. I'm used to working with longitude on a -180:180 basis. If you put in a negative longitude, the program doesn't throw any issues, it just sets it to zero. I didn't realize this until I had a different, forward looking issue, and I noticed the longitude was off.

It could be helpful to note this somewhere in the docs, and maybe plug the simple solution that the conversion can be found using the modulo, as seen at the bottom of the screenshot.

-100 Longitude == 100W == -100 % 360 == 260; Doesn't break for positive values. (10 % 360 = 10E still)

Example of longitude difference

Download NetCDF/GRIB files: consider using the GRIB filter service

As far as I understand, this repository uses the ASCII service of the OpenDAP server to download the data. To get a NetCDF instead of a CSV file (this seems to be on the roadmap), an optional dependency NetCDF4 could be added. Then the NetCDF file can also be downloaded directly from the OpenDAP server using, for instance, xarray (see an example here). However I found this to be quite unreliable, throwing this error quite frequently (even when everything seems OK):

RuntimeError: NetCDF: Malformed or inaccessible DAP2 DATADDS or DAP4 DAP response

For that reason, I would consider using the service GFS filter. This service is able to filter and download the GFS GRIB files directly. I wrote an example notebook if you want to take a look. The only problem is that you need to add the dependency cfgrib, which in turn depends on ecCodes, that cannot be installed with pip. Reading the description, avoid using ecCodes seems to be one of the main motivations of this library. However, I think you can get around ecCodes not working on Windows by installing it with conda.

In any case, I would consider adding either one of both of the previous ways with the optional dependencies NetCDF4 or cfgrib, since I think it is much easier than constructing the NetCDF file "by hand" from the ASCII response.

Issue when fetching range of coordinates

Hi,

I have encountered a bug in the library which occurs when fetching a range of coordinates instead of just a point.

By running:

>>> f = getgfs.Forecast("1p00")
>>> res = f.get(["gustsfc"], "20220509 5:30", "[-90:90]", "'[0:360]'")

We get

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [49], in <module>
----> 1 res = f.get(["gustsfc"], "20220509 5:30", "[-90:90]", "'[0:360]'")

File ~/dev/env/lib/python3.9/site-packages/getgfs/getgfs.py:103, in Forecast.get(self, variables, date_time, lat, lon)
    100 forecast_date, forecast_time, query_time = self.datetime_to_forecast(date_time)
    102 # Get latitude
--> 103 lat = self.value_input_to_index("lat", lat)
    105 # Get longitude
    106 lon = self.value_input_to_index("lon", lon)

File ~/dev/env/lib/python3.9/site-packages/getgfs/getgfs.py:259, in Forecast.value_input_to_index(self, coord, inpt)
    257 if isinstance(inpt, str):
    258     if inpt[0] == "[" and inpt[-1] == "]" and ":" in inpt:
--> 259         val_1 = float(re.findall(r"\[(.*?):", inpt))
    260         val_2 = float(re.findall(r"\:(.*?)]", inpt))
    261         val_min = self.value_to_index(coord, min(val_1, val_2))

TypeError: float() argument must be a string or a number, not 'list'

The reason for the bug is that re.findall returns a list, not a string, and therefore the casting to float fails.
By changing lines 259 and 260 of getgfs/getgfs.py to:

val_1 = float(re.findall(r"\[(.*?):", inpt)[0])
val_2 = float(re.findall(r"\:(.*?)]", inpt)[0])

the issue is solved.

Sometimes the function get cannot get the data.

It seems that when I run the code several times, there are both error and data in the output which means the function is not strong enough to get the data every time. Is is the problem of the website?

Thanks for your time.

Max retries exceeded with url: /dods/gfs_0p25/gfs20220824/gfs_0p25_00z.das

Hi, I just installed this library and tried the example but I am getting below error:

import getgfs
f=getgfs.Forecast("0p25")

Error:

SSLError: HTTPSConnectionPool(host='nomads.ncep.noaa.gov', port=443): Max retries exceeded with url: /dods/gfs_0p25/gfs20220824/gfs_0p25_00z.das (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:997)')))

There is no information regarding an extra setup, do I need to register somewhere? Get an API key? or something similar to that or the library should work out of the box with the public URL?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.