Giter Club home page Giter Club logo

weatheraugmentation's Introduction

WeatherAugmentation

Given geo-located data, for machine learning/deep learning purposes, apply coarse weather attribution by looking at best consistent data available.

In US, weather underground has hundreds of sensors in large cities, and very fine granularity around rural US. For locations outside the US there is no high resolution data available for weather, so leverages Airport weather sensors around the world.

Airports need fairly high accuracy sensors, and fairly high frequency which covers high accuracy, high frequency, and decent coverage around the world, including lower developed countries as weather reporting standards are set internationally. METAR stations are of specific interest.

To assign weather discretely, coordinates are applied through reference point, current implementation required looking for cloest stations in a given country, but this can easily be aggregated for all stations in world.

Original Design and Design Considerations.

Original design leveraged https://en.wikipedia.org/wiki/Voronoi_diagram but this was generalized to KDTree structure, to collect k nearest points, which would be useful for more fine-grain analysis. The main considerations for KDTree was performance and being able to maintain multiple nearest points to check each nearby station for consistentish weather. METAR stations were selected for decent distribution, international requirements on quality, and ability to assign weather to events retroactively and globally. METAR data is saved since recording began, while the quality has drifted in the positive direction over time with additional features being added with newer features, in general was selected as best possible source for international selection. Open Weather Map was looked at for domestic data for extremly high resolution, and being able to query data through library directly, but no history, or guarentee on quality of data was found.

Open Weather Map also used to validate with high granular data exactly on weather sensor of OWM but not included in METAR stations. Approximate weather can be diffed for temperature, perciptiation, and visibility such that accuracy of coarse approximation can be measured.

Limitation

The issue with coarse analysis is while weather in general will not change drastically in short spatial locality, there are natural boundaries such as mountains and deserts which can create barriers for many of the features of interest including visibility, temperature, and percipitation. Therefore by looking at the k closest stations, more trust can be built on data if there is concensus. Additionally if a data source such as an earth wind map can be crawled and interpreted for downwind, and added with meteorologist expertise, more assurance can be made on the quality of data.

Execution

python Weather.py

This queries user for country of interest, in this example was looking at highly diverse terrain and looking for weather samples that varied greatly at the time in region with numerous natural boundaries, so chose area in middle east.

weatheraugmentation's People

Contributors

reversingwithme avatar ryanvrecenar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.