Comments (8)
With the report form properly customized, the adapter is dealing with smaller export file, and some useless stations are skipped early. This takes the adapter's running time from 4min down to about 1min.
I'm done with this source, but I'll be looking at adapting the adapter for other countries (e.g. Israel).
from openaq-fetch.
Quick note, the above site makes heavy use of iframes. The table with the data can be found on this url: http://live.rbcaa.org.za/DynamicTable.aspx?G_ID=1
from openaq-fetch.
Another source recently mentioned to us: http://www.saaqis.org.za/RequestData.aspx
from openaq-fetch.
http://live.rbcaa.org.za/Default.htm
You can find the data here.
from openaq-fetch.
Another source recently mentioned to us: http://www.saaqis.org.za/RequestData.aspx
This is a slow one. You submit your request and have to wait. Not a good channel for OpenAQ.
It aggregates data from all other South African data providers. It would be nice if we had a direct access to it. Even aqicn
doesn't have much coverage of the country, let alone the continent.
This one only covers the Richards Bay town.
from openaq-fetch.
Given @dolugen's comment, above, this one just focuses on the Richard's Bay data source (http://live.rbcaa.org.za/Default.htm) and organizes the data source for dev review.
-
These data are shared in a very similar way to the Israeli data, shared in this comment: #146 (comment)
-
Click: Dynamic Table->Richard's Bay
-
Follow similar procedures as Number 5 in #146 (comment)
- Suggest 15 minute or higher averaged data from stations
- Station name and coordinates are accessible by clicking on station name.
from openaq-fetch.
Data addresses
- The
/MenuSite.aspx
file has a variable (search forvar o
orDynamicTable
) that contains data for the Dynamic Table menu items (as well as the Envista system build version). - Read the list of station IDs from page
/DynamicTable.aspx?G_ID=X
- Read station coordinates from
/StationDetails.aspx?ST_ID=X
- Read the AQ data from
/StationReportFast.aspx?ST_ID=X
Underlying system
Looked around a bit, and found that the Israeli and South African sources both use the Envista Web system. Maybe they could share a single adapter that handles Envista sources.
Although they share similar "build numbers", their advertised version number differs:
Country | Build number in MenuSite.aspx |
Version number in About.aspx |
---|---|---|
IL | 2.0.20091.207 | 6.4.14 |
ZA | 2.0.20063.108 | 4.0.8.9 |
(Turkey's system has the same build number as Israel's)
I think @maxgrossman (he just started working on the Israeli adapter) and I should attempt to make a unified Envista adapter. We'll see.
Edit: The build numbers are from a web component that's used in the system.
from openaq-fetch.
I spent some excess amount of time debugging an export issue in the adapter, which at the end was an issue with the export page. In short: the export functionality was throwing the same files for different station exports.
This is worked around by giving each station requests their own cookie jar.
There are a few things that should be improved, but overall, the adapter is in working order.
Currently, a URLencoded XML is hardcoded in the adapter. The following is a note for dealing with that (moved it out of adapter to keep it cleaner):
const getLstMonitors = function () {
/*
TODO
An XML tree that describes the checkedness of
the monitors is URLencoded and passed as a string parameter
in the form.
It looks like below when decoded, going up to 20 something monitors:
<WebTree><Nodes><lstMonitors_1 Checked="true"></lstMonitors_1><lstMonitors_2 Checked="true"></lstMonitors_2><lstMonitors_3 Checked="false"></lstMonitors_3></Nodes></WebTree>
It cannot be queried with cheerio, as it's generated
on the client. But as structure of the tree is known,
it can be generated using the list of monitors and their tag ID.
*/
};
from openaq-fetch.
Related Issues (20)
- Sao Paulo not updating HOT 2
- Investigate Australia, Queensland adapter for quality and coverage HOT 7
- Remove moscow adapter HOT 1
- Turkiye reporting inconsistently HOT 7
- Canterbury, NZ data hasnt updated in 3 month HOT 4
- Tasmania update frequency issues HOT 1
- Japan missing data HOT 3
- Nepal data HOT 2
- Trinidad and Tobago not updating HOT 1
- Support regional fetchers HOT 2
- EEA using deprecated battuta service HOT 2
- Possible source for AQ data from Vietnam HOT 4
- Sinaica Mexico adapter - site down HOT 2
- Japan data gaps HOT 5
- Provider name clean-up HOT 3
- Kosovo adapter not updating HOT 1
- Hanoi reporting very incosistently HOT 3
- Adapter Issue: Trinidad & Tabago HOT 1
- Ust-ist adapter failing HOT 1
- Buenos Aires timezone issue
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openaq-fetch.