Computers & Geosciences: forked code repositories photo

cageo Goto Github PK

repos: 132.0 gists: 0.0

Name: Computers & Geosciences: forked code repositories

Type: Organization

Location: To search: type "firstauthorlastname-yearofpaperaccept"

Blog: http://www.journals.elsevier.com/computers-and-geosciences/

The Canadian Climate Data Scraping Tool (CCDST)

Canadian Climate Data Scraping Tool

Charmaine Bonifacio (a), Thomas E. Barchyn (b), Chris H. Hugenholtz (b), Stefan W. Kienzle (a,c)

(a) Department of Geography, University of Lethbridge, 4401 University Drive, Lethbridge, AB, Canada T1K 3M4 (b) Department of Geography, University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4 (c) Applied Behavioural Ecology and Ecosystems Research Unit, University of South Africa, PO Box 392, Florida, Pretoria, South Africa

Abstract

In this paper we present a new software tool that automatically fetches, downloads and consolidates climate data from a Web database where the data are contained on multiple Web pages. The tool is called the Canadian Climate Data Scraping Tool (CCDST) and was developed to enhance access and simplify analysis of climate data from Canada's National Climate Data and Information Archive (NCDIA). The CCDST deconstructs a URL for a particular climate station in the NCDIA and then iteratively modifies the date parameters to download large volumes of data, remove individual file headers, and merge data files into one output file. This automated sequence enhances access to climate data by substantially reducing the time needed to manually download data from multiple Web pages. To this end, we present a case study of the temporal dynamics of blowing snow events that resulted in ~3.1 weeks time savings. Without the CCDST, the time involved in manually downloading climate data limits access and restrains researchers and students from exploring climate trends. The tool is coded as a Microsoft Excel macro and is available to researchers and students for free. The main concept and structure of the tool can be modified for other Web databases hosting geophysical data.

Keywords

Climate data online
Scraping tool
Canada

How To Use CCDST

Please download the user documentation and macro from the Download folder.

License

The Canadian Climate Scraping Tool is licensed under a GNU General Public License v2.0

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

Computers & Geosciences: forked code repositories's Projects

davis-2013

Derivative analysis for layer selection of geophysical borehole logs Aaron C Davis / Niels B Christensen

de-siqueira-2014

An automatic method for segmentation of fission tracks in epidote crystal photomicrographs

delipetrev-2013

Development of a web application for water resources based on open source software, Blagoj Delipetrev, Andreja Jonoski and Dimitri Solomatine

density-partition

Example code for "Parallel Variable-Resolution Bathymetric Estimation with Static Load Balancing"

dioguardi-2014

PYFLOW: a computer code for the calculation of the impact parameters of Dilute Pyroclastic Density Currents (DPDC) basing on field data Authors: Fabio Dioguardi and Pierfrancesco Dellino

dms

Direct Multivariate Simulation

elwaseif-2016

A Matlab-Based Frequency-Domain Electromagnetic Inversion Code (FEMIC) with Graphical User Interface"

eranen-2014

A full graphics processing unit implementation of uncertainty-aware drainage basin delineation, Eränen, D., Oksanen, J., Westerholm, J. and T. Sarjakoski

espana-2013

ArcB: a GIS tool for modelling annual difusse infiltration on a plot scale.

etherington-2016

Python code in support of: Etherington TR, Perry GLW (2017) Spatially adaptive probabilistic computation of a sub-kilometre resolution lightning climatology for New Zealand. Computers & Geosciences 98:38-45

euler-deconvolution-python

Reliable Euler deconvolution estimates throughout the vertical derivatives of the total-field anomaly

fernandez_2018

Angular Distance Sample Selection

ferrari-2016

LibHalfSpace: a C++ object-oriented library to study deformation and stress in elastic half-spaces

ficklin-2015

Code for the paper "Incorporation of the Penman-Monteith potential evapotranspiration method into a Palmer Drought Severity Index tool" in Computers and Geosciences

flinders-2014

Evaluation of Arctic Multibeam Sonar Data Quality using Nadir Crossover Analysis and Compilation of a Full-Resolution Data Product Ashton F. Flinders, Larry A. Mayer, Brian A. Calder, Andrew A. Armstrong

folkclassification-grainsizeparameters

gan-2012

Extracting paleoclimate signals from sediment laminae: an automated 2-D image processing, Stoney Q. G and Christopher A. Scholz

geilhufe-2012

Two-dimensional wavelet variance estimation with application to sea ice SAR images, Geilhufe et al., accepted 2012

ghcnd_access

The ghcnd_access toolbox is utilised to extract data from the Global Historical Climatology Network (GHCN)-Daily database and change the .dly file format into a more accessible structure. The toolbox can be run in either MATLAB or open source alternative GNU Octave.