This is a research tool for global urban environmental assessment. The program takes as an input a directory (file folder) of shapefiles with income and population data. Based on these shapefiles, the program will produce estimates for air quality, transportation, urban form, and green space. To replicate this analysis for a city you have in mind, begin by cloning the repo with git clone http://github.com/ryanthomas/urban_epi.git
and overwrite the contents of the city_boundary
folder with the shapefile for your city. Then follow the instructions below.
All the tools used in this analysis are open source, including the data, which are freely available on the internet.
- Unix environment (e.g. the Yale HPC, any Linux or MacOS. If you're going to use a virtual machine on your laptop, I suggest OSGeoLive in a virtual machine). Allocate at least 4GB RAM and 50GB storage.
- GDAL/OGR
- GRASS 7.0 or GRASS 7.2 (linked to the commandline call
grass
)- GRASS extensions installed prior to running the program: r.li, v.in.osm.
- AWK
- NodeJS (After downloading OSM data, we use the
osmtogeojson
commandline tool.) - Anaconda with Python 2 or 3
- Access to the data used in this application - see the Setup section below.
This repo contains only the code for an application that builds a database.
- Run
git clone http://github.com/ryanthomas/urban-epi.git
to clone and rename the diectory. - Run `echo '#!/bin/bash
export DIR=$PWD # home or parent directory for the code base of the project
export DATA="/project/fas/hsu/rmt33/urban-epi/data" # CHANGE ME!
export IND="${DIR}/indicators"
export SH="${DIR}/src/bash"
export GRASSDB="${DIR}/grassdb"
export RAS="${DATA}/raster"
export VEC="${DATA}/vector"' > src/bash/grass_variables.sh
source src/bash/grass_variables.sh
The data are available on the Yale High Performance Computing Cluster in the /project/fas/hsu/rmt/urban-epi/data/
directory. Once you have access to the cluster, you can run scp [from] [to]
.
For example: scp [email protected]:/project/fas/hsu/rmt/urban_epi/data/ location/on/your/computer
.
Run this from your Mac command line when you are NOT connected to the cluster. Windows users should look up the appropriate command on Yale's HPC help site.
I recommend installing conda, because it makes cloning the Python environment very easy. See the page here for instructions.
Once you do this, you have to restart your terminal or type . ~/.bashrc
to get access to the conda
command.
conda env create -f source/environment.yml
source activate uepi
This is only relevant if you are going to be downloading data from OSM using their API. The OSM extracts are saved in a file format that we convert to GeoJSON via a command line utility available through Node - osmtogeojson. Node installs take a little time to run, but are pretty straightforward on a Mac.
If you are running a download script from the cluster, run the command line script bash src/install_node_on_cluster.sh
New cities can be added by putting a shapefile in the seed_data folder. It must be a shape file of the neighborhoods and have the proper fields in the attribute table:
- population
- per capita income*
- This may change.
From your parent directory, run src/bash/grass_batch_script.sh grassdb/urban [path/to/georeferenced_file]
From your parent directory, run git clone https://github.com/ycrc/dSQ.git
- run
src/bash/write_jobs.sh
to write a tasks.txt. - run
dSQ/dSQ --taskfile tasks.txt > tasks.sh
- run
sbatch tasks.sh
To come!: To run all these scripts at once, run src/bash/run_scripts_in_parallel.sh
.