Giter Club home page Giter Club logo

urban-epi's Introduction

Urban Environmental Assessment Tool

This is a research tool for global urban environmental assessment. The program takes as an input a directory (file folder) of shapefiles with income and population data. Based on these shapefiles, the program will produce estimates for air quality, transportation, urban form, and green space. To replicate this analysis for a city you have in mind, begin by cloning the repo with git clone http://github.com/ryanthomas/urban_epi.git and overwrite the contents of the city_boundary folder with the shapefile for your city. Then follow the instructions below.

Requirements

All the tools used in this analysis are open source, including the data, which are freely available on the internet.

  • Unix environment (e.g. the Yale HPC, any Linux or MacOS. If you're going to use a virtual machine on your laptop, I suggest OSGeoLive in a virtual machine). Allocate at least 4GB RAM and 50GB storage.
  • GDAL/OGR
  • GRASS 7.0 or GRASS 7.2 (linked to the commandline call grass)
    • GRASS extensions installed prior to running the program: r.li, v.in.osm.
  • AWK
  • NodeJS (After downloading OSM data, we use the osmtogeojson commandline tool.)
  • Anaconda with Python 2 or 3
  • Access to the data used in this application - see the Setup section below.

Setup

This repo contains only the code for an application that builds a database.

Set up directory tree and get code

  • Run git clone http://github.com/ryanthomas/urban-epi.git to clone and rename the diectory.
  • Run `echo '#!/bin/bash
    export DIR=$PWD # home or parent directory for the code base of the project
    export DATA="/project/fas/hsu/rmt33/urban-epi/data" # CHANGE ME!
    export IND="${DIR}/indicators"
    export SH="${DIR}/src/bash"
    export GRASSDB="${DIR}/grassdb"
    export RAS="${DATA}/raster"
    export VEC="${DATA}/vector"' > src/bash/grass_variables.sh
    source src/bash/grass_variables.sh

Get data

The data are available on the Yale High Performance Computing Cluster in the /project/fas/hsu/rmt/urban-epi/data/ directory. Once you have access to the cluster, you can run scp [from] [to].
For example: scp [email protected]:/project/fas/hsu/rmt/urban_epi/data/ location/on/your/computer. Run this from your Mac command line when you are NOT connected to the cluster. Windows users should look up the appropriate command on Yale's HPC help site.

Anaconda Installation

I recommend installing conda, because it makes cloning the Python environment very easy. See the page here for instructions.

Once you do this, you have to restart your terminal or type . ~/.bashrc to get access to the conda command.

Create a new python environment using conda.

conda env create -f source/environment.yml

Activate the new environment

source activate uepi

Install Node and NPM

This is only relevant if you are going to be downloading data from OSM using their API. The OSM extracts are saved in a file format that we convert to GeoJSON via a command line utility available through Node - osmtogeojson. Node installs take a little time to run, but are pretty straightforward on a Mac.

If you are running a download script from the cluster, run the command line script bash src/install_node_on_cluster.sh

Adding Data

New cities can be added by putting a shapefile in the seed_data folder. It must be a shape file of the neighborhoods and have the proper fields in the attribute table:

  • population
  • per capita income*
  • This may change.

Running the script

From your laptop/VM

From your parent directory, run src/bash/grass_batch_script.sh grassdb/urban [path/to/georeferenced_file]

From the cluster - in parellel

Install dead Simple Queue from the YCRC github page.

From your parent directory, run git clone https://github.com/ycrc/dSQ.git

Run the batch script

  • run src/bash/write_jobs.sh to write a tasks.txt.
  • run dSQ/dSQ --taskfile tasks.txt > tasks.sh
  • run sbatch tasks.sh

To come!: To run all these scripts at once, run src/bash/run_scripts_in_parallel.sh.

urban-epi's People

Contributors

geoffreyhershmartin avatar greenisagoodcolor avatar ryanthomas avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.