Light

awong234 / laundry-day Goto Github PK

Data cleaning projects for the Cleveland R User Group

R 100.00%

laundry-day's Introduction

Saturday morning is a good time to do laundry.

It's common knowledge that much of data analytical work comes in the form of cleaning data; that is to say, acquiring and molding data into a conventional format ready to be used by data analytical tools.

Sometimes it's easier -- scrape an HTML table using rvest::read_html() |> rvest::html_table() and you might get something ready to use. Sometimes it's harder, perhaps you're parsing text via OCR out of images and need to format it properly.

In these sessions, we'll take messy data and wrangle them into cleaner formats. We focus on the cleaning portion because a) the result is more objective and b) everyone needs to do it and can learn from other's styles.

laundry-day's People

Contributors

Stargazers

Watchers

Forkers

codijote

laundry-day's Issues

TSA Passenger Volume

https://www.tsa.gov/travel/passenger-volumes

Tasks: light webscraping, tidying, date formatting

Has potential for some light modeling

Cleveland voter polling locations

https://boe.cuyahogacounty.gov/docs/default-source/boe/maps-and-data-library/pollinglocationlist.pdf?sfvrsn=6d0955e8_22

Similar aim to voter registration locations; PDF is formatted slightly differently however, introducing new challenges.

Cleveland voter registration locations

link broken ~~https://boe.cuyahogacounty.gov/docs/default-source/boe/voters-page/voter-registration-locationsd09cbbb22d48418489a4d4a173347df7.pdf?sfvrsn=29d4fac1_6~~

https://boe.cuyahogacounty.gov/docs/default-source/boe/voters-page/voter-registration-locations.pdf?sfvrsn=2feea41c_6

Reading PDF will be dirty from the outset. Text formatting after read in. Potential for map creation

USGS Earthquakes API

https://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php

Fuel Economy

https://fueleconomy.gov/feg/ws/

(At least) two data tables with a shared key.

Potential to get more data via API.

Will use DBI and duckdb to store / interact with data.

Cleveland Open Data

The City of Cleveland Created a data website here:

https://data.clevelandohio.gov/

Documentation and Examples here:
https://github.com/City-of-Cleveland

this is also a great discussion topic of missing data, positivity assumption violation for causal inference, etc.

Format images for analysis

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.