strategist922 / using-a-hadoop-cluster-to-crunch-data-from-new-york-city-cab-system Goto Github PK

This project forked from prinshul/using-a-hadoop-cluster-to-crunch-data-from-new-york-city-cab-system

Create a Hadoop/HDFS cluster in a Cloud environment (Microsoft Azure), write program(s) and run jobs to process public available data. Task is to set up a Hadoop cluster running Azure VMs. The cluster needs to have at least several (3+) nodes, including a master node. You will use the Yellow Cab data, from New York City Taxi And Limousine Commission dataset, available at http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml. You will need to load the years 2009 to 2016 of the dataset in your cluster. You will then need to answer the following questions using the dataset and mapreduce programs: What is the average number of passengers per trip in general, per month, and per day of the week? What is the average trip distance in general, per month, and per day of the week? What is the most used payment type? Create an ordered list for payment types. Create a graph showing the average number of passengers over the day (per hour). Create a version for work days and another for weekend days. Create a graph showing the average trip distance over the day (per hour). Create a version for work days and another for weekend days. Create a graph showing the average number of passengers over the day (per hour). Create a version for work days and another for weekend days.