• Nice use of plots to help you determine the number of neighbors for KNN
• Good use of scale. Adding the additional scaling jumps the predictive accuracy from 70% to 90%. Exploration of your data through plots and other means would have revealed this.
• There are other ways to calculate the accuracy of your model. For instance most models will have a score function. i.e. "clf.score(X_test, y_test)" http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
• And while the problem didn't specify it, I think using a few basic histograms to visualize the distribution of the data can go a long way in catching things like the differing scales and other peculiarities with which you can modify your model.
• Also add comments or a short writeup comparing the models you've done. This will not only help the reader but also yourself when you go back to look at your code.
• All in all, good work!

Came out two weeks ago, you might find this interesting!

http://fivethirtyeight.com/features/fastest-airlines-fastest-airports/
http://fivethirtyeight.com/interactives/flights/

@bingbingboo

"Comments refer to the question number:
#3. You did this correctly, but to see the full functionality of Pandas, don't use the print command, just type ""df.head(20)"" This keeps the output formatted in a table form that is easier to read.
#4 Good job!
#6 You're sql commands are correct! You just didn't use the pd.read_sql() command. For example, if you modified your code to read pd.read_sql(""SELECT * FROM sf_df limit 5"",con) - if runs just as you want it to! 7. Same as 6, SQL query is correct, so just modify for pandas: pd.read_sql(""SELECT * FROM sf_df where neighborhood='Haight Ashbury' and category LIKE '%Sidewalk Cleaning%' LIMIT 10"",con). "

A few things ~ don't forget to think about % of total flights late. If you just do raw counts, you'll miss some of the smaller airports!

Really like the weather idea!!

@bingbingboo

You wrote:
I want to collect 5 or 10 years of flights arrival data into a big data store probably Hadoop like AWS EMR.

Comment:
How will you collect this data?
http://www.transtats.bts.gov/OT_Delay/OT_DelayCause1.asp
http://openflights.org/data.html

You wrote:
Based on the dataset, I can render some historical analysis for flight departure or arrival delay based on airline, aircraft type, date/time and airport name.

Comment:
Great! I like this idea. Can you think of some useful questions to answer that you, as a flight customer might be curious about?

You wrote:
I hope I will be able to build a predictive model based on the past data to predict what is the chance a particular flight will be delayed.

Comment:
Great!!!! What sort of features do you think could be predictive in this format? What data is available? A regression model could be a good place to start, but also pay attention during our Naive bayes lectures!

Lets keep this conversation going here! Tag me in your reply (@ostegm)