amplab / datascience-sp14 Goto Github PK

Hello!
I have read your excellent material on the lab8. I have appreciated the pictures that you have added explaining the partitioning of the data across the worker nodes. I found them very helpful for understanding what is happening behind the scene with the RDD.
It would be great if you can add more examples like this for the spark streaming (with pictures and data flowing between the nodes and DAG).
Thank you.
Regards,
Florin

Any chance of you sharing access to the course slides ?

 half_hour_grouped = log_df.groupby(lambda row: pd.to_datetime(str(log_df['date'][row].hour) + ":" +
str((log_df['date'][row].minute / 30)*30))) 

is really smart but maybe better to highlight the nature of the integer division,

 half_hour_grouped = log_df.groupby(lambda row: pd.to_datetime(str(log_df['date'][row].hour) + ":" +
str((log_df['date'][row].minute // 30)*30))) 

Hello, I'm a new in big data field. I'm interesting learning spark, but Taiwan where I live is a few people about it. So I want to try to translate this course to Chinese to spread it out , and let more people know about spark. I don't know if you have any concerning about what I want to do, hope receive your reply, thank you