This is README documents my learning process for Hadoop.
I initially started learning about Big Data in general, but honed in on Hadoop after some research

1. Decided to learn NoSQL and Big Data 🥳

2. Watched video comparing relational databases (which I know) to NoSQL databases (which I don't know)

3. Started a course on Big Data + Hadoop

4. Started a MongoDB crash course, as I've already used MongoDB for work, and for personal web projects

5. Did more research and decided to learn Hadoop instead

6. Learned about evolution of technology that lead to Big Data

7. Learned definition of Big Data, datasets so large and complex they can't be processed using traditional tools

8. Learned about the 5 Vs of Big Data; Volume, Velocity, Variety, Value, Veracity

9. Decided to start a more practical course, as my current Hadoop course is very theoretical and lecture-based

10. Started downloading HDP Sandbox for VirtualBox (6 hour download time 🙄)

11. Sandbox won't import from downloaded file

    ☝ Error "Failed to import appliance C:/Users/nikun/Downloads/HDP_2.5_virtualbox.ova.

Result Code: E_INVALIDARG (0x80070057)"

11. Searched the internet extensively for a solution

12. Finally found a workaround by extracting the VMDK file and running it seperately

13. Workaround didn't work, turns out virtual box file is corrupted, will download it again overnight

14. Downloaded the HDP Sandbox again

15. Got further along than before

16. Ran into a memory error

17. Stack Overflow tells me I don't have enough RAM

    ☝ Need 8GB free memory, I only have 8GB total memory

18. Finally got Hadoop running after upgrading my RAM

19. Navigated to localhost port provided by Ambari but I'm seeing loads of errors

20. Errors fixed themselves (the processes must have been initialising)

21. Uploaded 2 movie record databses using the Hive interface

22. Used Hive's SQL to query for the most popular movie up to 1998

    ☝ The winner was Star Wars (1977) 🎉

23. Decided to brush up on my SQL and started a seperate SQL course

24. Completed SQL course 🎉

25. Returned back to Hive, where I'm now extremely comfortable using its SQL-like syntax

26. Used the HDFS web interface to to upload and view data files through its explorer

27. Opened a command line SSH connection to HDP Sandbox using Putty

28. Created and deleted data files using command line

29. Learned about MapReduce on a conceptual level; mapper, shuffle & sort, reducer

30. Also learned how MapReduce works across a cluster

31. Self-discovered VirtualBox Snapshots, to skip the long loading time for th Ambari dashboard to load all processes

32. Pausing Hadoop to Learn Kafka! https://github.com/johnobla/kafka