tlin41390 / musicclassification Goto Github PK

Currently loading data into MongoDB manually using the mongo shell. Looks easy enough to do in Python with GridFS. Unclear how useful this is, would be useful if we want to incorporate web-scraping...

We cannot make simple changes to our csv files within MongoDB after we store them. Turns out overwriting files is not possible using GridFS.

Instead, to overwrite one must delete the file from MongoDB and re-add the new one:

$ mongofiles -d=Music_db list
2021-04-17T21:21:13.925-0700	connected to: mongodb://localhost/
features_3_sec.csv	10713

$ mongofiles -d=Music_db delete features_3_sec.csv
2021-04-17T21:22:11.797-0700	connected to: mongodb://localhost/
2021-04-17T21:22:11.799-0700	successfully deleted all instances of 'features_3_sec.csv' from GridFS

$ mongofiles -d=Music_db put Data_Sample/features_3_sec.csv 
2021-04-17T21:15:09.129-0700	connected to: mongodb://localhost/
2021-04-17T21:15:09.129-0700	adding gridFile: Data_Sample/features_3_sec.csv

2021-04-17T21:15:09.254-0700	added gridFile: Data_Sample/features_3_sec.csv

Worse, turns out whatever you pass to put becomes the file identifier which needs to be used in Python, so this will be the relative path from wherever you run mongofiles. I.e in this case our file identifier is Data_Sample/feature_3_sec.csv, but previously it was just features_3_sec.csv just because we originally ran mongofiles one directory up.

This is not ideal and we should at least automate it so its always consistent. There may also be another option we can pass to mongofiles to specify an identifier that we can use.