santina / master_thesis_ubc Goto Github PK

Need to record how the wordlist was created, what versions of other third-party software were used and how to download them.

Will be keeping this issue open until the end of the thesis as more things need to be added to this repo.
As of now, I need to add

• Documentation on the dataset
  • How to download them
  • How big is it
  • What does it look like once downloaded
• Parsing code for the the data

Geniatagger prints everything to the terminal. It'd be nice to modify it so that it provides warnings or other statuses that can be captured by the python code that's using it.

When the Medline_13M is done, there will be the need to validate the result beyond just using PubMed as ground truth. I.e., how do our SVD predictions compare to PubMed's closely related papers that are calculated from manual assignments of MeSH headings?

Plan A : Look into whether there are existing survey tools that would allow me to set up survey programmatically.

• Survey Monkey
• Google form + spreadsheet, existing plugins or write my own plugins

Plan B : If Plan A fails, would need to set up my own website for human annotations.

• Idea: Set up an SQL database (on Azure? GSC?), Setup a website (Get to try out AngularJS with material design?)
• Need to figure out how to record the data.

Might want to write test code for the Python code.

It's known to me that using find_term to generate termID-termFreq file often results in

• the process stopped when there are other processes running on the same machine
• the process sometimes generates wrong results, as shown by running the same program on the same file and the results were slightly different outputs