brainhack-school2020 / koudyk_bhs_project Goto Github PK

Hi @koudyk,

Congrats for this project description!

What type of methods are you planning to use for text mining ? Are you planning to use word2vec for example ?

Hello @koudyk I'm very interested in the results from your project! Indeed that is a very worrying issue in neuroimaging. A big paper on that topic was published yesterday in Nature https://www.nature.com/articles/s41586-020-2314-9 maybe it could be helpful.
Here's also an article about some usefull way to extract features from text files using python
https://www.analyticsvidhya.com/blog/2018/02/the-different-methods-deal-text-data-predictive-python/

We need make tests and set up Travis for continuous integration.

Cool idea! Does your dataset give you access to the full text, or only the abstract? Would you limit your data mining to the abstract?

So it's easier to version control

Find fMRI literature in the PubMed open-access subset using the PubMed E-utilities, with a list of papers for each year in a range of years (tbd).

For most functions, I describe what it does, but I haven't gone through and described all the inputs & outputs

explore interactive components:

• zoom into network
• click on node to get paper title
• add time slider to see how network evolved in time

Once the real data is ready, visualize it in the way we visualized the simulated data.

It looks like something funny happens to the colors during the making of the gif.

The individual images have normal color, so the problem is with the gif.

For each paper, search for methods-related keywords (maybe starting with 'python' and 'matlab'). If a keyword is found, append the paper ID to a list for that keyword. (need to verify if this is a good format for the visualization)

For each paper, list who they cite. Put this info into a binary matrix including all papers in the search results and the papers cited in all the search results. (need to verify in step 1 whether this is a good format for the visualization)

Visualize simulated data in the same way as we want to visualize the real data. This step should be done first so we know what outcomes are possible; this will inform our code for getting data from PubMed.

I'm envisioning a visualization of the entire citation network (over all years), coloured by whether they mention 'python' or 'matlab' (or some other keyword). The interactive component will be a slider that allows the user to step through each year, such that future years are not visible in the figure. Let's see if this is possible

Hi Kendra,

Cool idea :) I was wondering - do you already know how you will evaluate the performance of the keyword extraction?

improve the visualization in these ways:

• get citation data from dataframe, without needing the citation matrix (which is time-consuming to build)
• add the software info by colouring the nodes
• make new fig for each month or year, then save all images as gif

the pubmed IDS don't match the pubmed central ids