rafelafrance / herbarium_phenology Goto Github PK

I'll need to create a new trait table to handle pseudo-labels.

This should help with using larger models on HiPerGator

I don't need to change this right away but it should get handled sometime.

At the very least I can use multiprocessing. It is very likely that I can speed up other parts of the program.

We have a multi-label problem with missing labels. I don't need the latest & greatest solution to the issue but I do want to create a single model for all of the traits. We want to gracefully handle missing labels in the data. If possible it would also be nice to add labels to an existing model.

Track what, how, and when jobs are run.

See the example in the digi_leap repository.

Splits need to follow the image so that a test image should always be a test image. The same logic for splitting the record applies, it just hitting a different table.

Classes (positive/negative) are heavily imbalanced. I am currently using weighted classes. See if oversampling works better for our case.

We need to filter the iDigBio information to only include angiosperm data (among other things). Two web pages have a fairly exhaustive list of this information along with some alternate names. Alternate names will be very helpful for older data because the names will change over time.

We need to write a script that will download the pages and then extract the data from them and write them to a text file that can be read programmatically. I'm OK with either CSV or JSON-lines output. Links to the pages:

• https://academic.oup.com/botlinnean/article/181/1/1/2416499#84268287
• https://academic.oup.com/botlinnean/article/141/4/399/2433548#84152951

We need to be able to run this script (or jupyter notebook) repeatedly... Or more accurately all of us need to be able to run this script to gather the information ourselves.

Note that I have had success using the BeautifulSoup4 python library. I just used it to parse a checklist for lice hosts.

.

.

This library will confuse people.

I definitely need to simplify installation.