PDFs with GROBID 'success' but empty GROBID fulltext body lack any access option on scholar.archive.org (eg, presentation slides) about fatcat-scholar HOT 1 OPEN

What seems to be happening in this particular corner case is that GROBID parses the PDF successfully, but the "body" is empty. Here is what the "intermediate" hydrated object looks like: https://archive.org/~bnewbold/tmp/release_hneyekhayrdwvpvzdlnbzqalfu.scholar_intermediate.json

Because the body is empty, we don't create a "fulltext" sub-object in the indexed document here: https://github.com/internetarchive/fatcat-scholar/blob/master/fatcat_scholar/transform.py#L250

There are a couple ways we could try harder here. We could link to the file even though the extracted text is empty (aka, add access options even if the fulltext object is emtpy). We could detect the empty GROBID body earlier in the pipeline, and substitute raw extracted text ("pdftotext") earlier, so there would be at least something. We could try to improve GROBID extraction for slides, or detect that case and always use a different tool.

Slides are in-scope for both fatcat and scholar, and it would be good to fix this. I think this would be on the backburner for me to fix in the near future, but if you (or somebody else) would like to dig in and try to improve the behavior, I would be happy to review and give pointers.

I'm going to edit the title of this issue, I hope that is ok with you.

from fatcat-scholar.