HTTPError for paperscraper.get_dumps.chemrxiv() about paperscraper HOT 6 CLOSED

But this is correct. First one .json is created for each paper, and later they are aggregated into a single .jsonl that contains one entry per line. The .json files will then be deleted automatically, but if the code stops during scraping (like in your case), you will find only the .json files, not the jsonl yet

from paperscraper.

I manually added the fixed code yesterday and successfully ran chemrxiv(). Thanks for your help!

from paperscraper.

Hi @LioHong,

Thanks for the interest in paperscraper and reporting this issue. This is caused by your OS being Windows, I should not have used os.path.join to join URLs.

Sorry for that, I will fix it soon in a PR

from paperscraper.

Thank you for the prompt response!

Just to confirm, I'm assuming the expected output is a single chemrxiv JSONL dump rather than a bunch of JSONs.

from paperscraper.

@LioHong Please have a look at #29

from paperscraper.

Thanks. Thew new version is available via pypi now (0.2.7).
pip install --upgrade paperscraper and your issue will be resolved

from paperscraper.