Hi, Iām Andrew.
I write web scrapers in Python and scale them hosting Celery in AWS.
It's nice when a site presents all its data in full HTML, but more often than not, there's some obstacle, whether it be javascript-rendered HTML, pdf-embedded data, ips-bans, or captchas. It's quite fun to find ways around these things, and I've these scripts and others here.