403 forbidden error about vacme-zurich-parser HOT 4 CLOSED

Hey @deuxnids, great you found this useful!

Usually invalid token gives 401. 403 might mean you are scraping to often. As far as I understand they use this FortiADC WAF, which might trigger rate limiting. I got my IP blocked once when my service was misbehaving - had to switch IP and I think it was 403. But this is just my guess.

Try to log 403 response body (often content type is text/html) or open in browser from affected IP. They usually describe cause better. Let me know if you have more info on that!

(I will try to add Bern as well, once I stabilize current version a bit and automate location mapping fetching)

from vacme-zurich-parser.

got few more cases of 403 Forbidden - for me usually means my IP got blocked by their WAF, and I need to rotate to the new one.

Last case for me was when they added filtering to the list, and my code was still trying to scrape noFreieTermine ones. This requests were considered 'suspocious' and my IP got blocked.

Feel free to reopen/comment if you have new info

from vacme-zurich-parser.

update: I had recently lots of 401 and 403s with IPs blocked. I ended up adding delay 1s between each request and it improved stability by a lot. 9971a71

I was able to make scraping more often (once per 3 min) now, and still have no issues with being blocked. Hope this helps, cheers

from vacme-zurich-parser.

Thanks @golonzovsky. Let me know If I can help you ( to the best of my ability) for be.vacme.

from vacme-zurich-parser.