Some method of properly passing Cloudflare protections? about wrp HOT 8 OPEN

Maybe cloudflare is detecting the headless Chrome?

from wrp.

The particular website (fanfiction.net) is actually known for being very aggressive about it (or maybe Cloudflare is particularly aggressive about that website? who knows). All I know is that I see 30-50 "checking the security of the connection" pages on an average day (and some days - at every page load), using a regular browser.

I don't think hiding the headless Chrome better will fix it. There has to be a way to pass this check.

from wrp.

Headless Chrome is often used for data scraping. Websites like Cloudflare block it more aggressively.

What exactly happens when you go to the page? Is there an "I'm not a robot" checkbox?

from wrp.

yeah crap, we need to come up with some solution for this

from wrp.

@TheTechRobo Yes, there is. It does the whole song & dance, and then redirects. Assuming the check works, the "browser" would need to be able to navigate, and/or send screen updates or something.

from wrp.

do you have any handy examples of pages that fail this?

from wrp.

Anything on fanfiction.net should qualify. They pursue verification so aggressively, that I have to go through the verification almost every other chapter on my desktop. It happens most often when switching chapters using the prev/next buttons.

That was actually the website I was hoping to use it with, as previously mentioned.

from wrp.

🤷 I have released 4.6.2; it has some basic anti crawler detection bits, also allows setting user agent with a flag; this seems to help the most;

I have tried ./wrp -ua="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" and it seems better

from wrp.