ghdb_scraper.py no longer retrieves dork about pagodo HOT 5 CLOSED

Hi @Astroida - thank you for alerting me to that! Site definitely looks different. I'll take a look at updating the code. For the time being, you can change the URL (https://github.com/opsdisk/pagodo/blob/master/ghdb_scraper.py#L34) to point to https://old.exploit-db.com which is the old site that worked with ghdb_scraper.py

from pagodo.

Hi @opsdisk, thanks for the temp fix! Wasn't aware that the old link still exists.

from pagodo.

@Astroida Try taking this branch for a spin: https://github.com/opsdisk/pagodo/tree/issue-21

My original testing shows they start blocking attempts after 500-1000 requests even with 1 thread, so I may have to add some logic to back off / randomize the request rate like I do in https://github.com/opsdisk/metagoofil/blob/master/metagoofil.py

from pagodo.

nm testing this. I figured out how to pull all Google dorks with 1 HTTP GET request. May take a day or two to push the code.

from pagodo.

Pull the latest from master branch. Pushed some fresh updates.

from pagodo.