Comments (3)
mitchellkrogza
commented on July 30, 2024
I assume you are referring to Qwantify? Must have lingered in there since my original lists, removed with next Commit coming shortly. Has been removed with latest commit - 20dbd00
Thanks for alerting me to this. This one must have slipped through from one of my very early bad bot lists. Will go through bot lists again for a good double check. Please notify me if you spot anything else.
from nginx-ultimate-bad-bot-blocker.
eurobank
commented on July 30, 2024
Qwant/Qwantify doesn't read/respect robots.txt.
from nginx-ultimate-bad-bot-blocker.
chrisastley
commented on July 30, 2024
I'm having big issues with a group of Useragents/bots that don't respect robots.txt and are crawling the same URL's over and over.
Qwantify
VelenPublicWebCrawler
CCBot
ZoominfoBot
Seekport
They appear to be linked as they all crawl the same handful of pages at the same rates and all ignore robots.txt and I'm often finding 3-4 of the above bots hitting the same page on the same seconds continually.
from nginx-ultimate-bad-bot-blocker.
Related Issues (20)
- Updated BingBot IP Ranges HOT 7
- [User-Agent] BrandVerity HOT 1
- [User-Agent] AwarioBot HOT 3
- [User-Agent] Bytespider, Bytedance didn't get to block after install bad bot blocker
- linkfluence HOT 4
- AwarioBot Addition HOT 1
- [INSTALLATION] nginx -t return warn HOT 1
- Whitelist Yahoo HOT 2
- [User-Agent] InternetMeasurement
- [BUG] Nginx logs shows permissions denied error to /etc/nginx/conf.d/globalblacklist.conf HOT 1
- New "super rate limiting" zone (1r/m) in addition of the current one (2r/s) HOT 14
- blacklist-ips.conf does not do anything for me
- [User-Agent] GeedoProductSearch
- [User-Agent] Information Security Team InfraSec Scanner
- [INSTALLATION] Fail2ban/crowdsec and their interaction with this project, worth it? HOT 1
- [BUG] (Please add a way to uninstall completely) HOT 2
- [BUG] iptables-common.conf not in fail2ban v1.1.0
- [BUG] Certbot fails
- [BUG] facebookexternalhit is NOT a good bot HOT 7
- [User-Agent] ClaudeBot - Bad AI user-agent HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nginx-ultimate-bad-bot-blocker.