[BUG] facebookexternalhit is NOT a good bot about nginx-ultimate-bad-bot-blocker HOT 7 OPEN

Can you post the IP addresses that bot is using so we can establish that they belong to Facebook/Meta and is not an impersonator using their User Agent to trick people.

from nginx-ultimate-bad-bot-blocker.

This is the same bot that is blocked in the robots.txt file under: FacebookBot (and also facebookscraper).

from nginx-ultimate-bad-bot-blocker.

I found many IPs from this block: 173.252.64.0/18. For example 173.252.83.23 is one of them, but instead of posting individual IPs, I gave you a full subnet.

Another one besides 173.252.64.0/18 would be: 66.220.144.0/20, and also: 69.171.224.0/19... to get you started. A single 19 subnet prefix can have more than 8000+ hosts in theory. 18 subnet prefix can even hold more than 16k IPs.

from nginx-ultimate-bad-bot-blocker.

I got hits from:

2a03:2880::
66.220.149.0
173.252.83.0
173.252.107.0
69.171.249.0
57.141.0.0

from nginx-ultimate-bad-bot-blocker.

I got hits from:

2a03:2880::
66.220.149.0
173.252.83.0
173.252.107.0
69.171.249.0
57.141.0.0

These all resolve to Facebook so are legitimate. How much crawling are they doing can you post some log examples?

from nginx-ultimate-bad-bot-blocker.

I found many IPs from this block: 173.252.64.0/18. For example 173.252.83.23 is one of them, but instead of posting individual IPs, I gave you a full subnet.

Another one besides 173.252.64.0/18 would be: 66.220.144.0/20, and also: 69.171.224.0/19... to get you started. A single 19 subnet prefix can have more than 8000+ hosts in theory. 18 subnet prefix can even hold more than 16k IPs.

These also are IP owned by Facebook --- what exactly were they crawling that they used up 50GB ????

from nginx-ultimate-bad-bot-blocker.

For now you will have to add that user agent to your own custom include file with a value of 3 and reload nginx, this will block it outright. Unfortunately this can't be done for the thousands using this blocker as this would be a seriously breaking change

Add to: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/bots.d/blacklist-user-agents.conf

	"~*(?:\b)facebookexternalhit(?:\b)"			3;

GitHub

nginx-ultimate-bad-bot-blocker/bots.d/blacklist-user-agents.conf at master · mitchellkrogza/nginx-ultimate-bad-bot-blocker

Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail f...

from nginx-ultimate-bad-bot-blocker.