Giter Club home page Giter Club logo

fake-bot-plugin's Issues

Inverted block

Fake Bot plugin blocks valid bots and allows fake bots.

Sample googlebot transaction:

{
  "transaction": {
    "client_ip": "66.249.64.219",
    "time_stamp": "Tue Jul 12 15:54:19 2022",
    "server_id": "9e7bc6878f15155f664887f5952d257c0d032745",
    "client_port": 61702,
    "host_ip": "255.255.255.255",
    "host_port": 443,
    "unique_id": "1657641259",
    "request": {
      "method": "GET",
      "http_version": 1.1,
      "uri": "/",
      "headers": {
        "Host": "redacted.com",
        "AMP-Cache-Transform": "google;v=\"1..8\"",
        "Connection": "keep-alive",
        "Accept": "text/html,application/xhtml+xml,application/signed-exchange;v=b3,application/xml;q=0.9,*/*;q=0.8",
        "User-Agent": "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
        "From": "googlebot(at)googlebot.com",
        "Accept-Encoding": "gzip, deflate, br",
        "If-Modified-Since": "Tue, 24 May 2022 23:58:16 GMT"
      }
    },
    "response": {
      "body": "",
      "http_code": 403,
      "headers": {}
    },
    "producer": {
      "modsecurity": "ModSecurity v3.0.6 (FreeBSD)",
      "connector": "ModSecurity-nginx v1.0.3",
      "secrules_engine": "Enabled",
      "components": [
        "OWASP_CRS/4.0.0-rc1\""
      ]
    },
    "messages": [
      {
        "message": "Fake bot detected: ",
        "details": {
          "match": "Matched \"Operator `Pm' with parameter `applebot bingbot facebookbot facebookcatalog facebookexternalhit googlebot twitterbot' against variable `REQUEST_HEADERS:User-Agent' (Value: `Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chr (100 characters omitted)' )",
          "reference": "o153,9v265,200",
          "ruleId": "9504110",
          "file": "/usr/local/share/modsecurity-crs/plugins/fake-bot-after.conf",
          "lineNumber": "19",
          "data": "Matched Data: if-modified-since found within REQUEST_HEADERS:User-Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
          "severity": "2",
          "ver": "fake-bot-plugin/1.0.0",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-bot",
            "capec/1000/225/22/77/13",
            "PCI/6.5.10",
            "paranoia-level/1"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      }
    ]
  }
}

Sample fake transaction curl "https://redacted.com/" -H "User-Agent: asd googlebot asd":

{
  "transaction": {
    "client_ip": "1.2.3.4",
    "time_stamp": "Tue Jul 12 15:59:08 2022",
    "server_id": "9e7bc6878f15155f664887f5952d257c0d032745",
    "client_port": 64532,
    "host_ip": "255.255.255.255",
    "host_port": 443,
    "unique_id": "1657641548",
    "request": {
      "method": "GET",
      "http_version": 2,
      "uri": "/",
      "headers": {
        "host": "redacted.com",
        "accept": "*/*",
        "user-agent": "asd googlebot asd"
      }
    },
    "response": {
      "body": "redacted",
      "http_code": 200,
      "headers": {}
    },
    "producer": {
      "modsecurity": "ModSecurity v3.0.6 (FreeBSD)",
      "connector": "ModSecurity-nginx v1.0.3",
      "secrules_engine": "Enabled",
      "components": [
        "OWASP_CRS/4.0.0-rc1\""
      ]
    },
    "messages": [
      {
        "message": "Fake bot detected: Googlebot",
        "details": {
          "match": "Matched \"Operator `InspectFile' with parameter `fake-bot.lua' against variable `TX:0' (Value: `googlebot' )",
          "reference": "o0,9v69,9",
          "ruleId": "9504110",
          "file": "/usr/local/share/modsecurity-crs/plugins/fake-bot-after.conf",
          "lineNumber": "19",
          "data": "Matched Data: googlebot found within REQUEST_HEADERS:User-Agent: Googlebot",
          "severity": "2",
          "ver": "fake-bot-plugin/1.0.0",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-bot",
            "capec/1000/225/22/77/13",
            "PCI/6.5.10",
            "paranoia-level/1"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      }
    ]
  }
}

Using fb1381f, modsecurity 3.0.6, modsecurity-nginx 1.0.2, nginx 1.22.0.

Use "Impersonator"

really useful plugin @azurit !

Is it possible to use "Impersonator" instead of "fake bot"? According to other vendors, "Impersonator" should be the right category name of this kind of "bad bots".

Lots of FP from apple imessages

Hi there,

apple devices with iMessages using:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0

Will need to disable this plugin and/or modify lua/rule on my own to take care of these requests! (lots of)

Thanks for your work!

Fix wording in README

Testing

After configuration, antivirus protection should be tested, for example, using:
curl http://localhost --header "User-Agent: Googlebot"

It would also be useful if you would indicate the expected behaviour and log entry.

CDN request false positive

Hello, i use CDN on my wensite and somes request are redirected from.

Possible to add exeption ?

Exemple:

[Mon Feb 21 04:15:43.895269 2022] [:error] [pid 759926:tid 140629216577280] [client 137.74.122.3:28024] [client 137.74.122.3] ModSecurity: Warning. Fake Bot Plugin: Detected fake Bingbot. [file "/etc/modsecurity/plugins/fake-bot-after.conf"] [line "27"] [id "9504110"] [msg "Fake bot detected: Bingbot"] [data "Matched Data: bingbot found within REQUEST_HEADERS:User-Agent: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"] [severity "CRITICAL"] [ver "fake-bot-plugin/1.0.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-bot"] [tag "capec/1000/225/22/77/13"] [tag "PCI/6.5.10"] [tag "paranoia-level/1"] [hostname "cdn.achatpc.com"] [uri "/media/catalog/product/cache/7d38c09cea45996b9bdc3949df76cebd/icecatconnect/48/1053924807500_6877864685.jpg"] [unique_id "YhMD3xtye7aMD6dFGT_JYAAAFAY"]

Thank you.

Support for Twitterbot.

I saw the following UA faking twitterbot:

twitterbot/1.0

Do not know if a real twitterbot is scanning the net.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.