Giter Club home page Giter Club logo

bad-bot-blocker's Introduction

Bad Bot Blocker

223 Apache htaccess rules to block bad bots.

Bad bots are defined as:

  • E-mail harvesters
  • Content scrapers
  • Spam bots
  • Vulnerability scanners
  • Aggressive bots that provide little value
  • Bots linked to viruses or malware
  • Government surveillance bots
  • Russian search engine Yandex
  • Chinese search engine Baidu

Yandex/Baidu

Unless your website is written in Russian or Chinese, you probably don't get any traffic from them. They mostly just waste bandwidth and consume resources.

Bots Are Liars

Bots try to make themselves look like other software by disguising their useragent. Their useragents may look harmless, perfectly legitimate even. For example, "^Java" but according to Project Honeypot, it's actually one of the most dangerous.

Setup

If you have a bizarre or complicated setup, be sure to look everything over before using it. But for anyone with something that resembles a standard Apache installation, this should work without any issues.

bad-bot-blocker's People

Contributors

bluedragonz avatar firesphere avatar instancezero avatar leonb avatar ryankearney avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

floofydoug miekrr darshit-websmith libin leonb elitelinux vpadhariya instancezero firesphere ryanplant123 kijin jprice nicolabavaro treiber mariusv hugominas chrisfu quacos salmoomen mazemax nolybom iamkingsleyf cybercog tecumsehmaverick zenlabisrael hammerspb zimamedia abiodun alaev darongmean ezanltd solcre pagelab modulexcite leonirlopes kamran11 mallat flatcapco codehead cptplastic udienz cloudxtreme mikostn josephtung kylerchin mitchellkrogza crossbit kevin50406418 romanadamita mijndesign wesavetheworld mrcatania davidsayre ianwork d0ng4667 kakuye andydigi sweettddy johnsage25 helresa gronau-it-cloud-computing murtuza-alientechsol maharramoff webilav pr-olga helturkey nsrau fenomennal21 degerli karendev talyne67 dev-cho aemre kezenwa ducquyen felix-d1strict cc-robc webmix-dev transource densol92 devkwon97 haidang9x blackjyn scorpio18 ip-cam alliline

bad-bot-blocker's Issues

Yandex

yandex is good bot ))
its more popular that google in Russia and have some audience in exUSSR, Turkey and so on

Thanks + question

Hi
Thank you for this resource. I found this as site ground staff recommending this as a resource.
I have a wordpress site and have a question on any side effects of having these lines -

Options All -Indexes
RewriteEngine on

I already have the later in a block so have removed it. The other I am not sure about or if I need it. I am also using this in a parent folder for a few sub sites to all share the same bad crawl traffic.

Thanks

ZmEu

Please, add the user agent "ZmEu".

Typo in var name

SetEnvIfNoCase User-Agent "^LinkScan/8.1a.Unix" bad_bo
SetEnvIfNoCase User-Agent "^LNSpiderguy" bad_bott

should be:
SetEnvIfNoCase User-Agent "^LinkScan/8.1a.Unix" bad_bot
SetEnvIfNoCase User-Agent "^LNSpiderguy" bad_bot

Nginx rules

Are there plans to make this rules for Nginx Webserver?

Preventing Website Scraping

How can i specifically block a site or any type of bot using by that site to scrap my content. It is scraping my content and publishing as their own.
Please help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.