Giter Club home page Giter Club logo

hostlistsregistry's Introduction

AdGuard Host Lists Registry

This repository contains the known hosts blocklists that are made available to the users of AdGuard products (AdGuard DNS, AdGuard Home, etc).

Some of these blocklists are automatically converted to the rules format that AdGuard product understand better.

Where Blocklists Are Published

  • filters.json contains all the blocklists added to the repo. downloadURL is the location of the re-hosted blocklist.
  • services.json is the meta-data of "Web Services". This is a part of the parental control functionality of AdGuard Home and AdGuard DNS.

What Blocklists Can Be Added Here

  • The blocklist should be oriented towards DNS-level content blockers. There is a different repo for browser content blockers.

  • We prefer (mostly) original lists to compilations.

  • We prefer blocklists that are specifically made for AdGuard Home and use adblock-style syntax. Lists that use /etc/hosts can also be added if there is no alternative.

  • The blocklist should have a clear purpose.

    Examples:

    • good: "blocks TV ads".
    • bad: "John Doe's personal blocklist"
  • The blocklist should have a place for receiving user complaints and holding discussions, such as a repository on github.com, or a public website/forum.

  • The blocklist should be relatively popular, meaning:

    • if there is a repository on GitHub, the number of stars should be at least 50.
    • if there is no repository on GitHub, the number of reported issues and discussions should be at least 10 per month.
    • the blocklist should be actively supported for at least 6 months.
  • The blocklist should be regularly updated with at least 10 updates per month.

  • Previously added blocklists that haven't received any support for a year will be removed. We reserve the right to remove the blocklist earlier, depending on circumstances.

  • If the blocklist contains too many problematic rules, it will not be added. A rule is considered problematic if it causes false positives or otherwise displays unitended behavior. Decisions about blocklists with problematic rules are arbitrary and there may be exceptions.

  • If the blocklist intentionally blocks or restricts access to a service for no reason other than being a reflection of the filter author's opinion, the blocklist will not get added, or will get removed if already added.

  • If the blocklist is popular in a specific region and there are no alternatives to it, then it can be added as is even if it does not satisfy requirements above.

How to add a filter list

To add a new filter, you need to:

  1. Go to the filters folder.
  2. Go to the category to which the new filter will belong.
  3. Create a new folder with the name filter_(id)_(filter name).
  4. Create two files in this folder: configuration.json and metadata.json. The contents of these files are described below.
  5. In the file locales/en/filters.json add an object with the name and description of the filter according to the example added above.

The files /assets/filters.json and /assets/filters-dev.json must not be edited manually.

Filters Metadata

  • metadata.json

    Filter metadata. Includes name, description, etc.

    • filterId — unique human-readable filter identifier (string)

    • id - unique internal filter identifier (integer)

    • name — filter name; can be localized

    • description — filter description

    • timeAdded — time when this filter was added to the registry; milliseconds since January 1, 1970; you can exec new Date().getTime() in the browser console to get the current time

    • homepage — filter website/homepage

    • expires — filter's default expiration period

    • displayNumber — this number is used when AdGuard sorts available filters (GUI)

    • environment - either dev or prod. Only prod lists are available in AdGuard DNS.

    • disabled - if set to true, the blocklist won't be updated.

    • tags — a list of tags

      Metadata example
    {
      "filterId": "adguard_dns_filter",
      "id": 1,
      "name": "AdGuard DNS filter",
      "description": "Filter composed of several other filters (AdGuard Base filter, Social Media filter, Tracking Protection filter, Mobile Ads filter, EasyList and EasyPrivacy) and simplified specifically to be better compatible with DNS-level ad blocking.",
      "timeAdded": 1404115015843,
      "homepage": "https://kb.adguard.com/general/adguard-ad-filters",
      "expires": "4 days",
      "displayNumber": 3,
      "environment": "prod",
      "tags": []
    }
  • revision.json

    Filter version metadata, automatically filled and overwritten on each build.

  • filter.txt

    Resulting compiled filter.

  • configuration.json

    Configuration defines your filter list sources, and the transformations that are applied to the sources. See Hostlist compiler configuration for details

Tags

Every filter can be marked by a number of tags. Every tag metadata listed in /tags/metadata.json.

Example
{
  "tagId": 1,
  "keyword": "purpose:ads"
}

Possible tags:

  • lang:* — for language-specific filters; one or multiple lang-tags can be used. For instance, AdGuard Russian filter is marked with the lang:ru tag.

  • purpose:* — determines filters purposes; multiple purpose-tags can be used for one filter list.

  • recommended — for low-risk filter lists which are recommended to use in their category. The category is determined by the pair of the lang:* and purpose:* tags.

  • obsolete — for abandoned filter lists; filter's metadata with this tag will be excluded from filters.json and filters_i18n.json.

How to add a blockable service

To add a new blocked service, a new .yml file must be added to the services directory, the contents of the directory and the resulting services.json file are described in the next section.

Services Metadata

The services.json file combines information about services received from separate .yml files located in the services directory. The services.json file is generated by a script, so there is no need to modify it manually.

  • Source .yml files: Each .yml file contains information about a specific service. The file name must match its id.

  • File saving: .yml files are stored in the services directory.

  • Updating information: In cases when it is necessary to change the configuration of a service, it is necessary to update the corresponding .yml file in the services directory.

  • Adding a new service: to add a new service to services.json it is necessary to add .yml file to the services directory, after the script is run the file will be updated.

WARNING!

  • Deletion of files is strictly forbidden. If a service file is deleted, the script will restore it from a previous version of the services.json file.

  • It is also forbidden to change service id inside the .yml file.

Service metadata includes id, name, rules, icon_svg:

  • id — unique human-readable service identifier (string, as snake_case)
  • name — service name (string)
  • rules — list of domain rules in Adblock syntax (list of strings)
  • icon_svg — svg icon (string)
Service examples

services.json - wechat object

{
  "id": "wechat",
  "name": "WeChat",
  "rules": ["||wechat.com^", "||weixin.qq.com^", "||wx.qq.com^"],
  "icon_svg": "<svg xmlns=\"http://www.w3.org/2000/svg\" className=\"d-none\"><symbol id=\"service_wechat\" viewBox=\"0 0 50 50\" fill=\"currentColor\"><path d=\"M 19 6 C 9.625 6 2 12.503906 2 20.5 C 2 24.769531 4.058594 28.609375 7.816406 31.390625 L 5.179688 39.304688 L 13.425781 34.199219 C 15.714844 34.917969 18.507813 35.171875 21.203125 34.875 C 23.390625 39.109375 28.332031 42 34 42 C 35.722656 42 37.316406 41.675781 38.796875 41.234375 L 45.644531 45.066406 L 43.734375 38.515625 C 46.3125 36.375 48 33.394531 48 30 C 48 23.789063 42.597656 18.835938 35.75 18.105469 C 34.40625 11.152344 27.367188 6 19 6 Z M 13 14 C 14.101563 14 15 14.898438 15 16 C 15 17.101563 14.101563 18 13 18 C 11.898438 18 11 17.101563 11 16 C 11 14.898438 11.898438 14 13 14 Z M 25 14 C 26.101563 14 27 14.898438 27 16 C 27 17.101563 26.101563 18 25 18 C 23.898438 18 23 17.101563 23 16 C 23 14.898438 23.898438 14 25 14 Z M 34 20 C 40.746094 20 46 24.535156 46 30 C 46 32.957031 44.492188 35.550781 42.003906 37.394531 L 41.445313 37.8125 L 42.355469 40.933594 L 39.105469 39.109375 L 38.683594 39.25 C 37.285156 39.71875 35.6875 40 34 40 C 27.253906 40 22 35.464844 22 30 C 22 24.535156 27.253906 20 34 20 Z M 29.5 26 C 28.699219 26 28 26.699219 28 27.5 C 28 28.300781 28.699219 29 29.5 29 C 30.300781 29 31 28.300781 31 27.5 C 31 26.699219 30.300781 26 29.5 26 Z M 38.5 26 C 37.699219 26 37 26.699219 37 27.5 C 37 28.300781 37.699219 29 38.5 29 C 39.300781 29 40 28.300781 40 27.5 C 40 26.699219 39.300781 26 38.5 26 Z\" /></symbol></svg>"
}

wechat.yml

id: wechat
name: WeChat
rules:
- '||wechat.com^'
- '||weixin.qq.com^'
- '||weixin.qq.com.cn^'
- '||weixinbridge.com^'
- '||wx.qq.com^'
icon_svg: <svg xmlns="http://www.w3.org/2000/svg" fill="currentColor" viewBox="0 0 50 50"><path d="M 19 6 C 9.625 6 2 12.503906 2 20.5 C 2 24.769531 4.058594 28.609375 7.816406 31.390625 L 5.179688 39.304688 L 13.425781 34.199219 C 15.714844 34.917969 18.507813 35.171875 21.203125 34.875 C 23.390625 39.109375 28.332031 42 34 42 C 35.722656 42 37.316406 41.675781 38.796875 41.234375 L 45.644531 45.066406 L 43.734375 38.515625 C 46.3125 36.375 48 33.394531 48 30 C 48 23.789063 42.597656 18.835938 35.75 18.105469 C 34.40625 11.152344 27.367188 6 19 6 Z M 13 14 C 14.101563 14 15 14.898438 15 16 C 15 17.101563 14.101563 18 13 18 C 11.898438 18 11 17.101563 11 16 C 11 14.898438 11.898438 14 13 14 Z M 25 14 C 26.101563 14 27 14.898438 27 16 C 27 17.101563 26.101563 18 25 18 C 23.898438 18 23 17.101563 23 16 C 23 14.898438 23.898438 14 25 14 Z M 34 20 C 40.746094 20 46 24.535156 46 30 C 46 32.957031 44.492188 35.550781 42.003906 37.394531 L 41.445313 37.8125 L 42.355469 40.933594 L 39.105469 39.109375 L 38.683594 39.25 C 37.285156 39.71875 35.6875 40 34 40 C 27.253906 40 22 35.464844 22 30 C 22 24.535156 27.253906 20 34 20 Z M 29.5 26 C 28.699219 26 28 26.699219 28 27.5 C 28 28.300781 28.699219 29 29.5 29 C 30.300781 29 31 28.300781 31 27.5 C 31 26.699219 30.300781 26 29.5 26 Z M 38.5 26 C 37.699219 26 37 26.699219 37 27.5 C 37 28.300781 37.699219 29 38.5 29 C 39.300781 29 40 28.300781 40 27.5 C 40 26.699219 39.300781 26 38.5 26 Z" /></svg>

SVG icon requirements:

  • The icon must have a square shape, i.e. the third and fourth parameters of the viewBox attribute must be equal.
  • The icon must be black and white.
  • It should be as small as possible.
  • Svg tag must contain fill="currentColor" attribute. This is important for themes to work properly.
  • Svg tag must not contain width and height attributes

Safe Search Filters

  • engines_safe_search.txt contains a set of rules to enforce Safe Search in the following searching engines:

    • Bing
    • DuckDuckGo
    • Google
    • Pixabay
    • Yandex
  • youtube_safe_search.txt contains a set of rules to enforce Safe Search and hide comments on YouTube.

How to Build

yarn install

Run the following command:

yarn run compose

The build result can be found in the assets directory.

Localizations

Blocklist names, descriptions, and tags' names are translated via crowdin.

/locales contains translations for filters, groups, and tags.

Base language strings:

  • locales/en/filters.json
  • locales/en/tags.json

In order to prepare these files run yarn run locales:prepare. This script will scan filters meta and add filters names, descriptions, and tags meta to the base language files.

  • Upload base language strings: yarn run locales:upload
  • Download translations: yarn run locales:download

hostlistsregistry's People

Contributors

0m0r3l avatar ainar-g avatar alex-302 avatar ameshkov avatar arfshl avatar ars28fox avatar atropnikov avatar dandelionsprout avatar eugeneone1 avatar gakudva avatar github-actions[bot] avatar gork7777 avatar hagezi avatar ildarkamalov avatar jamesalfei avatar jellizaveta avatar knightmareviiviixc avatar l8x avatar litong avatar masterain98 avatar mkqts avatar nwpr avatar paragm avatar piquark6046 avatar rootmelo92118 avatar sandervankasteel avatar sergey-lyapin avatar slavaleleka avatar tg-twilight avatar umarhadi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hostlistsregistry's Issues

Remove Energized Spark

I feel the matter is urgent enough that 4b35ee6#commitcomment-74500604 alone doesn't get the point through fast enough.

In addition to https://github.com/EnergizedProtection/block/issues/916, Energized Protection is a mess where the one remaining admin doesn't do anything (All of the commits are just his bot), other team members don't have sufficient rights to fix the issue backlog (Take my word for it. I was one of those team members for a while), and I don't recommend using the list for any purpose by any person. It was already changed from default to non-default in Blokada due to all this if I understand things correctly, and is reportedly close to being removed from PersonalDNSblocker for the same reasons.

Add Dandelion Sprout's Anti-Malware List

List's URL: https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareAdGuardHome.txt
Support URL: https://github.com/DandelionSprout/adfilt/issues
Advantages:

  • Syntax is tailorsewn for AdGuard Home, to the point it even uses $denyallow.
  • Three years in the making, with 2 active maintainers.
  • Most effective tool to prevent malware redirections in the history of the internet.
  • Its version for regular adblockers forms part of Dandelion Sprout's Annoyances List, thus widening the theorethical pool of user reports if any list problems occur.

Add 1Hosts (Lite)

Please consider adding 1Hosts (lite) blocklist to Adguard DNS.

https://github.com/badmojr/1Hosts

I have been using it on NextDNS for months and it's one with most of the blocks and I haven't seen any false flags with it.

In total, I have got best blocking coverage with these filters in NextDNS. These are recommended by many other NextDNS users so I recommend to consider to add the missing ones.

1Hosts (Lite) (https://badmojr.github.io/1Hosts)
oisd (oisd.nl)
notracking (https://github.com/notracking/hosts-blocklists)
Steven Black (github.com/StevenBlack/hosts)
AdGuard DNS filter

"Game Console Adblock List" becomes nonsense without all its "!"-comments

At the time of writing, https://github.com/AdguardTeam/HostlistsRegistry/blob/main/filters/other/filter_6_GameConsole/filter.txt does not include https://github.com/DandelionSprout/adfilt/blob/master/GameConsoleAdblockList.txt 's very extensive commentary on which consoles it cover, and which consoles I need help from enthusiasts with.

The scenario I'm imagining, is if someone with an Xbox Series X turns on the list, expects it to work with that console out of the box, and then becomes confused when it doesn't.

Please add my blocklists

Hello,

please, add my blocklists:

AmmoniteFilter. Repo here:
https://github.com/AmmoniteFilter/AmmoniteFilter
Direct URL here:
https://ammnt.app/filter.txt
Tags: ads, privacy, tracking, phishing, malware, NFSW, adult, family, parental etc.
Syntax: adblock
License: GPL-3.0
Software: AdGuard Home
Home: https://ammnt.app/
email: [email protected]

TrilobiteFilter. Repo here:
https://github.com/AmmoniteFilter/TrilobiteFilter
Direct URL here:
https://raw.githubusercontent.com/AmmoniteFilter/TrilobiteFilter/main/filter.txt
Tags: phishing, ransomware, malware, iOCs, security, threat intelligence, fraud, fake etc.
Syntax: adblock
License: GPL-3.0
Software: AdGuard Home
Home: https://ammnt.app/
email: [email protected]

Thank you.
Cheers😙

PersianBlocker list for Persian websites

AdguardTeam/FiltersRegistry#617

PersianBlocker is currently available in uBlock Origin and AdGuard extensions.

I'm in contact with @DRSDavidSoft and therefore I've fully combined the Unwanted-Iranian list from @DRSDavidSoft into the PersianBlocker list and created a list suitable for DNS-level blockers, PersianBlockerHosts:
https://raw.githubusercontent.com/MasterKia/PersianBlocker/main/PersianBlockerHosts.txt

Unwanted-Iranian is currently a part of AdGuard Home but it has some dead links, so the PersianBlockerHosts list (or you could use the name "PersianBlocker (Hosts)") could either replace it or be added alongside it.

Add List-KR DNS

Please add List-KR DNS.

Subscription URL:

https://github.com/List-KR/List-KR/raw/master/filter_DNS.txt

Add EasyList Lithuania Hosts

Whitelists section

I don't have the expertise to send a PR for this report, however Honey which is a discount finder, requires a whitelist to function:

https://www.joinhoney.com/whitelist/honey-smart-shopping.txt

Would be nice if it was added here, so that it could be easily enabled on AdGuard DNS and inside the AdGuard apps.

My current blocklist-setup

My current blocklist setup with another service that gives me great privacy with minimal white-listing. Would be nice to try the same setup here...

  • 1Hosts (Pro)
  • Lightswitch05 - Ads & Tracking
  • notracking
  • oisd (full)
  • AdGuard DNS filter
  • Fanboy's Annoyance List
  • Steven Black

Energized Spark list

Hello I have a question will we be able to add custom host lists in AdGuard dns when the service launches properly out of beta ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.