squidGuard/ufdbGuard Adblock Plus Expression Lists Converter

A shell script that converts Adblock Plus lists into expressions files compatible with squidGuard and ufdbGuard

Many similar scripts exist to convert such lists, but some are quite old and use sed patterns that actually cause problems.

Dependencies

• bash
• sed
• wget
• squid
• squidGuard or ufdbGuard

Setup

The script is designed to automatically detect the required aspects of your Squid and squidGuard/ufdbGuard configuration, paths, file locations etc. When you run the script it will output what it thinks is correct, with the ability to stop the script if anything is wrong.

While I try and test this script on multiple Linux distributions, there may be issues with detection in some cases. If you experience any problems, please file an issue on GitHub and I'll look into it.

Installation

No formal installation is required, other than to get the source and to extract it to any directory on the machine you are running Squid on.

You will need to grant executable permission to the shell script before running it:

chmod +x get-easylist.sh

See usage section for instructions on how to use the script

Usage

The script requires a couple of user parameters based on what setup you are running:

./get-easylist.sh [squidGuard/ufdbGuard] [autoconfirm]

For additional guidance, you can run the script without any parameters which will show the help message

Examples:

squidGuard

./get-easylist.sh squidGuard

ufdbGuard

./get-easylist.sh ufdbGuard

The lists will be automatically be downloaded, converted and written to the database folder of the respective URL filter. The conversion will need a bit of processing power as they can be quite large.

Adding the converted Adblock expression files to your URL filter

squidGuard

dest adblock {
	expressionlist adblock/easylist
	expressionlist adblock/easyprivacy
}

acl {
	default {
		pass !adblock all
	}
}

ufdbGuard

category adblock {
	expressionlist  adblock/easylist
	expressionlist  adblock/easyprivacy
}

acl {
	default {
		pass !adblock any
	}
}

Replacing advertisements with a transparent image

You should also consider putting in a redirect directive in your dest/category adblock config, to serve a transparent image that will replace the original advert space/content when a expression rule matches. You can find a 1x1 blank.gif image within the source. This needs to be hosted on a web server and referenced like so in a dest or category block:

squidGuard

redirect http://yourwebsite.com/blank.gif

The image file could also be hosted on an internal server, as long as the server is accessible when using the proxy.

ufdbGuard

With ufdbGuard you have the option to run a local Apache HTTPD instance which will serve the transparent image for you as it already includes one, the settings for Apache are present in the ufdbGuard.conf. An example redirect in ufdbGuard that is generally non-intrusive is below:

redirect http://your-proxy-server.com:8080/cgi-bin/URLblocked.cgi?admin=%A&mode=default&color=red&size=normal&clientaddr=%a&clientname=%n&clientuser=%i&clientgroup=%s&targetgroup=%t&url=%u

Make sure you change the proxy address and port number to match your setup. It doesn't necessarily have to be a registered DNS name, an IP address will also be sufficient.

Automatic updates of expressions lists via cron

The Adblock list files themselves are updated quite regularly, you can also run this script through cron, you just need to pass an additional parameter to avoid the user confirmation prompt:

./get-easylist.sh [squidGuard/ufdbGuard] autoconfirm

You can then schedule the job through cron, be sure to update the path of where you've actually stored the script, an example cron could be:

0 0 * * * /path/to/get_easylist.sh squidGuard autoconfirm >/dev/null 2>&1

This would run the script everyday at midnight 12 AM (00:00 AM) for squidGuard

Comparison to Adblock Plus Browser Addon

There are advantages and disadvantages of running Adblock expression lists through Squid, I'll cover them here:

Advantages

• Filtering is applied at the proxy level, no need for browser addons for each client
• Enables ad filtering for devices that don't support browser addons but can use a user-defined HTTP proxy
• Useful for devices with low RAM as the proxy will take care of all the filtering
• Being able to point mutiple clients at the proxy means you don't have to configure each client with ad blocking
• Works with Squid as a transparent proxy

Disadvantages

• Cannot detect JavaScript based ads
• Some ad space can sometimes be left behind

Contributing

Pull Requests and feedback welcome!