privoxy-blocklist

Script converting AdBlock Plus rules into privoxy format.

How does it work

The script privoxy-blocklist.sh downloads AdBlock Plus filter files and generates privoxy compatible filter and action files based on these. After the generation is done it modifies the privoxy configuration files /etc/privoxy/config to import the generated files.

Due to this behaviour the script must run as root user to be able to modify the privoxy configuration file.

Usage

Either run privoxy-blocklist.sh manually with root privileges (e.g., sudo privoxy-blocklist.sh) or via root cronjob.

To activate content filters specify the corresponding filter types either in the configuraiton file or via cli-flag -f, e.g.:

privoxy-blocklist.sh -f attribute_global_name -f attribute_global_exact -f attribute_global_contain -f attribute_global_startswith -f attribute_global_endswith -f class_global -f id_global

To see all supported filter types check the help privoxy-blocklist.sh -h.

Installation

1. Install all dependencies:
   • privoxy
   • sed
   • grep
   • bash
   • wget
   • can be simplified by running helper/install_deps.sh which support Debian, ArchLinux and Alpine based installation
2. Download privoxy-blocklist.sh from the asset list of latest release

Feature Support

The following table shows features of AdBlock Plus filters and there status within privoxy-blocklist:

Feature	Type	Status	Test
:-abp-contains()	extended CSS selector	❓	❓
:-abp-has()	extended CSS selector	❓	❓
:-abp-properties()	extended CSS selector	❓	❓
||…	block domain matching excluding scheme	✅	✅
|…|	block exact domain matching including scheme	❓	❓
!…	comments	✅
csp=	filter options	❓	❓
##.class	global CSS attribute selector with matching for class	✅ (via -f class_global)	✅
###id	global CSS attribute selector with matching for id	✅ (via -f id_global)	✅
##[attribute]	global CSS attribute selector with matching for attribute-name	✅ (via -f attribute_global_name)	✅
##[attribute=value]	global CSS attribute selector with matching for attribute-value pair	✅ (via -f attribute_global_exact)	✅
##[attribute^=value]	global CSS attribute selector with matching for attribute with value starting with	✅ (via -f attribute_global_startswith)	✅
##[attribute$=value]	global CSS attribute selector with matching for attribute with value ending with	✅ (via -f attribute_global_endswith)	✅
##[attribute*=value]	global CSS attribute selector with matching for attribute with value containing	✅ (via -f attribute_global_contain)	✅
##html-tag[attribute]	global CSS attribute selector for html-tag with matching for attribute-name	🚧	🚧
##html-tag[attribute=value]	global CSS attribute selector for html-tag with matching for attribute-value pair	🚧	🚧
##html-tag[attribute^=value]	global CSS attribute selector for html-tag with matching for attribute with value starting with	🚧	🚧
##html-tag[attribute$=value]	global CSS attribute selector for html-tag with matching for attribute with value ending with	🚧	🚧
##html-tag[attribute*=value]	global CSS attribute selector for html-tag with matching for attribute with value containing	🚧	🚧
[…]#$#	domain based CSS selector - Snippet filter	❓	❓
[…]##	domain based CSS selector - Element hiding	❓	❓
[…]#?#	domain based CSS selector - Element hiding emulation	❓	❓
[…]#@#	domain based CSS selector - Element hiding exception	❓	❓
document	filter options	❓	❓
~domain=	filter options	❓	❓
domain=	filter options	❓	❓
~elemhide	filter options	❓	❓
elemhide	filter options	❓	❓
@@||…	exception for domain blocking rules	✅	🚧
font	filter options	❓	❓
genericblock	filter options	❓	❓
generichide	filter options	❓	❓
~image	filter options	❓	❓
image	filter options	✅	🚧
match-case	filter options	❓	❓
media	filter options	❓	❓
~object	filter options	❓	❓
object	filter options	❓	❓
~other	filter options	❓	❓
other	filter options	❓	❓
~ping	filter options	❓	❓
ping	filter options	❓	❓
popup	filter options	❓	❓
rewrite=	filter options	❓	❓
~script	filter options	❓	❓
script	filter options	❓	❓
sitekey=	filter options	❓	❓
~stylesheet	filter options	❓	❓
stylesheet	filter options	❓	❓
~subdocument	filter options	❓	❓
subdocument	filter options	❓	❓
~third-party	filter options	❓	❓
third-party	filter options	❓	❓
~webrtc	filter options	❓	❓
webrtc	filter options	❓	❓
~websocket	filter options	❓	❓
websocket	filter options	❓	❓
~xmlhttprequest	filter options	❓	❓
xmlhttprequest	filter options	❓	❓

• ❓ => status must be checked
• ✅ => implemented
• 🚧 => work in progress

Sources:

• Adblock Plus Help
• Adblock Plus Cheatsheet

Development

Release

The release process is automated via github action Release and triggered by pushing a tag to the main branch.

The following tags are recognized:

Tag-Schema	Result
[0-9]+.[0-9]+.[0-9]+	create public release e.g., 0.4.0
[0-9]+.[0-9]+.[0-9]+-a	create private alpha release e.g., 0.4.0-a (only visible to collaborators)
[0-9]+.[0-9]+.[0-9]+-rc	create public release candidate e.g., 0.4.0-rc (pre-release)

Tests

Code changes must be tested to ensure that all functionality is working as intended. For that a pytest based test suite is maintained and runs on every pull request within Gitlab Actions.

Run Local Tests

The test suite is designed to run within a docker container based on the definition of this repository. It is currently only tested on Ubuntu Linux but should work on every system with a POSIX compliant shell and docker.

To start all tests of the test suite just run:

./tests/run.sh

To start a single test file you can run:

./tests/run.sh tests/test_….py

To start all tests on ubuntu:

./tests/run.sh -o ubuntu

The whole process simplified by ./tests/run.sh runs the following:

1. build docker image based on project Dockerfile
2. start container based on created image with git repository mounted to /app and persistent volume for pytest-cache mounted to /pytest_cache

Within the container all pytest magic happens and all scripts matching test_*.py within tests/ are executed.

Kudos

• Badge Generator for providing easy method to generate badges/shields