A simple null file http and https server originally written using Go 1.5.
Go has evolved considerably since 1.5 but I've tried to keep
the code as close to this version. I also haven't used any third-party
packages to minimize dependency build issues. If your platform has
Go then nullserv will likely run on it.
Because you're running a DNS ad blocker and you want a server that returns minimal valid files for each based on file suffixes in the URL.
nullserv reduces web page layout problems when blocking ads. It also
fixes layout and "good" ad detection for moble apps because the responses
from nullserv are valid images.
The modern web as moved on since I started this project with almost all
websites using encrypted https traffic for ads and trackers. The good
news is nullserv also listens on https port 443 and aborts all SSL/TLS
connections early. It claims the client's SSL certificate of authority (CA)
is invalid. This side-steps the requirement for proxies or self-generated
certificate authorities needed to run a real https server.
Pull the repo, install Google Go and run
make. I use make instead of go build to dynamically
generate buildinfo.go as well as file2gobyte.
If you prefer to manually build nullserv run:
$ cd /path/to/cloned/repo/nullserv
$ cc -Wall -O3 -o file2gobyte file2gobyte.c # OPTIONAL
$ sh mkbuildinfo.sh
$ go build -o nullserv *.go
The optional file2gobyte utility emits Go's []byte{...} array syntax
on stdout similar to xxd -i filename for C. It's not necessary to
run nullserv and only used to assist adding file data to files.go.
There's also a Dockerfile to build and run nullserv inside a Docker
instance.
The emitted nullserv binary is self-contained and relies on no external
files (except maybe a JSON config file). You may run it from within the
repo or copy it to somewere common like /usr/local/bin.
It's recommended to run daemons as a non-root user whenever possible.
Unfortunately nullserv bind to TCP ports 80 (http) and 443 (https)
by default. Most Unix operating systems forbid anyone but root to bind to
privileged ports lower than 1024. The Go language also has
problems with Setuid/Setgid
across all threads which prevents handling the issue inside nullserv.
The simplest way to run nullserv as non-root is to temporarily disable
privileged ports checking at the OS level, run nullserv as a daemon,
and re-enable privileged port checking. Here's a Linux eample:
# (as user root or via sudo)
# Disable unprivileged port binding on 80 and 443 to allow nullserv
# to run as user nobody. After the daemon starts re-enable on < 1024.
# Even though this is in the net.ipv4 namespace it also allows binding
# of IPv6 ports.
sysctl -q net.ipv4.ip_unprivileged_port_start=80
# start-stop-daemon if found in any Debian, or openrc, based distro.
# Other distros have comperable dameonizing tools. Red Hat based
# distros can use a combination of 'nohup' and 'runner'.
start-stop-daemon --start --background \
--exec "/path/to/binary/nullserv" \
--stdout "$LOG/nullserv.log" --stderr "$LOG/nullserv.log" \
--user nobody
# re-enable low port checking
sysctl -q net.ipv4.ip_unprivileged_port_start=1024
Usage of ./nullserv:
-A string
https address (default '' = all)
-P int
https port (default 443)
-a string
http address (default '' = all)
-c string
JSON config file
-m int
content cache age in secs (default 31536000)
-p int
http port (default 80)
-v int
verbose level
nullserv supports JSON configuration files as an alternative to command
line arguments. See the example_confs/ subdirectory for a few use-cases.
The contents of the file takes precedence over command line arguments and
any missing parameters use nullserv defaults.
For example if you only wish to change the max_age parameter to use
no-store and never have clients cache responses:
$ cat max_age.conf
{ "max_age" : -1 }
$ nullserv -c max_age.conf
The ConfFile struct in conf.go lists all the valid JSON parameters
names as does the JSON file example_confs/all.json.
nullserv interprets certain file suffixes as requests for internal state.
Any URL ending with .ver or .version will return version information in
JSON:
$ curl http://127.0.0.1/.ver
{
"build_date": "Wed, 10 Jun 2020 09:44:58 -0500",
"commit_date": "Sat, 30 May 2020 17:33:56 -0500",
"reset_date": "Wed, 10 Jun 2020 12:45:34 -0500",
"sha": "56596c6",
"version": "1.3.0"
}
Any URL ending with .stat or .stats will return statistics:
$ curl http://127.0.0.1/.stat
{
"http_1.1": 6,
"https_tls_1.0": 9,
"stats_ok": 1,
"stats_version": 3,
"suffix_" : 1,
"suffix_stats": 3,
"suffix_version": 2
}
Keys starting with http_ and https_ track the client protocol requests
counts. In the example above we've seen 6 http 1.1 and 9 https 1.0
connections.
The stats_ keys are meta-data. The stats_ok value returns 1 if
the .stats structure is valid, correct data (0 otherwise). The
stats_version is a monotonically increasing version number which will
be incremented whenever the layout of this JSON response is altered.
The suffix_* keys show counts for each of the file suffixes encountered
since the last .reset event. The above data has 1 request for a URL
with no file suffix, 3 for file suffix .stats, and 2 for file
suffix .version.
Any URL ending with .res or .reset will reset all internal statistics.
The response is identical to .stat after resetting all counts. This also
updates reset_date in the .ver output to the current time on the host
running nullserv.
$ curl http://127.0.0.1/.res
{
"stats_ok": 1,
"stats_version": 3
}
The idea for nullserv came from another GitHub project,
pixelserv. I wanted a real project
to learn how to use Google Go. The running binary on a 64-bit x86 host
consumes about 8MB of real memory (mostly Go libraries). It's not as
small as pixelserv is but I think it's a lot easier to extend and
maintain. Go's low-level http library and go routines make scalability
a breeze.