Brotli Compression for Varnish 6.0 LTS: vmod-brotli Now Packaged

Open-source Varnish compresses exactly one way: gzip. Brotli, the compression that every modern browser advertises and that NGINX users have enjoyed for years, simply does not exist in the open-source Varnish tree. Varnish Enterprise ships brotli as a paid feature. For open-source Varnish, Uplex published libvfp-brotli, a brotli fetch processor, but it targets Varnish 6.2 and newer, and the 6.0 LTS line that Magento 2 and other stacks standardize on was left out.

GetPageSpeed has now backported libvfp-brotli to Varnish 6.0 LTS and packaged it as vmod-brotli in our getpagespeed-extras-varnish60 repository channel, the same channel that carries vmod-pesi for parallel ESI. Install one package, add a few lines of VCL, and Varnish caches brotli-compressed objects and serves them to brotli-capable clients.

What it saves, measured

A fetch processor compresses the object once, on the way into the cache; every later hit serves the already-compressed bytes. So the number that matters is the size on the wire. We verified on a clean Rocky Linux 10 machine, installed exactly with the commands below, against a real asset (jquery-3.7.1.js, 285,314 bytes uncompressed), decompressing the delivered body and confirming it byte-identical to the original:

Compression cost is paid once per object, on the cache miss. On our (deliberately modest, virtualized ARM) test machine the cold-miss response time for that 285 KB file was about 18 ms with gzip and about 333 ms with brotli at the default quality 11; at quality 5 it was about 18 ms, indistinguishable from gzip. Repeated identical runs put the measurement noise floor at roughly 2 ms, so the quality-11 cost is real and the quality-5 parity is too. The practical reading: default quality 11 is the right choice for cacheable static assets, where a third of a second once per cached object buys 17% less bandwidth on every hit; for short-TTL or frequently-missing objects, create a quality-5 encoder and get gzip-speed compression that still beats gzip on size.

Every hit after that is served at full speed either way: the object is stored compressed, and Varnish streams stored bytes.

Installation

The package is available for Enterprise Linux 7 through 10 (RHEL, Rocky, Alma, CentOS Stream, Oracle), Amazon Linux 2 and 2023, Fedora 43 and 44, and SLES 16, on both x86_64 and aarch64. It is built against varnish-6.0.18 exactly and pins it, so the module and the daemon can never drift apart.

On RHEL 8 and later derivatives:

sudo dnf -y install https://extras.getpagespeed.com/release-latest.rpm dnf-plugins-core
sudo dnf config-manager --enable getpagespeed-extras-varnish60
sudo dnf -y install varnish vmod-brotli

On EL7:

sudo yum -y install https://extras.getpagespeed.com/release-latest.rpm yum-utils
sudo yum-config-manager --enable getpagespeed-extras-varnish60
sudo yum -y install varnish vmod-brotli

Using the repository in production requires a GetPageSpeed subscription, which also covers 100+ NGINX modules and the rest of the extras collection.

VCL that actually negotiates

One thing has to be understood first, and it applies to upstream libvfp-brotli on any Varnish version: this is a fetch processor. Objects are cached compressed and delivered exactly as stored; there is no decompress-on-delivery for clients that did not ask for brotli. Serving brotli to everyone would break curl, monitoring agents, and older clients, so the VCL has to keep separate cache variants per client capability.

There is a trap on the way there: Varnish normalizes the Accept-Encoding request header to just gzip (or removes it) right after vcl_recv, before cache lookup, so the string br never survives into Vary matching or backend-side conditions. The clean 6.0 pattern is to stash brotli capability in a custom header during vcl_recv and split the cache on it:

vcl 4.0;

import brotli;

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    unset req.http.X-Brotli;
    if (req.http.Accept-Encoding ~ "\bbr\b") {
        set req.http.X-Brotli = "br";
    }
}

sub vcl_hash {
    if (req.http.X-Brotli) {
        hash_data(req.http.X-Brotli);
    }
}

sub vcl_backend_response {
    if (beresp.http.Content-Type ~ "^(text/|application/(javascript|json|xml))") {
        if (bereq.http.X-Brotli == "br") {
            brotli.compress();
        } else {
            set beresp.do_gzip = true;
        }
    }
}

With this VCL, brotli-capable clients get brotli objects, everyone else gets the stock gzip path including Varnish’s usual gunzip-on-delivery for clients with no compression support at all. This exact configuration is pinned by a regression test in our build, verified against Varnish 6.0.18 on every distro and architecture we ship.

A few behaviors to know, all inherited from upstream and re-verified on 6.0:

• If the backend response is already gzipped, brotli.compress() transparently decompresses it first and then applies brotli, so dynamic backends that always gzip work out of the box.
• The module sets Content-Encoding: br, adds Accept-Encoding to Vary, drops the stale Content-Length, and weakens any ETag (the body is no longer byte-identical to the origin’s).
• Brotli compression cannot be combined with beresp.do_esi: an ESI-processed page cannot be assembled from a brotli body on delivery, so that combination fails the VCL loudly instead of corrupting responses. The reverse direction works: brotli.decompress() on a brotli-sending backend combines fine with ESI, decompressing before the ESI parse. And if ESI with parallel delivery is what you are after, that is vmod-pesi’s job.
• Partial (206) backend responses are ignored, same as Varnish’s own gzip handling.

Custom quality and decompression

The default brotli.compress() uses the library defaults (quality 11, window 22). For the gzip-speed variant from the table above, create an encoder object in vcl_init:

sub vcl_init {
    new br_fast = brotli.encoder("br_fast", quality=5);
}

sub vcl_backend_response {
    br_fast.activate();
}

The symmetric brotli.decompress() (and brotli.decoder objects) decompress brotli-sending backends on fetch, for caching uncompressed or for ESI processing. Counters for every filter appear in varnishstat under BROTLI.*: operations, input bytes, and output bytes, which makes the achieved compression ratio directly observable in production.

About the backport, honestly

vmod-brotli is a GetPageSpeed backport of Uplex’s open-source libvfp-brotli (BSD-2-Clause). Upstream targets Varnish 6.2 and newer and never supported 6.0; this build is not supported or endorsed by Uplex, and issues with it belong with us, not them. Upstream publishes no release tags, so the package is based on the pristine upstream 6.2-compatible branch, with provenance preserved, under our own 0.1.0 version line.

The interesting part of the port: upstream activates its filters through beresp.filters, and Varnish 6.0 has no beresp.filters and no filter registry at all. Our build activates the fetch processor directly on the backend request’s fetch pipeline at vcl_backend_response time, through the explicit calls shown above. The ordering against Varnish’s own gzip machinery is the delicate bit: stock Varnish decides its gzip/gunzip/ESI processors after your VCL returns, so the backport leans on a property of 6.0’s pipeline that we verified in source and pinned with tests: every stock gzip-family processor re-checks Content-Encoding when the pipeline opens and disables itself when it sees br. Two GetPageSpeed regression tests guard exactly this: one proves that activating brotli for one response leaves every other response’s stock behavior untouched (gzip delivery, integrity checking, Range requests), the other proves byte-level correctness of the gzip-to-brotli conversion by round-tripping it through a second Varnish instance.

The complete upstream varnishtest suite, adapted only in its activation syntax, passes against Varnish 6.0.18 on every one of the 18 distro/architecture combinations we ship, with zero tests skipped.

Try it

sudo dnf config-manager --enable getpagespeed-extras-varnish60
sudo dnf -y install vmod-brotli

Add the negotiation VCL above, reload, and watch varnishstat -f 'BROTLI.*' count the bytes you no longer send. Subscribe here if you are not on the repository yet.