yum upgrades for production use, this is the repository for you.
Active subscription is required.
Technical Briefing: Maintained zstd Module for NGINX — Streaming Fixes, Packaging, and Audit Response
Problem Statement
The original tokers/zstd-nginx-module (by Alex Zhang) has not seen a release since 2023 and carries bugs in its streaming path that silently truncate large responses. Over the buffer size, the filter could close the zstd frame over unconsumed input, delivering a valid but short file with no error and no log line — a failure mode that passes header assertions and is invisible without byte-level verification.
GetPageSpeed/zstd-nginx-module is the maintained continuation under the same BSD-2-Clause licence, with the original copyright intact.
Streaming State Machine Fixes
Three defects in the streaming path are addressed:
- Premature end-of-stream transition. The filter previously switched to zstd’s end-of-stream directive as soon as NGINX handed it the last buffer, without checking that the held input had been consumed. The transition is now gated on input actually being drained and the chain being empty.
last_bufstamped too early. The final buffer was markedlast_bufwhenever NGINX delivered its last input, not when zstd had written the frame epilogue. A flush at the wrong moment ended the response over an unterminated frame. Completion is now tied to the end directive returning zero.proxy_buffering offpath. A buffer carrying bothflushandlast_bufwas treated as a flush and discarded the last data; a zero-length buffer returned early, so the zero-length chunked-response terminator was ignored. Both fixed.
Other Corrections
- The deprecated
ZSTD_compressStream()/ZSTD_flushStream()/ZSTD_endStream()trio is replaced by a singleZSTD_compressStream2()call with an explicit end directive. $zstd_rationo longer overflows on large responses.- Dynamic compression is eligible across 2xx and 3xx, excluding bodyless 204 and 205 and already-partial 206.
- Earlier rounds replaced the
Accept-Encodingmatcher with an RFC 9110 tokenizer (sozstdmatches as a whole token andzstd;q=0is honoured as refusal), gave every request its own compression context, and added a pool cleanup handler so config reloads stop leaking the compression dictionary.
Vary Handling
Both modules now emit Vary: Accept-Encoding themselves, parsing any existing Vary first so they neither duplicate the token nor stomp a *. This previously depended on gzip_vary, which was a cache-poisoning risk for operators who never enabled it.
Packaging
Ships as nginx-module-zstd, a dynamic module built against your exact NGINX. The package version joins NGINX and module versions (e.g. 1.30.4+0.2.2), and the package Requires that precise NGINX build so the dependency solver prevents ABI mismatch. Every tracked NGINX release triggers rebuilds in lockstep.
- Platforms: RHEL, Rocky, AlmaLinux, CentOS, Fedora, Amazon Linux, Debian and Ubuntu.
- RHEL 7: ships no zstd in its base OS and only EPEL’s libzstd 1.5.5; zstd 1.5.7 is built and published, versioned to sort above EPEL’s.
- Linking: dynamic builds prefer shared libzstd; static NGINX builds prefer the archive; a custom
ZSTD_LIBpath is recorded as an RPATH. - Two shared objects:
ngx_http_zstd_filter_module.sofor on-the-fly compression andngx_http_zstd_static_module.sofor serving precompressed.zstfiles.
Testing
Every push runs the Test::Nginx::Socket harness twice (once with AddressSanitizer linked in), decompresses output with the zstd binary, and compares SHA-1 against the original. A truncation bug returning a valid short file passes header assertions but fails this check. The suite also exercises zstd_buffers against large output_buffers, flush ordering, bodyless status codes and range requests. The stated rule: a fix without a test that fails before it is a hypothesis, not a fix.
Response to a Competing Fork’s “36 Bugs” Audit
All 36 claims were read against the actual code:
- Real, adopted: the streaming defects above, plus two catches adopted from the audit — the cross-compilation feature probe and the brotli filter-ordering fix.
- Not applicable to this codebase: the
.zstpath overflow doesn’t exist (extension length is reserved); theAccept-Encodingsubstring/qvalue bugs, dictionary reload leak, worker-wide shared compression context,SAVED_CC_TAST_FLAGStypo and non-PIC static archive were already fixed or never present. - Not defects:
zstd_max_length,zstd_window_log,zstd_longandzstd_bypassare new directives in that fork; changing the default compression level from 1 to 3 is a preference; one entry is only described as “a cluster of eight smaller correctness fixes.” - Mutually cancelling entries: the list claims both “only 200/403/404 were compressed” and “204 and 205 were compressed.” The original header filter only admitted OK, FORBIDDEN and NOT_FOUND, so a 204 was never eligible; the second entry describes a bug the first entry’s fix introduces — one net change counted twice.
Two Recommendations Rejected
- Clearing
Accept-Rangeson precompressed files is wrong. RFC 9110 says a range request addresses the selected representation, which for a negotiated zstd response is the.zstfile on disk, whose bytes are stable and rangeable; clearing the header breaks resumable downloads. This fork went the other way and tests that a range request against a.zstvariant returns itsContent-Range. - Warning at config load when
gzip_varyis off is wrong. The right answer is for the module to emitVaryitself, not to tell operators to enable a gzip directive to fix a zstd response.
Installation
RHEL / Rocky / AlmaLinux / CentOS / Fedora / Amazon Linux:
sudo dnf install https://extras.getpagespeed.com/release-latest.rpm
sudo dnf install nginx-module-zstd
Debian / Ubuntu: set up the APT repository, then:
sudo apt-get update
sudo apt-get install nginx-module-zstd
Load in nginx.conf:
load_module modules/ngx_http_zstd_filter_module.so;
load_module modules/ngx_http_zstd_static_module.so;
Then enable:
zstd on;
zstd_comp_level 3;
zstd_min_length 256;
zstd_types text/plain text/css application/json application/javascript text/xml application/xml image/svg+xml;
Repository metadata is public; downloading packages requires an active subscription.
Read the full article: zstd-nginx-module: Maintained, Tested and Packaged
