We maintain a number of open source nginx modules for our package repository at deb.myguard.nl. When we pulled in the zstd-nginx-module — a module that compresses HTTP responses with Zstandard — we did what we always do before shipping anything: we read the code. All of it. What we found was a collection of bugs ranging from “this will silently give you the wrong data” to “this will crash your server under the right conditions.” We fixed them. Our fork lives at github.com/eilandert/zstd-nginx-module.
The Buffer Overflow That Ate Your Path
The static module serves pre-compressed .zst files. To find them, it takes the request URI and appends .zst to the path. The original code reserved sizeof(".zst") - 1 bytes — that is, 3 bytes — and then wrote five bytes into them: the four characters of .zst plus a null terminator. Stack memory next to the path buffer got silently overwritten on every request for a static file. Depending on what lives next to that buffer in the nginx worker’s stack frame, this could corrupt request state, crash the worker, or in adversarial scenarios do considerably worse. We bumped the reservation to sizeof(".zst") and restored the missing t character that was also dropped.
Silent Data Truncation at Exactly 128 KB
This one had us staring at test output for a while. Compress a single-buffer response bigger than 131,072 bytes (libzstd’s internal stream buffer size), and the client gets exactly 131,072 bytes back. No error. No log message. Just a quietly truncated response.
The culprit was the compression state machine. When ZSTD_compressStream() returns a positive hint value — meaning “I still have data pending” — the code transitions to FLUSH mode. Once ZSTD_flushStream() drains, it unconditionally marked the output as last_buf=1 and set done=1, even when the input buffer still had unconsumed bytes. The downstream filter saw last_buf, declared the response finished, and stopped reading. ZSTD_endStream() was never called on the remaining input. The zstd frame finalized prematurely and took your CSS or JavaScript with it. A 141 KB file came out as 128 KB, every time, reproducibly, silently. We gated the END state transition on the input buffer actually being drained and the chain being empty.
The Header Parser That Matched Three Letters Instead of Four
HTTP content negotiation relies on the Accept-Encoding header. The filter module was searching for the string zstd using a length of sizeof("zstd") - 2. That evaluates to 3. So it was matching zst — anything starting with those three letters would trigger zstd compression on a client that never asked for it. The static module had the correct value. The filter module had the off-by-one. Fixed to sizeof("zstd") - 1.
The Variable That Could Overflow Your Memory Pool
nginx exposes a $zstd_ratio variable showing the compression ratio. The code allocated 13 bytes for it (NGX_INT32_LEN + 3) and then used ngx_sprintf to write two ngx_uint_t values. On 64-bit systems, ngx_uint_t is a 64-bit integer — up to 20 decimal digits each. With two of those plus a decimal point and null terminator, you need 44 bytes. Writing 44 bytes into 13 bytes of pool memory overwrites whatever nginx allocated immediately after. We changed the allocation to NGX_INT_T_LEN * 2 + 2. While in the same function: the ratio fraction did bytes_in * 1000 in 32-bit arithmetic. Files over 4 GB produced nonsense ratio values. We promoted to uint64_t before the multiply.
The Dictionary That Didn’t Load When the Parent Was Disabled
The module supports Zstandard compression dictionaries, which improve compression ratios on repetitive content. When merging location configuration, the code had a shortcut: if parent and child locations use the same compression level, reuse the parent’s dictionary object. Except when the parent had compression disabled, its dictionary pointer was NULL. The shortcut silently handed a NULL dict to the child, which then advertised dictionary-compressed responses without actually using any dictionary. We added a prev->dict != NULL guard before taking the shortcut.
Calling Free on a Pointer We Never Initialized
On compression failure, the code jumped to a failed: label and called ZSTD_freeCStream(ctx->cstream). If the failure was that cstream was never created — one of the first things that can go wrong — it was calling free on a NULL pointer. libzstd handles NULL gracefully but that’s an undocumented dependency. We added an explicit NULL guard and set cstream = NULL after freeing to prevent any double-free possibility.
Content-Type Was Looking at the Wrong Thing
The static module temporarily shortened path.len to hide the .zst suffix before calling ngx_http_set_content_type(). Clever idea, except that function doesn’t look at path at all — it uses r->exten, derived from the original request URI. Since the browser requests /file.html, r->exten is already html regardless of what you do to the path. The manipulation was a complete no-op that looked intentional. We removed it.
A Compiler Flag That Infected Everything Else
The static module’s build script used CFLAGS="$ngx_zstd_opt_I $CFLAGS" — a global assignment. In the static-linking path, ngx_zstd_opt_I contains -DZSTD_STATIC_LINKING_ONLY. This gets injected into the compilation of every other nginx module built in the same configure run. ModSecurity, GeoIP2, PageSpeed — all of them get a define they know nothing about. The filter module was already doing this correctly via ngx_module_incs. We removed the rogue global assignment.
The Linker Flag That Broke Everything Except Linux
When the module couldn’t find libzstd dynamically, it fell back to -l:libzstd.a — GNU ld syntax for “link this exact filename.” LLVM lld (FreeBSD, OpenBSD, RHEL 9+) doesn’t recognize it. macOS ld64 doesn’t either. There was also a stray space in the rpath flag: -Wl,-rpath, $ZSTD_LIB. That space splits one argument into two, breaking rpath on every linker on every platform. We replaced the fallback with pkgconf/pkg-config detection — portable across Debian, RHEL, Gentoo, FreeBSD, and OpenBSD — and fixed the rpath space.
An API Called Without Checking the Version
ZSTD_minCLevel() was introduced in zstd 1.4.0. The code called it unconditionally. RHEL 7 shipped zstd 1.3.3. Older FreeBSD ports can carry even older versions. On any of those systems, compilation fails at link time with an undefined symbol. We wrapped the call in #if ZSTD_VERSION_NUMBER >= 10400 with a safe fallback for older libraries.
Two Identical Functions, One in Each Module
Both the filter module and the static module contained byte-for-byte identical implementations of ngx_http_zstd_accept_encoding() and ngx_http_zstd_ok(). The Accept-Encoding parser is 80 lines of RFC 7231 quality-value parsing. Having it twice means any future fix to one copy has a good chance of not making it to the other. We extracted both into a shared header included by both modules.
The Quality-Value Parser That Accepted Anything
The Accept-Encoding quality-value parser handles requests like zstd;q=0.5. RFC 7231 §5.3.1 defines the grammar precisely: the only valid leading digits are 0 and 1, a decimal point must be followed by at least one digit, and q=1.x is only valid when every fractional digit is zero. Our parser was accepting all of these as valid:
q=999— out of range, treated as “client wants zstd”q=0.— trailing dot with no digits, treated as “client wants zstd”q=0X— non-dot character after zero, treated as “client wants zstd”q=1.5— invalid per RFC (q cannot exceed 1.0), treated as “client wants zstd”
The consequence of accepting q=999 or q=0. is that a client sending a malformed header gets compressed responses whether or not it can actually decode them. In practice browsers send well-formed headers, but load balancers and proxies sometimes forward unusual values. The parser now strictly enforces the RFC 7231 grammar, rejecting anything outside [0, 1] or malformed.
Default Compression Level Was Too Conservative
The default compression level was 1 — the fastest setting, which trades ratio for speed. The zstd library’s own default is level 3, which achieves meaningfully better compression with comparable throughput on typical web content. We changed the default to 3. Level 1 is still available; set zstd_comp_level 1; if you need it.
gzip_vary Was Required But Not Documented or Enforced
When the filter module compresses a response, it sets r->gzip_vary = 1. This is nginx’s internal signal to emit a Vary: Accept-Encoding response header — but only if gzip_vary on is set in config. Without it, the header is never sent. Without the header, any proxy or CDN in front of your server will cache the first response it sees (compressed or not) and serve it to all subsequent clients regardless of what they support. A client that doesn’t speak zstd gets a compressed body it cannot decode, and your CDN serves it confidently with a 200.
The same applies to zstd_static on. The module correctly sets r->gzip_vary in that path too, but again only takes effect when gzip_vary on is configured. Neither the directive documentation nor the Synopsis example mentioned this. Both do now.
ETag Weakening Was Undocumented
When nginx compresses a response, it automatically converts any strong ETag to a weak one: "abc123" becomes W/"abc123". This is correct per RFC 7232 — a compressed variant is a different representation of the resource and cannot share a strong ETag with the uncompressed original. But it has a practical consequence: If-Match validation will fail when a client received the uncompressed version and the server is now returning compressed. CDN edges that cache both variants will see two different ETags for the same URL. None of this was mentioned anywhere. It is now.
The Docs Told You to Do the Wrong Thing
The README said to prefer static linking (libzstd.a) over dynamic linking, on the grounds that the streaming API might change across shared library versions. This advice was backwards. Dynamic nginx modules are .so files, and .so files can only link against position-independent code. A standard system libzstd.a is not compiled with -fPIC and cannot be linked into a shared object — the linker will error out. The build scripts have always auto-detected and preferred dynamic linking for exactly this reason. We removed the incorrect guidance and replaced it with the accurate one: install libzstd-dev (Debian/Ubuntu) or libzstd-devel (RHEL/Fedora) and let the build scripts do the right thing.
A NULL Dereference Waiting in the Output Buffer Path
The output buffer allocation function ngx_http_zstd_filter_get_buf() sets ctx->out_buf from either the free-list or ngx_create_temp_buf(), then immediately dereferences it to set up the ZSTD output buffer. If out_buf is ever NULL at that point — whether from a recycled-buffer state oddity or a future code path — the worker crashes with a NULL dereference. The allocation paths make this unlikely today, but the code relied on that invariant silently. We added an explicit NULL check with an ALERT log before the dereference so the invariant is enforced rather than assumed.
Only Three HTTP Status Codes Got Compression
The filter module only compressed responses with status 200, 403, and 404. Every other status code — 201 Created, 202 Accepted, 206 Partial Content, and so on — was passed through uncompressed, even when carrying large compressible bodies. A REST API returning 201 with a substantial JSON payload got nothing. We expanded the check to all 2xx responses, keeping 403 and 404 for their compressible error pages.
Missing gzip_vary Was Silent Until Your CDN Broke
We documented the requirement and made the code enforce it. If you start nginx with zstd on but without gzip_vary on, the module now emits a [warn] message at startup: “zstd is enabled but gzip_vary is off; add gzip_vary on to emit Vary: Accept-Encoding so proxies and CDNs cache compressed and uncompressed responses separately.” Previously you would only discover the problem after your CDN had cached the wrong variant for a few thousand users.
zstd_max_length Doesn’t Apply to Chunked Responses
zstd_max_length only fires when the response includes a Content-Length header. Streaming or chunked responses without one are always compressed, regardless of final size. A large video stream or a multi-gigabyte S3 proxy response will be fully compressed no matter what you set zstd_max_length to. This is not a bug — it is how nginx gzip works too — but it is a CPU exposure vector on locations that proxy large unbounded streams. We added an explicit warning in the documentation.
No Warning When Building Against an Old zstd
The module uses version guards to fall back to deprecated APIs on older zstd installs — ZSTD_CCtx_reset() requires 1.5.0, ZSTD_minCLevel() requires 1.4.0. The code handles both cases correctly. But an operator building against RHEL 7’s zstd 1.3.3 would get no feedback whatsoever: the build succeeds, the module loads, and it quietly uses the deprecated fallback paths. The configure script now extracts the version number via the C preprocessor and emits an advisory if you are below 1.4.0 (missing ZSTD_minCLevel()) or below 1.5.0 (falling back to deprecated stream init API).
RPATH Baked In at Build Time
When ZSTD_LIB is set to a custom path, the build passes -Wl,-rpath,<path> to the linker. That path is baked into the module .so at build time. If the library is later moved — say, a package manager upgrade relocates libzstd.so.1 — the module fails to load at nginx startup with no useful error until someone looks at ldd output. We documented this in the README so operators using custom builds know what they are signing up for and understand why using the system package and leaving ZSTD_LIB unset avoids the problem entirely.
The CI Pipeline We Built to Keep It That Way
After finding and fixing twenty-plus bugs across multiple review passes, we wanted some assurance that the module stays fixed. We set up a four-job GitHub Actions pipeline on the fork:
- Validation — shellcheck on every build script, cppcheck static analysis on the C code, and a clang static analyzer pass.
- Hello World — builds nginx from source against the module with
-Wall -Wextra -Wshadow -Werror. A warning is a build failure. The resulting nginx binary is uploaded as an artifact for the test job to consume. - Tests — downloads the nginx binary from the previous job, installs the Test::Nginx::Socket Perl test framework, and runs 41 tests: 23 covering the filter module and 18 covering the static module. The tests were originally written by HanadaLee for their own zstd-nginx-module fork; we ported them, extended them to cover the edge cases we fixed, and added RFC quality-value tests that were missing entirely.
- Secure — flawfinder vulnerability scanner, clang-tidy with cert-*, bugprone-*, and clang-analyzer-security.* checks enabled, and a semgrep scan.
Every pull request and every push to master runs all four jobs. The Hello World job exists specifically because static analyzers miss things that a strict compiler catches, and a compiler with -Werror misses things that a real request through a running nginx catches. You need all three layers.
Why Bother
We run nginx in production with this module. The truncation bug alone was enough to review everything else. A JavaScript file arriving 10 KB short doesn’t produce an obvious error — the browser decompresses what it got, executes a partial script, and the result is a silently malfunctioning page. Users see broken behavior; logs show a clean 200.
All fixes are in our public fork: github.com/eilandert/zstd-nginx-module. Pre-built Debian packages are available through our repository at deb.myguard.nl.