You want to serve WebP images from NGINX automatically. A visitor’s browser supports WebP, so you want NGINX to convert and serve the smaller format on-the-fly. You search for an NGINX module that does this, find ngx_webp, drop webp; into your location block, and it works. Images shrink by 25-60%. Problem solved.
Then you deploy it to production. Under real traffic, your NGINX workers freeze. Requests pile up. Image requests start returning 500 errors. Your monitoring lights up. You revert the change, and everything recovers.
What happened? The module works, but it violates nearly every principle of NGINX module development. It blocks workers, skips caching, crashes instead of falling back, corrupts headers, and hardcodes everything. It is a textbook example of how not to write an NGINX module.
Let’s dissect what went wrong, why each pattern is dangerous, and what a properly written NGINX module should look like instead.
Anti-Pattern 1: fork() and wait() in a Worker Process
This is the showstopper. Here is the actual code from ngx_http_webp_module.c:
child_pid = fork();
switch( child_pid )
{
case 0:
execlp( "/etc/nginx/webp", "/etc/nginx/webp",
lpath.data, dpath.data, NULL );
default:
break;
}
wait( &status );
waitpid(child_pid, &status, WEXITED);
The module forks a child process, executes an external script, and then blocks the entire worker process with wait() until the conversion finishes. This is the opposite of how NGINX works.
Why This Is Catastrophic
NGINX uses a small, fixed number of worker processes (typically one per CPU core). Each worker handles thousands of connections using an event loop. When a worker calls wait(), it stops processing every other connection assigned to it. No new requests are accepted. No responses are sent. The worker is frozen until cwebp finishes converting the image.
A single large JPEG conversion can take 50-200ms. During that time, a worker handling 2,000 connections has effectively gone offline. With 4 workers, just 4 simultaneous conversion requests stall the entire server.
This is not a minor performance issue. It is a fundamental architecture violation that makes any NGINX module unusable under real traffic.
What Should Happen Instead
NGINX provides thread pools (ngx_thread_task_t) for exactly this kind of CPU-bound work. A properly written NGINX module would:
- Post the conversion task to a thread pool
- Return control to the event loop immediately
- Resume the request when the thread completes
This is how ngx_http_image_filter_module (NGINX’s built-in image processing module) handles expensive operations without blocking workers. Alternatively, a sidecar approach checks for a pre-converted .webp file and triggers background conversion only when needed, serving the original image in the meantime.
Anti-Pattern 2: No Caching, Convert Every Time
The module converts the image on every single request. Look at the code flow: it forks and runs cwebp before checking whether a .webp file already exists. The converted file is saved to disk (photo.jpg.webp), but the module never checks for it on the next request. It just converts again.
The module’s own README acknowledges this problem:
As webp convertion takes some CPU usage I recommend to use some kind of caching of nginx responses, like Varnish.
When your NGINX module’s documentation suggests putting an entire reverse proxy cache in front of it to compensate for the module’s own design, something has gone wrong.
What Should Happen Instead
The very first thing the handler should do is check for the sidecar file:
/* Pseudocode for proper caching */
if (webp_sidecar_exists(r, &webp_path)) {
return serve_file(r, &webp_path, "image/webp");
}
/* Only convert if sidecar doesn't exist */
This is trivial to implement. Check if photo.jpg.webp exists with ngx_open_cached_file(). If it does, serve it directly. If not, convert (ideally in a thread pool) and then serve. The second and all subsequent requests would be pure file serves with zero conversion overhead.
Anti-Pattern 3: 500 Error Instead of Graceful Fallback
When cwebp fails for any reason – corrupted image, disk full, binary missing, permissions error – the module returns HTTP 500:
if ( status != 0 ){
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}
The original image is sitting right there on disk, perfectly servable. But instead of falling back to it, the module throws a server error. This means a single broken image (or a missing cwebp binary after an OS update) turns every image request into a 500 error for WebP-capable browsers.
What Should Happen Instead
Always fall back to the original image:
if (status != 0) {
/* Conversion failed - serve original image */
return ngx_http_serve_original(r, &original_path);
}
This is a basic resilience principle. The WebP version is an optimization, not a requirement. If the optimization fails, serve the original. The visitor sees the same image either way. Never let a “nice to have” feature break the core functionality. This applies to any NGINX module that transforms content.
Anti-Pattern 4: Header Corruption Bug
The module sets the Content-Type header in an unusual way:
ngx_table_elt_t *h;
h = ngx_list_push(&r->headers_out.headers);
h->hash = 1;
ngx_str_set(&h->key, "Content-Type");
ngx_str_set(&h->value, "image/webp");
r->headers_out.content_encoding = h;
Notice the last line: r->headers_out.content_encoding = h. The content_encoding field in NGINX’s response headers is for Content-Encoding (gzip, deflate, br) – not Content-Type. By assigning a Content-Type header element to the content_encoding field, the module corrupts the response header structure.
This happens to “work” in many cases because NGINX adds the actual Content-Type separately and the misassigned field may get ignored or overwritten downstream. But it is semantically wrong and can produce unpredictable behavior with downstream filters (like gzip) that inspect content_encoding to decide whether to compress the response.
What Should Happen Instead
Set the content type through the proper NGINX API:
r->headers_out.content_type.len = sizeof("image/webp") - 1;
r->headers_out.content_type.data = (u_char *) "image/webp";
r->headers_out.content_type_len = r->headers_out.content_type.len;
This is how every well-written NGINX module sets content types. It’s the same pattern used in ngx_http_image_filter_module and ngx_http_static_module.
Anti-Pattern 5: Everything Is Hardcoded
This NGINX module has exactly one directive (webp;) with no arguments and no configuration options. Everything is baked into the code:
| What | Hardcoded Value | Should Be Configurable |
|---|---|---|
| Conversion script path | /etc/nginx/webp |
webp_script directive |
| Quality setting | 80 (in the script) | webp_quality directive |
| Accept header search limit | 50 characters | Full header parsing |
| Output format | WebP only | WebP + AVIF |
| Cache behavior | None | webp_cache_path directive |
| Fallback behavior | 500 error | webp_fallback on/off |
The quality setting deserves special mention. It lives in an external shell script (/etc/nginx/webp) rather than in the NGINX configuration. This means you cannot set different quality levels for different locations, and changing the quality requires editing a separate file that lives outside of your nginx.conf management.
What Should Happen Instead
A well-designed NGINX module would look something like this in configuration:
# What a proper implementation might look like
location ~ "\.(jpg|jpeg|png)$" {
webp on;
webp_quality 80;
webp_avif on;
webp_avif_quality 60;
webp_fallback on;
}
location /thumbnails/ {
webp on;
webp_quality 60; # Lower quality for thumbnails
}
Each setting would use ngx_command_t with proper merge functions (merge_loc_conf) so values cascade from http to server to location blocks, following NGINX’s standard configuration inheritance.
Anti-Pattern 6: Accept Header Parsing
The module’s content negotiation logic is a single call:
accept = r->headers_in.accept->value.data;
pos = ngx_strnstr(accept, (char *)"webp", 50);
This searches for the literal string webp in the first 50 bytes of the Accept header. There are several problems with this approach:
- It ignores quality factors. A browser might send
image/webp;q=0.5, image/avif;q=0.9, indicating a preference for AVIF. The module ignores this entirely. - The 50-byte limit is arbitrary. While browsers typically send
image/webpearly in image request Accept headers, there is no guarantee. A proxy or CDN might rewrite headers in a way that pushes thewebptoken past position 50. - It does not validate the MIME type. It searches for the substring
webpanywhere, so a hypotheticalapplication/foowebpwould match. - No NULL check. If a request arrives without an Accept header,
r->headers_in.acceptis NULL, and the module dereferences a null pointer – a crash.
What Should Happen Instead
Parse the Accept header properly using NGINX’s built-in string utilities. Check for the complete MIME type image/webp, respect quality factors, and consider format priority (AVIF > WebP > original). A robust NGINX module must also handle the case where no Accept header is present:
/* Pseudocode for proper Accept parsing */
if (r->headers_in.accept == NULL) {
return serve_original(r);
}
format = negotiate_format(r->headers_in.accept);
/* Returns AVIF, WEBP, or ORIGINAL based on
full MIME matching and quality factors */
What a Proper Rewrite Looks Like
If you wanted to build a production-quality image format negotiation NGINX module, here is the architecture:
1. Body filter, not content handler. Instead of replacing the location handler entirely, operate as a body filter. This lets the module compose with proxy_pass, try_files, alias, and other directives. The module intercepts the response after NGINX resolves the file and converts the body in-flight.
2. Sidecar file check first. Before any conversion, check for photo.jpg.webp (or photo.jpg.avif). If it exists and is newer than the original, serve it instantly. This makes the common case (cached conversion) as fast as a regular static file serve.
3. Thread pool for conversion. When a conversion is needed, post it to NGINX’s thread pool via ngx_thread_task_t. The worker immediately returns to its event loop and resumes the request when the thread finishes. Zero blocking.
4. Graceful degradation. If conversion fails, log a warning and serve the original image. If the cache directory is full, serve the original. If cwebp is missing, serve the original. The original image is always the safe fallback.
5. AVIF support. With a proper architecture, adding AVIF is trivial – it is just another output format. Parse the Accept header, pick the best supported format (AVIF > WebP > original), and convert accordingly.
6. Configuration directives. Quality, formats, cache path, enable/disable, and fallback behavior should all be configurable per location, with proper inheritance.
The Takeaway
The ngx_webp module is a useful teaching example because it demonstrates a pattern that’s common in early-stage NGINX module development: taking an approach that works in single-threaded, synchronous programs and dropping it into NGINX’s asynchronous architecture.
The core lessons for NGINX module development:
- Never block a worker process. Use thread pools for CPU-bound work, subrequests for IO-bound work, and event callbacks for everything else.
- Cache aggressively. If the result of an operation can be reused, store it and check for it first.
- Fail gracefully. An optimization module should never be the reason a request fails. Fall back to the unoptimized path.
- Use NGINX’s APIs. Don’t set
content_encodingwhen you meancontent_type. Don’t usefork()when you havengx_thread_task_t. Don’t hardcode paths when you havengx_command_t. - Parse headers properly. HTTP has rules for content negotiation. Respect them.
If you need on-the-fly image format conversion in production, consider NGINX’s built-in image filter module for basic transforms, or the Small Light module for more advanced dynamic image processing. For WebP serving specifically, the pre-generated sidecar approach using map and try_files is simple, efficient, and fully production-ready.