NGINX ZeroMQ Log Module: Real-Time Log Streaming Over ZMQ

When your NGINX server handles thousands of requests per second, traditional file-based access logging becomes a bottleneck. Log files pile up on disk, require manual rotation, and offer no way to process entries in real time. If you need to feed log data into a monitoring dashboard, trigger alerts on error spikes, or fan out access data to multiple consumers simultaneously, writing to flat files simply does not scale.

The NGINX ZeroMQ log module solves these problems by streaming log data directly to ZeroMQ subscribers over TCP or IPC — asynchronously, without blocking request handling, and without touching the filesystem.

Why Stream NGINX Logs Over ZeroMQ?

NGINX already supports native syslog: destinations for the access_log directive. However, the NGINX ZeroMQ log module offers capabilities that syslog cannot match:

• Fan-out to multiple subscribers: ZeroMQ’s PUB/SUB pattern delivers every message to all connected subscribers simultaneously. With syslog, you need one access_log line per destination.
• Topic-based filtering: Subscribers receive only messages matching their topic subscription. For example, a monitoring system can subscribe to /errors/ while an analytics pipeline subscribes to /all/.
• Structured message formats: Send JSON-formatted log data directly, avoiding syslog’s line-oriented format and parsing overhead.
• Transport flexibility: Choose TCP for cross-host delivery or IPC (Unix sockets) for high-speed local communication between processes.
• Queue buffering: ZeroMQ maintains an in-memory message queue, absorbing traffic spikes without dropping messages or blocking NGINX workers.

If you are interested in other advanced NGINX logging approaches, check out the NGINX SQLite logging module for queryable access logs stored in a database.

When to Use ZeroMQ Logging vs. Native Syslog

Use native syslog when you have a single, standard syslog destination. Use the NGINX ZeroMQ log module when you need fan-out, topic routing, structured data, or high-throughput local IPC delivery.

How the ZeroMQ Log Module Works

The module operates as a ZeroMQ publisher (PUB socket). On each HTTP request, NGINX evaluates the configured format string — expanding variables like $remote_addr, $status, and $request_uri — and sends the resulting message to all connected ZeroMQ subscribers.

Each message is prefixed with a configurable topic (the endpoint), enabling subscribers to filter messages by category. For example, a message sent to topic /access/ with format {"ip":"1.2.3.4"} arrives at the subscriber as /access/{"ip":"1.2.3.4"}.

The NGINX ZeroMQ log module creates one ZeroMQ context per logger definition, with a configurable number of I/O threads and queue size. All message delivery is asynchronous — if no subscribers are connected, messages are silently discarded. If a subscriber disconnects temporarily, the queue buffers messages up to the configured limit.

This design ensures that ZeroMQ logging never blocks NGINX request processing, even under network failures or slow subscribers. You can also enrich your log data before it reaches ZeroMQ by using the log_var_set module to set custom variables at log time.

Installation

RHEL, CentOS, AlmaLinux, Rocky Linux

Install the GetPageSpeed repository and the module package:

sudo dnf install https://extras.getpagespeed.com/release-latest.rpm
sudo dnf install nginx-module-log-zmq

Then load the module by adding the following line at the top of /etc/nginx/nginx.conf:

load_module modules/ngx_http_log_zmq_module.so;

Debian and Ubuntu

First, set up the GetPageSpeed APT repository, then install:

sudo apt-get update
sudo apt-get install nginx-module-log-zmq

On Debian/Ubuntu, the package handles module loading automatically. No load_module directive is needed.

For more details, see the module pages:

• RPM packages: nginx-extras.getpagespeed.com/modules/log-zmq
• APT packages: apt-nginx-extras.getpagespeed.com/modules/log-zmq

Configuration Directives

The NGINX ZeroMQ log module provides four directives. All logger definitions (log_zmq_server, log_zmq_endpoint, log_zmq_format) must be placed in the http context. The log_zmq_off directive is used in location blocks to selectively disable logging.

log_zmq_server

Defines a named ZeroMQ logger instance with connection details.

Syntax:

log_zmq_server <name> <address> <tcp|ipc> <threads> <queue_size>;

Context: http

Parameters:

• name — a unique identifier for this logger instance. Used by log_zmq_endpoint, log_zmq_format, and log_zmq_off to reference this definition.
• address — the subscriber’s address. For TCP, use ip:port format (e.g., 127.0.0.1:5556). For IPC, use the Unix socket path (e.g., /tmp/nginx-zmq.ipc).
• tcp|ipc — the transport protocol. Use tcp for network delivery, ipc for local Unix socket communication.
• threads — the number of ZeroMQ I/O threads for this context. Use 1 for most workloads. Increase for very high message throughput.
• queue_size — the maximum number of messages to buffer in the send queue. Messages beyond this limit are dropped silently. A value of 1000 is a reasonable default.

Example:

# TCP connection to a remote subscriber
log_zmq_server main 127.0.0.1:5556 tcp 1 1000;

# IPC connection via Unix socket
log_zmq_server local "/tmp/nginx-logs.ipc" ipc 1 1000;

log_zmq_endpoint

Sets the topic prefix for messages sent by a logger instance.

Syntax:

log_zmq_endpoint <name> "<topic>";

Context: http

Parameters:

• name — the logger instance name (must match a log_zmq_server definition).
• topic — a string prepended to every message. Supports NGINX variables for dynamic topics.

The topic enables ZeroMQ subscribers to filter messages using topic subscriptions. A subscriber that subscribes to /errors/ receives only messages whose topic starts with that prefix.

Example:

# Static topic
log_zmq_endpoint main "/access/";

# Dynamic topic based on response status
log_zmq_endpoint main "/status/$status/";

log_zmq_format

Defines the message body format for a logger instance.

Syntax:

log_zmq_format <name> '<format_string>';

Context: http

The format string follows the same rules as the standard NGINX log_format directive. You can use any NGINX variable and split the format across multiple lines.

Example:

# Simple text format
log_zmq_format main '$remote_addr - [$time_local] "$request" $status';

# JSON format (recommended for structured processing)
log_zmq_format main '{"time":"$time_iso8601",'
                     '"ip":"$remote_addr",'
                     '"method":"$request_method",'
                     '"uri":"$uri",'
                     '"status":$status,'
                     '"bytes":$body_bytes_sent,'
                     '"referer":"$http_referer",'
                     '"ua":"$http_user_agent",'
                     '"rt":$request_time}';

log_zmq_off

Disables ZeroMQ logging for a specific location.

Syntax:

log_zmq_off <name>|all;

Context: location

Parameters:

• name — disable a specific logger instance by name.
• all — disable all ZeroMQ logger instances.

Example:

location /health {
    log_zmq_off all;
    return 200 "OK";
}

location /internal-api {
    log_zmq_off main;  # Disable only the "main" logger
}

Practical Use Cases

Real-Time JSON Log Processing

Stream structured JSON logs to a Python consumer for real-time analysis using the NGINX ZeroMQ log module:

NGINX configuration:

load_module modules/ngx_http_log_zmq_module.so;

http {
    log_zmq_server analytics 127.0.0.1:5556 tcp 1 1000;
    log_zmq_endpoint analytics "/logs/";
    log_zmq_format analytics '{"time":"$time_iso8601",'
                              '"ip":"$remote_addr",'
                              '"method":"$request_method",'
                              '"uri":"$uri",'
                              '"status":$status,'
                              '"bytes":$body_bytes_sent,'
                              '"rt":$request_time}';

    server {
        listen 80;
        server_name example.com;

        location / {
            root /usr/share/nginx/html;
        }

        location /health {
            log_zmq_off all;
            return 200 "OK";
        }
    }
}

Python subscriber (the subscriber must bind because NGINX connects to it):

import zmq
import json

context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.bind("tcp://127.0.0.1:5556")
socket.setsockopt_string(zmq.SUBSCRIBE, "/logs/")

print("Listening for NGINX logs...")
while True:
    message = socket.recv_string()
    # Remove topic prefix
    json_data = message[len("/logs/"):]
    log_entry = json.loads(json_data)
    print(f"{log_entry['ip']} {log_entry['method']} {log_entry['uri']} -> {log_entry['status']}")

Important: Start the subscriber before NGINX, or restart NGINX after the subscriber starts. The NGINX module connects to the subscriber’s bound address on its first request. Messages sent before the subscriber is ready are lost (see the “slow joiner” behavior in Troubleshooting).

Multiple Log Destinations

Send different log data to separate subscribers — one for access analytics, another for error tracking:

load_module modules/ngx_http_log_zmq_module.so;

http {
    # Access log stream
    log_zmq_server access_log 127.0.0.1:5556 tcp 1 1000;
    log_zmq_endpoint access_log "/access/";
    log_zmq_format access_log '{"ip":"$remote_addr","uri":"$uri","status":$status}';

    # Error-only stream (uses dynamic topic with status code)
    log_zmq_server error_log 127.0.0.1:5557 tcp 1 1000;
    log_zmq_endpoint error_log "/error/$status/";
    log_zmq_format error_log '{"ip":"$remote_addr","uri":"$uri","status":$status,"ua":"$http_user_agent"}';

    server {
        listen 80;
        server_name example.com;

        location / {
            root /usr/share/nginx/html;
        }
    }
}

A subscriber can then connect to port 5557 and subscribe to /error/5 to receive only 5xx server errors, or /error/4 for client errors.

High-Speed Local IPC Pipeline

For maximum throughput on the same machine, use IPC transport with Unix sockets:

load_module modules/ngx_http_log_zmq_module.so;

http {
    log_zmq_server pipeline "/tmp/nginx-logs.ipc" ipc 2 5000;
    log_zmq_endpoint pipeline "/";
    log_zmq_format pipeline '$remote_addr|$time_iso8601|$request_method|$uri|$status|$body_bytes_sent|$request_time';

    server {
        listen 80;
        server_name example.com;

        location / {
            root /usr/share/nginx/html;
        }
    }
}

IPC transport avoids TCP overhead entirely, making it ideal for architectures where a log processing daemon runs on the same server as NGINX.

Performance Considerations

The NGINX ZeroMQ log module adds minimal overhead to request processing:

1. Asynchronous delivery: Message sending happens in ZeroMQ’s I/O threads, not in NGINX worker threads. The NGINX worker only formats the message and enqueues it.
2. Non-blocking send: The module uses ZMQ_DONTWAIT for message sending, ensuring NGINX workers are never blocked by ZeroMQ I/O operations.
3. Graceful degradation: If the message queue fills up (subscriber too slow or disconnected), excess messages are dropped silently rather than blocking the worker.

Tuning Recommendations

• I/O threads: Start with 1. Increase to 2-4 only if you observe message queue buildup under high throughput.
• Queue size: The default 1000 handles most workloads. For high-traffic sites (10,000+ requests/second), increase to 5000-10000.
• IPC vs TCP: Use IPC when the subscriber runs on the same host. It eliminates TCP overhead and achieves significantly higher throughput.

Troubleshooting

SELinux Blocking ZeroMQ Connections

On RHEL-based systems with SELinux enabled, NGINX may fail to connect to ZeroMQ subscribers. SELinux blocks NGINX from making outbound TCP connections to non-standard ports by default.

Symptoms: nginx -t succeeds, no errors in the error log, but the subscriber receives zero messages.

Diagnosis: Use strace on the NGINX worker process to check for EACCES errors:

strace -p $(pgrep -f "nginx: worker") -e trace=network -f 2>&1 | grep EACCES

Fix: Allow NGINX to connect to the ZeroMQ port:

sudo setsebool -P httpd_can_network_connect 1

Alternatively, for IPC transport, ensure the Unix socket path is accessible by the NGINX worker user.

Module Fails to Load

If you see dlopen() failed when starting NGINX, verify that the ZeroMQ library is installed:

# RHEL/CentOS/Rocky Linux
sudo dnf install zeromq

# Debian/Ubuntu
sudo apt-get install libzmq5

Messages Not Arriving at Subscriber

1. Check SELinux first (see above). This is the most common cause on RHEL-based systems.

2. Verify the subscriber is running and binding. The subscriber must call bind() because the NGINX module uses connect(). This is the reverse of typical PUB/SUB topology.

3. Check the topic subscription. The subscriber must subscribe to a prefix that matches the configured log_zmq_endpoint. An empty string subscription ("") receives all messages.

4. Verify network connectivity:

# Test TCP port
nc -zv 127.0.0.1 5556

# Test IPC socket exists
ls -la /tmp/nginx-logs.ipc

1. Check NGINX error log for ZeroMQ errors:

grep -i zmq /var/log/nginx/error.log

Subscriber Missing Initial Messages

ZeroMQ PUB/SUB has a known “slow joiner” behavior: when a subscriber first connects, it may miss messages sent during the connection handshake. This is by design in ZeroMQ. To work around it:

• Start subscribers before NGINX (or restart NGINX after subscribers are running)
• Send a warm-up request after NGINX starts to trigger the ZeroMQ connection, then wait 1-2 seconds before relying on message delivery
• For lossless delivery, consider adding a message sequence number in your format and detecting gaps on the subscriber side

Debug Logging

Enable NGINX debug logging to see detailed ZeroMQ module output:

error_log /var/log/nginx/error.log debug;

Then filter for ZMQ-specific messages:

grep "ZMQ:" /var/log/nginx/error.log

Security Best Practices

When deploying the NGINX ZeroMQ log module in production, consider the following:

• Bind to localhost: Use 127.0.0.1 for TCP connections unless the subscriber genuinely runs on a different host. ZeroMQ does not provide authentication by default.
• Use IPC for local delivery: Unix socket permissions prevent unauthorized access. Ensure the socket path is not world-readable.
• Sanitize log data: If your subscriber processes log data as structured input (JSON, SQL), be aware that user-controlled fields like $http_user_agent or $request_uri may contain injection payloads. For privacy-sensitive deployments, consider using the ipscrub module to anonymize IP addresses before they reach your ZeroMQ subscribers.
• Limit queue size: An unbounded queue can consume excessive memory under subscriber failures. Always set an explicit queue_size.
• Firewall ZMQ ports: If using TCP, ensure ZeroMQ ports are not exposed to untrusted networks.

Conclusion

The NGINX ZeroMQ log module transforms access logging from a passive file-writing operation into an active, real-time data stream. By leveraging ZeroMQ’s PUB/SUB pattern, you can build flexible log processing pipelines with multiple consumers, topic-based routing, and high-throughput local delivery — all without blocking NGINX request handling.

For system administrators who need to feed NGINX log data into monitoring dashboards, analytics pipelines, or custom processing scripts, the NGINX ZeroMQ log module provides a robust foundation. Its asynchronous architecture ensures that logging never impacts your server’s ability to serve requests, even under heavy load.

Install the module from the GetPageSpeed RPM repository or the APT repository and start streaming your NGINX logs today.