NGINX-MOD: a better, faster NGINX build

As you may know, our repository holds the latest stable NGINX and vast array of dynamic modules for it.

However, some performance oriented folks are always looking for speeding up what’s already fast – that is NGINX itself.

There are some open source patches for it, mainly by Cloudflare to improve things further. So I’ve decided to save trouble for many people relying on manual compilation, and build this better patched NGINX as a package that is compatible with all the NGINX modules we have! I call it – NGINX-MOD.

At present, the NGINX-MOD is based on latest stable NGINX with the following:

  • Latest OpenSSL 1.1.x (allows for TLS 1.3 to be configured)
  • Patch for HTTP/2 HPACK (performance)
  • Patch for dynamic TLS records (performance)
  • Patch that allows ngx_http_limit_req_module to support rates of an hour, minute, day, week and year

More on those patches in the documentation below.

How to install NGINX-MOD

yum -y install https://extras.getpagespeed.com/release-el7-latest.rpm
yum -y update getpagespeed-release
yum -y install yum-utils
yum-config-manager --enable getpagespeed-extras-nginx-mod
yum install nginx

How to switch to NGINX-MOD from our regular NGINX

If you were using our regular NGINX build, you can run a series of commands to upgrade to NGINX-MOD without affecting installed modules or configuration:

yum -y update getpagespeed-release
yum -y install yum-utils
yum-config-manager --enable getpagespeed-extras-nginx-mod
yum update nginx

There are some caveats here. Primarily, there is one module that depends on specific OpenSSL version: nginx-module-lua.
So as long as you use NGINX-MOD and want to install the Lua module, keep this in mind:

  • there are 2 versions of lua module: one in the nginx-mod repository and one in the extras repository
  • the version in mod repository is comprised of <nginx version>.<openssl version>.<module version> so you can easily tell which OpenSSL it was compiled with and whether it’s a “mod” version
  • since NGINX-MOD is compiled with newer OpenSSL, you need the “mod” version of Lua module. If you already have it, you will be offered an upgrade to it

How to switch back to stable NGINX

Going back to the stable package while preserving existing configuration:

yum-config-manager --disable getpagespeed-extras-nginx-mod
rpm --erase --justdb --nodeps nginx-mod # add "nginx-mod-module-lua" if you use Lua module
yum install nginx # add "nginx-module-lua" if you use Lua module
yum history sync

Quick Documentation

ngx_http_limit_req_module patch

Some NGINX users seek to define rate limiting of once in a day for specific resources. This is not possible with stock NGINX.
Our patch allows for a more fine-grained rate limit configuration. Examples:

limit_req_zone $binary_remote_addr zone=one:10m rate=1r/h; # 1 request per hour
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/d; # 1 request per day
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/w; # 1 request per week
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/M; # 1 request per month
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/Y; # 1 request per year

It is important to note, that your defined zone memory size should allow retaining old IP entries before the defined rate will apply.

Example: you have defined a 10m zone and 1r/d for a particular resource. 10m can store around 160,000 IP addresses.
So if someone visits your rate limited resource, and your traffic to it exceed 160K unique visitors within 24 hrs, then the same visitor can theoretically not be rate-limited within the same day, because information about his IP address will be evicted from memory after enough visitors visited the resource.

This note applies to stock module’s configuration as well, but less so.

So the rules of thumb are:

  • You likely need to increase memory zone, if your traffic is sufficient to be able to evict old IP addresses “too early”
  • This is more appropriate for rate limiting specific resources, not the whole website

What is HPACK Patch

HPACK patch implements full HPACK in NGINX. In short, this allows for compressing HTTP headers

There are some configuration directives in this build, which are not otherwise available in regular builds. Let’s document them here.

Configuration Directives

The following set of configuration directives are added by dynamic TLS records patch.

ssl_dyn_rec_enable on|off

Whether to enable dynamic TLS records.

ssl_dyn_rec_size_lo

The TLS record size to start with. Defaults to 1369 bytes (designed to fit the entire record in a single TCP segment: 1369 = 1500 – 40 (IPv6) – 20 (TCP) – 10 (Time) – 61 (Max TLS overhead))
ssl_dyn_rec_size_hi: the TLS record size to grow to. Defaults to 4229 bytes (designed to fit the entire record in 3 TCP segments)

ssl_dyn_rec_threshold

The number of records to send before changing the record size.

Because we build with latest OpenSSL:

ssl_protocols [SSLv2] [SSLv3] [TLSv1] [TLSv1.1] [TLSv1.2] [TLSv1.3];

Not a new directive. But since we build with most recent stable OpenSSL, it allows for TLSv1.3 value to be used.

Verification

To verify how you benefit from NGINX-MOD, you can run some tests.

Check HTTP/2 headers compression

yum install nghttp2
h2load https://example.com -n 2 | tail -6 |head -1

Example output:

traffic: 71.46KB (73170) total, 637B (637) headers (space savings 78.68%), 70.61KB (72304) data

If you see 50% or more space savings, then it means that full HPACK compression is utilized.