Protect your signup forms from SPAM with NGINX and CleanTalk

Suppose you’re running Mautic on CentOS 7 machine. You’ve created a signup form and placed it on the front page of your website.
The form is very simple: an input for an email address and Subscribe button.

Bots are hitting it hard. They submit email addresses to your form, your database of subscribers have lots of fake emails.
How do you prevent all of this, without adding the inconvenience of CAPTCHA?

NGINX can be more than just your Web Application Firewall. It can do smart data processing and filter out SPAM signups, all with the help of the Lua module and a third party SPAM database API like CleanTalk.

First, ensure our RPM repository is set up on your system. Then:

yum install nginx-module-lua lua-cjson 

Load the modules in nginx.conf:

load_module modules/ndk_http_module.so;
load_module modules/ngx_http_lua_module.so;

We are going to talk to external API and cache its requests. Create /etc/nginx/conf.d/ctcache.conf:

proxy_cache_path /var/lib/nginx/ctcache levels=1:2 keys_zone=ctcache:16m max_size=128M;
proxy_temp_path  /var/cache/nginx/cleantalk_temp 1 2;
proxy_cache_use_stale error timeout invalid_header http_502;
proxy_cache_bypass $cookie_session;
proxy_no_cache $cookie_session;

Create site-specific configuration by adding internal location that will pass requests over to the CleanTalk API.
And also we create a separate location for the signup form, which will launch our .lua script for checking submitted data:
In our case, we decided to put this to a separate include file at /srv/www/example.com/cleantalk.conf:

# Cache API requests
location /spam_check {
        internal;
        # access_log off;

        proxy_cache ctcache;
        proxy_cache_valid any 10m;
        proxy_cache_valid 404 500 501 502 503 1m;
        proxy_pass https://api.cleantalk.org/;
        # this strips Cookie from the request to API, useful if you have large enough to trigger their server with 400 error
        proxy_set_header Cookie ""; 
}
# POST /m/form/submit?formId=1
location = /m/form/submit {
    # API key for Cleantalk.org
    set $apikey 'xxxxxxxxx';
    # Checking script
    access_by_lua_file /etc/nginx/global/cleantalk.lua;

    fastcgi_read_timeout 360;
    fastcgi_pass unix:/var/run/php-fpm/php-fpm-example.com.sock;
    fastcgi_param SCRIPT_FILENAME $document_root/m/index.php;
    include fastcgi_params;
    fastcgi_param SCRIPT_NAME /m/index.php;
}

Download this file and put it as /etc/nginx/global/cleantalk.lua.

Make sure permissions for the Lua script:

chmod 0644 /etc/nginx/global/cleantalk.lua

Caveats

• The cleantalk.lua does not support forms with enctype attribute set to multipart/form-data. If your form uses that, and does not realy on file upload, you may want to change it to application/x-www-form-urlencoded so it’s compatible with the script. It would be possible to have wider form support with the help of extra Lua libraries, but some bugs have to be resolved
• You can’t apply the above NGINX location to an HTTP/2 website:

2019/09/05 18:17:47 [error] 1084#1084: *9 lua entry thread aborted: runtime error: /etc/nginx/global/cleantalk.lua:34: http2 requests not supported yet

The workaround is creating separate server {} listening on a private port, and having your actual server proxy_pass request data to it.