NGINX Device Detection Module: Block AI Crawlers

The NGINX Device Detection Module (ngx_http_device_type_module) performs high-performance device classification directly at the edge. It identifies mobile phones, tablets, desktops, gaming consoles, smart TVs, wearables, and bots—including AI crawlers like GPTBot and ClaudeBot.

Traditional device detection solutions rely on external API calls. This adds latency and per-request costs. The NGINX device detection module performs detection in-process using precompiled regex patterns from over 4,000 User-Agent signatures. Results arrive without network round-trips.

Why Use NGINX Device Detection?

Device detection at the edge enables several powerful capabilities:

• Adaptive content serving — serve mobile-optimized assets without client-side redirects
• Cache segmentation — vary cached content by device type for CDN and proxy caches
• Bot management — distinguish search engines from scrapers, apply rate limiting to crawlers
• AI crawler control — identify GPTBot, ClaudeBot, CCBot, and other AI training bots
• Analytics enrichment — pass device context headers to backend applications

For comprehensive bot protection beyond device detection, see our NGINX bot protection guide.

Module Features

Installation

The NGINX device detection module is available as a prebuilt dynamic module for RHEL-based systems.

RHEL, CentOS, AlmaLinux, Rocky Linux

sudo dnf install https://extras.getpagespeed.com/release-latest.rpm
sudo dnf install nginx-module-device-type

Then load the module in your NGINX configuration:

load_module modules/ngx_http_device_type_module.so;

Note: Debian and Ubuntu packages are coming soon. Check the GetPageSpeed repository for updates.

Variables Reference

The module exposes 22 NGINX variables for comprehensive device detection.

Device Classification Variables

Device Details Variables

Browser Detection Variables

Operating System Variables

Bot Classification Variables

JSON Output Variable

Configuration Examples

Block AI Training Crawlers

Prevent AI crawlers like GPTBot, ClaudeBot, and CCBot from accessing your content:

server {
    listen 80;
    server_name example.com;

    location / {
        if ($is_ai_crawler) {
            return 403 "AI crawlers not permitted";
        }
        proxy_pass http://backend;
    }
}

This configuration returns a 403 Forbidden response to AI crawlers. Regular search engine bots and human visitors pass through normally.

Adaptive Content Serving

Serve device-optimized content by passing variant headers to your backend:

server {
    listen 80;
    server_name example.com;

    location / {
        set $variant "desktop";
        if ($is_mobile) { set $variant "mobile"; }
        if ($is_tablet) { set $variant "tablet"; }

        proxy_set_header X-Device-Type $device_type;
        proxy_set_header X-Device-Variant $variant;
        proxy_set_header X-Device-Brand $device_brand;
        proxy_set_header X-Browser $browser_name;
        proxy_set_header X-OS $os_name;
        proxy_pass http://backend;
    }
}

Your backend receives device context in HTTP headers. This enables server-side responsive design without client-side detection overhead.

Cache Key Variation

Vary cached content by device type for proxy caches or CDN integration. For details on NGINX caching strategies, see our browser caching guide.

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=content:10m;

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_cache content;
        proxy_cache_key "$scheme$request_uri|$device_type";
        proxy_pass http://backend;
    }
}

For simpler mobile vs desktop caching:

proxy_cache_key "$scheme$request_uri|$is_mobile";

Device Detection Debug Endpoint

Create a debug endpoint that returns all detection information:

location = /debug/device {
    default_type text/plain;
    return 200 "Type: $device_type
Mobile: $is_mobile
Tablet: $is_tablet
Desktop: $is_desktop
Bot: $is_bot ($bot_name)
AI Crawler: $is_ai_crawler
Browser: $browser_name $browser_version
Engine: $browser_engine
OS: $os_name $os_version ($os_family)
Device: $device_brand $device_model
";
}

JSON Detection Endpoint

Return the complete detection result as JSON for API integrations:

location = /device.json {
    default_type application/json;
    return 200 $device_json;
}

Example JSON response for a mobile device:

{
  "type": "mobile",
  "browser": {"name": "Chrome Mobile", "version": "120.0.6099.43", "engine": "Blink"},
  "os": {"name": "Android", "version": "14", "family": "Android"},
  "device": {"brand": "Google", "model": "Pixel 8"},
  "bot": null
}

Example JSON response for an AI crawler:

{
  "type": "bot",
  "browser": {"name": "", "version": "", "engine": ""},
  "os": {"name": "", "version": "", "family": ""},
  "device": {"brand": "", "model": ""},
  "bot": {"name": "GPTBot", "category": "ai_crawler", "producer": "OpenAI OpCo, LLC", "is_ai": true}
}

Custom Log Format with Device Information

Include device detection data in your access logs for analytics:

log_format device_log '$remote_addr - $remote_user [$time_local] "$request" '
                       '$status $body_bytes_sent "$http_referer" '
                       '"$http_user_agent" '
                       'device=$device_type mobile=$is_mobile bot=$is_bot ai=$is_ai_crawler';

server {
    listen 80;
    server_name example.com;

    access_log /var/log/nginx/device.log device_log;

    location / {
        proxy_pass http://backend;
    }
}

Log output example:

127.0.0.1 - - [16/Feb/2026:12:30:45 +0000] "GET / HTTP/1.1" 200 1234 "-" "GPTBot/1.0" device=bot mobile=0 bot=1 ai=1

Client Hints Support

For improved NGINX device detection accuracy with Chromium-based browsers, advertise Client Hints:

add_header Accept-CH "Sec-CH-UA, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version";
add_header Critical-CH "Sec-CH-UA-Mobile";

The module processes these Client Hints headers:

Client Hints provide more accurate device information than User-Agent parsing. They represent the future of device detection on the web.

Detection Priority

The module evaluates signals in a specific order for accurate classification:

1. Bot detection — regex match against 800+ bot patterns
2. Client Hints — if present, used for mobile/tablet/browser/OS detection
3. Quick UA checks — fast substring matches for iPad, iPhone, PlayStation, etc.
4. Full device patterns — 2,000+ regex patterns for brand/model extraction
5. Android tablet heuristic — Android without “Mobile” token classified as tablet
6. Mobile fallback — presence of “Mobile” token classifies as mobile
7. Browser detection — 500+ patterns with version capture
8. Engine detection — Blink, Gecko, WebKit identification
9. OS detection — 300+ patterns with family classification

Results cache per-request in the module context. Detection runs only once regardless of how many variables you access.

Performance Considerations

The module compiles all regex patterns at NGINX startup. It stores them in shared memory for efficient per-request matching:

• Startup compilation — patterns compile once when NGINX loads
• In-memory matching — no file I/O or network calls during requests
• Per-request caching — detection runs once per request, results reused
• Zero configuration — no runtime configuration parsing

For high-traffic deployments, the module adds negligible latency. External device detection APIs require network round-trips that this module eliminates.

Comparison with Alternatives

Troubleshooting

Module Not Loading

If NGINX fails to start with an unknown variable error:

nginx: [emerg] unknown "device_type" variable

Verify the module is loaded in your main nginx.conf:

load_module modules/ngx_http_device_type_module.so;

The load_module directive must appear before any http, stream, or mail blocks.

Verify Module Installation

Check that the module file exists:

ls -la /usr/lib64/nginx/modules/ngx_http_device_type_module.so

List the variables exported by the module:

strings /usr/lib64/nginx/modules/ngx_http_device_type_module.so | grep -E '^(device_|is_|browser_|os_|bot_)'

Empty Variable Values

If variables return empty values, verify the request includes a User-Agent header. The module requires User-Agent or Client Hints headers for detection.

Test with curl:

curl -s -A "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X)" http://localhost/debug/device

Conclusion

The NGINX Device Detection Module provides fast, accurate device classification directly at the edge. It eliminates the need for external API calls or client-side detection libraries. With support for AI crawler identification, Client Hints, and comprehensive bot categorization, the module enables sophisticated content delivery strategies while maintaining high performance.

For more information and installation options, visit the module documentation.