Your CLAUDE.md Is Public: Block AI Agent Files in NGINX

AI coding agents leave their instruction files in your repository, and deployments copy them straight into the web root. To block AI agent files in NGINX, add a short deny list for CLAUDE.md, AGENTS.md, .cursorrules and prompts/, then go one step further: auto-ban every client that probes for them, using the GetPageSpeed honeypot stack.

The quick answer

Deny the files with four lines of NGINX config (below), and turn the probes into instant kernel-level bans:

sudo dnf install https://extras.getpagespeed.com/release-latest.rpm
sudo dnf install nginx-honeypot nginx-module-nftset-access

The nginx-honeypot package is free to install from the GetPageSpeed repository and detects bot-bait requests out of the box. The kernel-level ban path uses nginx-module-nftset-access, which requires the Pro plan (or higher) of the GetPageSpeed NGINX Extras subscription. On Debian and Ubuntu, copy the honeypot configs from the GitHub repository; the deny list below works on any NGINX.

Why your agent files are on the internet

AI-assisted projects accumulate instruction files that were never meant to leave the team:

• CLAUDE.md, CLAUDE-*.md for Claude Code
• AGENTS.md, the cross-tool agent instructions standard
• .cursorrules and .cursor/ for Cursor
• .github/copilot-instructions.md for GitHub Copilot
• prompts/ directories with task briefs and internal playbooks

A typical quick deployment is a git clone into the web root, or a build step that copies the whole repository. Consequently, everything above becomes a public URL. These files describe your infrastructure layout, internal hostnames, database schemas, business logic and operational conventions. Prompt directories are worse: they often quote real tickets, customer emails and API keys verbatim.

Scanners have noticed. The same crawlers that have hammered /.env and /.git/config for years now request /CLAUDE.md, /AGENTS.md and /.cursorrules on any host that answers. An exposed agent file is reconnaissance gold: it tells an attacker how your system is wired together, in plain English, written specifically to be machine-readable.

One important distinction before blocking anything: robots.txt and llms.txt are meant to be public. They are documents you publish for crawlers and language models. Agent instruction files are the opposite: private operational context. Do not confuse the two, and do not block the former while hiding the latter.

The free fix: a deny list

The simplest way to block AI agent files is a deny list; stock NGINX can do this, and no module is needed for a plain 404. Add this inside every server {} block that serves an application:

# Agent instruction files: not for the public
location ~* ^/(claude[^/]*|agents)\.md$ {
    return 404;
}

# Prompt libraries deployed with the repo
location ^~ /prompts/ {
    return 404;
}

# Dotfiles (.env, .git, .cursorrules, .github/) - keep ACME reachable
location ~ /\.(?!well-known/) {
    return 404;
}

The first block catches CLAUDE.md, CLAUDE-reference.md, claude.md and AGENTS.md in any letter case. The second hides prompt directories. The third is the classic dotfile deny that covers .env, .git, .cursorrules and .github/copilot-instructions.md, with a negative lookahead so Let’s Encrypt HTTP-01 challenges under /.well-known/ keep working.

Verify it:

$ curl -o /dev/null -sw '%{http_code}\n' http://localhost/CLAUDE.md
404
$ curl -o /dev/null -sw '%{http_code}\n' http://localhost/.github/copilot-instructions.md
404
$ curl -o /dev/null -sw '%{http_code}\n' http://localhost/llms.txt
200

We return 404 rather than 403 deliberately: a 403 confirms the file exists, a 404 looks identical to a host that never had it.

This is where most guides stop. However, a deny list only protects the one server block you remembered to edit, and the scanner moves on to its next probe against your site within milliseconds. You silenced one request; the reconnaissance continues.

Do not just 404 the scanner. Ban it.

Here is the key insight: no legitimate visitor ever requests /CLAUDE.md. Not one. A request for an agent instruction file is a self-identifying malicious probe, which makes it perfect honeypot bait. Instead of answering politely, you can use the request to drop every subsequent packet from that IP address at the kernel level, before NGINX even sees it.

That is exactly what the nginx-honeypot package does. It ships a curated “honey” map of bot-bait request patterns (/.env, phpMyAdmin paths, exploit probes and friends) and returns 410 Gone for all of them:

sudo dnf install https://extras.getpagespeed.com/release-latest.rpm
sudo dnf install nginx-honeypot
sudo ln -s /etc/nginx/honeypot/honey.conf /etc/nginx/conf.d/honey.conf

The shipped honey map already traps all dotfiles, so .env, .cursorrules and .github/copilot-instructions.md are covered on day one. Agent Markdown files and prompt directories are our own addition. Drop this map next to it:

# /etc/nginx/conf.d/agent-bait.conf
# Agent instruction files as honeypot bait
map $uri $agent_bait {
    default 0;
    "~*^/(claude[^/]*|agents)\.md$" 1;
    "~*^/prompts/" 1;
}

Wiring the ban

Detection alone (the 410 Gone response) is free and needs nothing but NGINX. The ban is where the honeypot pays off, and the production-grade path is the nginx-module-nftset-access module from the GetPageSpeed repository (Pro plan). It writes the offending IP straight into an nftables set from inside NGINX: no fail2ban tailing logs, no CGI scripts, no sudo chains, and the ban lands in microseconds.

sudo dnf install nginx-module-nftset-access

Load the module at the top of /etc/nginx/nginx.conf:

load_module modules/ngx_http_nftset_access_module.so;

One-time nftables preparation (IPv4 shown; mirror it with an ip6 table and ipv6_addr type for IPv6):

sudo nft add table ip filter
sudo nft add set ip filter honeypot '{ type ipv4_addr; flags timeout; }'
sudo nft add chain ip filter input '{ type filter hook input priority 0; }'
sudo nft add rule ip filter input ip saddr @honeypot drop

Then, in each server {} block, include the ban variant and point your agent-bait map at it:

server {
    listen 80;
    server_name example.com;
    root /srv/www/site;

    include honeypot/server-ban-nftset.conf;

    if ($agent_bait) {
        return 410;
    }
}

The included server-ban-nftset.conf routes every 410 through the trap location:

error_page 410 = @honeypot;
if ($has_flies = 1) {
    return 410;
}
location @honeypot {
    nftset_autoadd filter:honeypot timeout=86400 status=410;
}

Any request matching either the stock honey map ($has_flies) or your agent-bait map lands in @honeypot, where nftset_autoadd adds the client IP to the honeypot set with a 24-hour auto-expiring timeout. The module needs CAP_NET_ADMIN; its selinux subpackage grants the capability and installs the SELinux policy automatically, so the whole setup is dnf install plus the four nft commands above.

Watching it work

Reload NGINX and probe your own server once:

$ curl -o /dev/null -sw '%{http_code}\n' http://localhost/CLAUDE.md
410
$ sudo nft list set ip filter honeypot
table ip filter {
    set honeypot {
        type ipv4_addr
        flags timeout
        elements = { 127.0.0.1 timeout 1d expires 23h59m59s }
    }
}
$ curl --max-time 4 http://localhost/
curl: (28) Connection timed out

The first request for /CLAUDE.md got 410 Gone and a set entry. The second request, for a perfectly innocent URL, never reached NGINX at all: the kernel dropped it. That scanner is done enumerating your host for the next 24 hours, and your access log stays quiet. (Testing from localhost bans 127.0.0.1; clean up with sudo nft flush set ip filter honeypot.)

If you operate from a fixed IP address, insert an accept rule above the drop so you can never lock yourself out:

sudo nft insert rule ip filter input ip saddr 198.51.100.7 accept

For a broader never-ban list covering monitoring services and search engine crawlers, see Trusted IP Lists for FirewallD, NGINX and fail2ban.

Prefer a no-subscription ban path? The package also ships a free fcgiwrap variant (server-ban-fcgiwrap.conf) that drives ipset through a CGI script. It works, at the cost of extra moving parts and a slower ban; the setup is covered in NGINX Honeypot 2.0. On legacy ipset-based hosts, the equivalent Pro module is nginx-module-ipset-access.

The bigger honeypot picture

Agent files are just the newest bait. The honeypot approach, returning destruction instead of information for requests only bots make, is one of the highest-leverage security moves on a LEMP stack, and we have refined it over three generations:

• NGINX honeypot: the easiest and fastest way to block bots introduced the concept
• NGINX Honeypot 2.0 removed external scripts from the hot path
• NGINX Honeypot 3.0 moved the ban into nftables

This article’s agent-bait map simply extends the same machinery to the newest class of leaks.

Not sure whether your current config serves files it should not? Run it through our free NGINX configuration check, which flags security issues in seconds.

Conclusion

A honeypot you verified today is only as stable as its next dependency update. A routine package upgrade can silently change the exact trap behavior you just tested. GetPageSpeed Amplify runs scheduled gixy scans across every host and ties findings to live NGINX runtime metrics. Drop-in compatible with the deprecated nginx-amplify-agent (EOL January 2026).

Your CLAUDE.md is a map of your infrastructure written for machines to read, and machines are reading it. Block AI agent files with the deny list, then make the probes cost something: nginx-honeypot detects them for free, and nginx-module-nftset-access turns each probe into a 24-hour kernel-level ban. Both install in minutes from the GetPageSpeed repository. The ban module is included in the Pro plan of the NGINX Extras subscription, alongside 100+ other production-grade NGINX modules.