Troubleshooting
fail2ban bans the proxy IP instead of the visitor
Behind a reverse proxy, load balancer or CDN, every request appears to come from the proxy. fail2ban bans that one address and locks out everyone, or bans nothing at all. The fix has two parts: log the real client IP, then ban where the real IP can be blocked.
If fail2ban bans 127.0.0.1, your load balancer or Cloudflare addresses, the log it reads records the proxy as the client. First make the web server log the real visitor IP: nginx set_real_ip_from plus real_ip_header, or Apache mod_remoteip, trusting only your proxy's addresses. Then ban at a layer the visitor actually connects to. That's the proxy host, Cloudflare's firewall via the cloudflare-token action, or the web server itself. Never ban the proxy.
Quick diagnosis#
Look at what the log fail2ban reads actually records as the client address, and what fail2ban banned:
sudo tail -n 5 /var/log/nginx/access.log
sudo fail2ban-client banned
sudo grep "Ban " /var/log/fail2ban.log | tail| You see | Meaning |
|---|---|
Every log line starts with 127.0.0.1 or ::1 |
A local proxy forwards to this app. ignoreself silently drops every failure, so nothing is banned. |
Every line shows one LAN IP, e.g. 10.0.0.5 |
Load balancer or proxy host. fail2ban bans it and the whole site goes down. |
| Lines show Cloudflare addresses | The origin sees Cloudflare, not visitors. Bans hit Cloudflare edges at random. |
| Lines show real visitor IPs, bans don't work | Logging is fine. The ban is applied where traffic doesn't pass, see step 2. |
Step 1: make the web server log the real client IP#
The proxy passes the visitor's address in a header, X-Forwarded-For, X-Real-IP or Cloudflare's CF-Connecting-IP. The backend must be told to trust that header only from your proxies. If you trust it from anyone, an attacker can send a fake X-Forwarded-For to dodge bans or get someone else banned.
Nginx behind a proxy or load balancer#
The ngx_http_realip_module is included in the Debian, Ubuntu and nginx.org packages:
set_real_ip_from 10.0.0.5; # your proxy / load balancer (repeat per address or CIDR)
real_ip_header X-Forwarded-For;
real_ip_recursive on;With this, $remote_addr becomes the visitor's address in both access and error logs, which is what the stock nginx-* filters read. Reload nginx and check the log again.
Nginx behind Cloudflare#
Cloudflare publishes its ranges at https://www.cloudflare.com/ips-v4 and https://www.cloudflare.com/ips-v6. Generate the config from the live lists rather than copying a list that goes stale:
{
for u in ips-v4 ips-v6; do curl -fsS "https://www.cloudflare.com/$u"; echo; done \
| sed '/^$/d; s/.*/set_real_ip_from &;/'
echo 'real_ip_header CF-Connecting-IP;'
} | sudo tee /etc/nginx/conf.d/cloudflare-real-ip.conf
sudo nginx -t && sudo systemctl reload nginxRe-run it now and then, for example from a monthly cron job, since Cloudflare occasionally adds ranges.
Apache: mod_remoteip#
RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy 10.0.0.5sudo a2enmod remoteip
sudo a2enconf remoteip
sudo systemctl reload apache2For Cloudflare use RemoteIPHeader CF-Connecting-IP and one RemoteIPTrustedProxy line per Cloudflare range. In custom access log formats use %a (the client address after mod_remoteip) instead of %h. The error log, which most apache-* filters read, uses the corrected address automatically.
Traefik, Caddy and Nginx Proxy Manager#
When the proxy itself is the thing writing the log fail2ban reads, configure it to trust the upstream layer:
- Traefik: set
forwardedHeaders.trustedIPson the entry point to your CDN or load balancer ranges, and make sure the access log contains the forwarded address. See the Traefik guide. - Caddy: add
trusted_proxies(for exampletrusted_proxies static <ranges>) in the globalserversoptions. The access log'sclient_ipfield then holds the real address. See the Caddy guide. - Nginx Proxy Manager: its logs record the client address in the
[Client …]field. Behind Cloudflare, add the real_ip settings in a custom config. See fail2ban with Nginx Proxy Manager.
Apps behind a local proxy#
Apps such as Nextcloud, Vaultwarden, Gitea or Home Assistant behind a local nginx log 127.0.0.1 unless you tell them which proxies to trust. Each has its own setting, like Nextcloud's trusted_proxies or Home Assistant's use_x_forwarded_for plus trusted_proxies. After changing it, fail2ban's app filter sees real addresses.
Step 2: ban at a layer that sees the real IP#
Logging the right IP is only half the job. A firewall ban works only where the attacker's packets arrive from the attacker's own address.
| Setup | Where to ban |
|---|---|
| Nginx/Apache/Caddy on the same host as the app, visitors connect directly | Local firewall works. Run the jail on the proxy's log. |
| Separate proxy or load balancer host | Run fail2ban on the proxy host, reading its logs. Or keep fail2ban on the app host and ban in the web server (below). |
| Cloudflare (orange cloud) in front | Ban at Cloudflare with the cloudflare-token action. Origin firewall bans do nothing for proxied traffic. |
| Cloudflare Tunnel | No inbound connections at all. Ban at Cloudflare. |
Ban at Cloudflare#
[nginx-http-auth]
enabled = true
action = cloudflare-token[cfzone="your-zone-id", cftoken="your-api-token"]
%(action_)sThe action creates a Cloudflare IP access rule in block mode for each ban and deletes it on unban. Create an API token limited to the zone's firewall settings, and keep jail.local at mode 640, since it contains the token. Setup details are in the Cloudflare guide.
Ban inside nginx#
If you can't touch the edge, let nginx refuse banned visitors by their real address. The nginx-block-map action appends banned IDs to a map file and reloads nginx. Point it at a file you map against $remote_addr, which already holds the real IP thanks to step 1:
# http {} block
map $remote_addr $f2b_banned { include /etc/nginx/f2b-banned.map; }
# server {} block
if ($f2b_banned) { return 403; }[nginx-http-auth]
enabled = true
action = nginx-block-map[blck_lst_file="/etc/nginx/f2b-banned.map"]The action creates the map file when the jail starts, writes one line per ban, tests the nginx config and reloads nginx. Make sure the file exists before your first nginx -t (sudo touch /etc/nginx/f2b-banned.map).
Never ban the proxy#
Even with correct logging, add your proxy and CDN addresses to ignoreip. A single misconfigured log line could otherwise ban the address all your traffic comes through. fail2ban 1.1.1 can read the list from a file:
[DEFAULT]
ignoreip = 127.0.0.1/8 ::1 10.0.0.5 file:/etc/fail2ban/cloudflare-ips.txtfor u in ips-v4 ips-v6; do curl -fsS "https://www.cloudflare.com/$u"; echo; done | sudo tee /etc/fail2ban/cloudflare-ips.txtOn 1.1.0 and older, list the ranges inline instead. See whitelisting with ignoreip.
Confirm it's fixed#
- Make a failing request from an outside address, such as a phone on mobile data.
- The log line shows that address, not the proxy's.
/var/log/fail2ban.logshowsFoundand thenBanfor that address.- The client is blocked: by Cloudflare, by nginx returning 403, or by the proxy host's firewall.
Then unban your test address.
FAQ#
How do I make fail2ban use X-Forwarded-For?#
fail2ban reads whatever address is in the log, so make the web server write the forwarded address. In nginx use set_real_ip_from <proxy> and real_ip_header X-Forwarded-For; in Apache use mod_remoteip. Writing a filter that grabs the header from the log also works, but trusting the header only from known proxies is safer.
Why does fail2ban ban 127.0.0.1 or never ban anything behind a proxy?#
The app behind a local proxy logs 127.0.0.1 as the client. With ignoreself = true, the default, fail2ban ignores that address, so nothing is banned. If 127.0.0.1 isn't ignored, every visitor is banned at once. Configure the app to trust the proxy and log the forwarded client address.
Spotted something wrong or out of date? Tell us — every page is checked against the current fail2ban release.