fail2ban high CPU and memory usage

When fail2ban uses a lot of CPU, the usual cause is the polling backend scanning big or many log files, a huge unrotated log being read from the start, or a regex that backtracks on long lines. Install python3-pyinotify (or use backend = systemd), rotate your logs, and test custom filters with fail2ban-regex. For "too many open files" with journal jails, upgrade to fail2ban 1.1.1, which ignores rotated journal files by default.

Quick diagnosis#

top -p "$(pgrep -d, -f fail2ban-server)"             # CPU and memory of the server
sudo grep -E "uses (poller|pyinotify|systemd)" /var/log/fail2ban.log | tail   # backend per jail
sudo ls /proc/"$(pgrep -f fail2ban-server | head -1)"/fd | wc -l            # open file descriptors
ls -lh /var/lib/fail2ban/fail2ban.sqlite3                                   # database size
sudo fail2ban-client stats                                                  # 1.1.0+: per-jail backend and counters

Cause 1: the polling backend#

With backend = auto, fail2ban tries pyinotify first and falls back to polling if the Python module is missing. Polling checks every watched file about once per second, so dozens of busy log files mean constant work. The log says which one each jail got:

INFO    Jail 'nginx-http-auth' uses poller {}
INFO    Jail 'sshd' uses pyinotify {}

Fix: install the inotify bindings and restart:

sudo apt install python3-pyinotify      # Debian/Ubuntu
sudo dnf install python3-inotify        # Fedora/EPEL
sudo systemctl restart fail2ban

For services that log to the systemd journal, backend = systemd is cheaper still, because it waits on journal events instead of watching files. See backends.

Cause 2: huge or unrotated log files#

The first time a jail sees a log file, fail2ban reads it from the beginning to catch failures that happened before it started (within findtime). After that it remembers its position in the database, but a new file, a deleted database or dbfile = :memory: starts from the top again. A multi-gigabyte access.log that has never been rotated means minutes of regex work each time that happens. A busy log run through an expensive filter, such as nginx-botsearch on every access-log line, costs CPU all the time.

Fixes:

• Rotate logs. Check /etc/logrotate.d/ for the service, and make sure fail2ban's own log is rotated too. A forgotten loglevel = DEBUG makes /var/log/fail2ban.log explode, which also feeds the recidive jail.
• Start reading at the end of a large file by adding tail after the path:

[nginx-botsearch]
enabled = true
logpath = /var/log/nginx/access.log tail

• Point jails at error logs instead of access logs where the filter allows it. Error logs are far smaller.

Cause 3: a regex that backtracks#

Some regular expressions take exponential time on certain input. Typical patterns are nested quantifiers like (.+)+, several .* in a row, or alternations that overlap. One crafted or just unusually long log line can then pin a CPU core for seconds or minutes, and the jail stops processing lines in the meantime.

• Bundled filters: fail2ban 1.1.1 fixed catastrophic backtracking in the dovecot and domino-smtp filters (advisory GHSA-33wh-ccjc-p397), and 1.0.2 fixed a very slow dovecot pattern (gh-3370). If you run these filters on an older version, upgrade or copy the fixed filter. See security advisories.
• Custom filters: time them against a real, large log. fail2ban-regex prints [processed in N sec] at the end:

sudo fail2ban-regex /var/log/myapp.log /etc/fail2ban/filter.d/myapp.local

Make expensive regexes cheaper. Anchor with ^, replace .* with \S+ or [^"]* where possible, move the common prefix into prefregex, and keep maxlines = 1 unless you really need multi-line matching. The regex tester helps you try variants quickly.

Cause 4: too many open files with journal jails#

Each systemd-backend jail opens the journal itself. Before 1.1.1, that meant opening every journal file, including rotated ones, per jail. On a host with a large journal and many journal jails, fail2ban could exhaust its file descriptor limit (Too many open files) and burn CPU seeking through old data.

Fixes:

• Upgrade to fail2ban 1.1.1. The systemd backend has a new rotated option, false by default, so each jail opens only the current journal files, normally one or two descriptors.
• On older versions, shrink the journal: sudo journalctl --vacuum-size=500M, and set SystemMaxUse= in /etc/systemd/journald.conf.
• Exclude per-user journals with backend = systemd[journalflags=4] (system journal only).
• As a stopgap, raise the limit with sudo systemctl edit fail2ban and LimitNOFILE=65536 under [Service].

Cause 5: a large SQLite database#

fail2ban keeps bans in /var/lib/fail2ban/fail2ban.sqlite3, along with up to dbmaxmatches (default 10) log lines per ticket. Old bans are purged after dbpurgeage (default 1d). If you raised dbpurgeage to months for incremental bans and have many bans, the file grows and startup slows down while bans are restored.

[Definition]
dbpurgeage = 7d
dbmaxmatches = 5

dbpurgeage is how long fail2ban remembers previous offenders for bantime.increment, so don't set it shorter than the history you want to count. See fail2ban.conf options.

Cause 6: DNS lookups (usedns)#

With usedns = warn (default) or yes, a log line containing a host name instead of an IP triggers a DNS lookup. Slow or failing DNS stalls the jail. Host names in ignoreip are resolved too, with a five-minute cache. If your logs contain IPs, set usedns = no for those jails, and prefer IPs and CIDRs in ignoreip.

Cause 7: thousands of bans as individual firewall rules#

If top shows the load in ksoftirqd or the system as a whole rather than in fail2ban-server, the cost is in the firewall. With plain iptables actions or ufw, each ban is a separate rule that every packet walks through linearly. Use set-based actions, nftables (the default on Debian and Ubuntu packages) or iptables-ipset, which look up thousands of addresses in constant time. See ban actions.

Memory usage#

fail2ban's memory grows with the number of distinct addresses it's tracking within findtime, and with the matched lines it keeps per address (maxmatches, by default equal to maxretry). During a large distributed attack that can be tens of thousands of entries. Lower findtime on very noisy jails, keep maxmatches small, and let recidive handle long-term offenders instead of a huge findtime.

Confirm it's fixed#

Restart and watch for a few minutes. CPU should settle near zero between bursts, and the descriptor count should stay small and stable:

sudo systemctl restart fail2ban
top -p "$(pgrep -d, -f fail2ban-server)"
sudo ls /proc/"$(pgrep -f fail2ban-server | head -1)"/fd | wc -l

FAQ#

Why is fail2ban using 100% CPU?#

Usually it's the polling backend watching many or busy log files, a very large log being read from the beginning after a start or reload, or a regex that backtracks on long lines. Check /var/log/fail2ban.log for uses poller, install python3-pyinotify, rotate big logs, and time custom filters with fail2ban-regex.

How do I fix "too many open files" in fail2ban?#

It typically comes from many systemd-backend jails opening every rotated journal file. fail2ban 1.1.1 fixes this by ignoring rotated journals by default (rotated = false). On older versions, vacuum the journal, use journalflags=4, or raise LimitNOFILE in a systemd drop-in for the fail2ban service.

Is pyinotify or polling better for fail2ban?#

pyinotify. It reacts to kernel change notifications instead of checking files every second, so it uses almost no CPU while logs are quiet and reacts faster. Polling is the fallback when the pyinotify module isn't installed. For journal-based services, backend = systemd is the efficient choice.

How much memory does fail2ban use?#

It depends mostly on activity, not on log size. It grows with the number of addresses tracked inside findtime, the matched lines kept per address, and the number of jails. Sustained growth during an attack is normal and shrinks again as entries expire. Steady growth in quiet times points to an old version or a misconfigured, very long findtime.