Symptoms

After enabling a jail, the Fail2ban service can’t be restarted or stopped.

The service status is shown incorrectly, without the jails list:

# service fail2ban status
fail2ban-server (pid  3291) is running

/var/log/fail2ban.log shows that the service stopped when adding log files to its monitoring pool:

2014-07-27 21:09:25,487 fail2ban.filter [25047]: INFO    Added logfile = /var/www/vhosts/system/dom1.com/logs/proxy_access_log
2014-07-27 21:09:25,985 fail2ban.filter [25047]: INFO    Added logfile = /var/www/vhosts/system/domain.com/logs/proxy_access_ssl_log

Cause

Fail2ban has plesk-apache-badbot and plesk-apache (or other big) jails enabled. That jail forces Fail2ban to parse all the access and error logs for each virtual host and Apache’s access log.

If there are a lot of virtual host access logs, the service hangs as a result of resource overusage when trying to parse them.

NOTE: When you enable this jail in Plesk, you may see the warning:

Warning: Fail2Ban might not work well if there are many domains and Fail2Ban has to monitor too many log files.

Resolution

The following instructions should be applied if there are less then 300 domains, and the number of log files in the jail should be reduced:

1. Kill the stuck processes by PID (exercise caution: it may be a good idea to check the PIDs which will be killed first, by omitting the last part of the command after |):

   # ps aux | grep fail2ban|awk '{print $2}'|xargs kill -9
   

2. Remove the .pid file:

   # rm -f /var/run/fail2ban/fail2ban.pid
   

3. Reduce the number of logs to parse for the jail plesk-apache-badbot (or disable the jail altogether). Open the file /etc/fail2ban/jail.d/plesk.conf and change the mask of the path to logs from '*access*log' to'*access_log':

   [plesk-apache-badbot]
   
   enabled  = true
   filter   = apache-badbots
   action   = iptables-multiport[name=BadBots, port="http,https,7080,7081"]
   logpath  = /var/www/vhosts/system/*/logs/*access_log
              /var/log/httpd/*access_log
   

4. If the Fail2ban service is running, execute fail2ban-client reload. Otherwise, start the service.

The following instructions are for a large amount of domains (more than 300):

Fail2ban can use a lot of RAM on the server if it monitors numerous jails with many log files. Make sure that the server will not experience an out-of-memory condition before applying this solution. If it does, disable some jails.

If you have a very large number of domains on your Plesk server and the above workaround did not help, divide the logs by different jails, so they are loaded one-by-one. This minimizes the number of logs in one jail.

Since the issue is caused by a single big jail with a large number of logs, creating a higher number of jails containing smaller amounts should help.

Use the following commands to create separate jails for domains according to the first name letter\digit:

1. Get the admin email:

   admin_email=`mysql -Ns -uadmin -p\`cat /etc/psa/.psa.shadow\` psa -Ne"select email from clients where login='admin'"`
   

2. Set plesk-apache jails:

    for i in a b c d e f g h i j k l m n o p q r s t u v w x y z 1 2 3 4 5 6 7 8 9 0;do find /var/www/vhosts/system/$i*/logs/error_log 2>/dev/null 1>/dev/null; found=`echo $?`;if [ $found == "0" ];then echo "[[\"usedns\",\"no\"],[\"logpath\",\"\\/var\\/www\\/vhosts\\/system\\/$i*\\/logs\\/error_log\"],[\"enabled\",\"true\"],[\"filter\",\"apache-auth\"],[\"maxretry\",\"6\"],[\"__source__\",\"jail.d\\/plesk.conf\"],[\"action\",\"iptables-multiport[name=apache, port=\\\"http,https,7080,7081\\\"]\"],[\"ignoreip\",\"127.0.0.1\/8\"],[\"bantime\",\"600\"],[\"destemail\",\"$admin_email\"],[\"findtime\",\"600\"],[\"backend\",\"auto\"]]"|/usr/local/psa/admin/bin/f2bmng --set-jail plesk-apache-$i ;fi;done
   

3. Set plesk-apache-badbot jails:

    for i in a b c d e f g h i j k l m n o p q r s t u v w x y z 1 2 3 4 5 6 7 8 9 0;do find /var/www/vhosts/system/$i*/logs/error_log 2>/dev/null 1>/dev/null; found=`echo $?`;if [ $found == "0" ];then echo "[[\"usedns\",\"no\"],[\"logpath\",\"\\/var\\/www\\/vhosts\\/system\\/$i*\\/logs\\/*access_log\"],[\"enabled\",\"true\"],[\"filter\",\"apache-badbots\"],[\"maxretry\",\"100\"],[\"__source__\",\"jail.d\\/plesk.conf\"],[\"action\",\"iptables-multiport[name=BadBots, port=\\\"http,https,7080,7081\\\"]\"],[\"ignoreip\",\"127.0.0.1\/8\"],[\"bantime\",\"172800\"],[\"destemail\",\"$admin_email\"],[\"findtime\",\"600\"],[\"backend\",\"auto\"]]" |/usr/local/psa/admin/bin/f2bmng --set-jail plesk-apache-badbot-$i;fi;done
   

4. In the regular plesk-apache-badbot and plesk-apache jails, only leave the general error\access logs file paths:plesk-apache-badbot:

   /var/log/httpd/*error_log
   

   plesk-apache:

   /var/log/httpd/*access_log
   

5. On steps 2 and 3, we only created jails if there were some domains matching the first digit\letter of the domain name. Otherwise, Fail2ban will not start due to configuration errors. Now, we need to set up a script that adds a jail when new domains are created:
   • Download the attached script , put it on your server, and grant it executable permissions:

     wget http://kb.odin.com/Attachments/kcs-32570/add_jails.sh
     
     chmod +x add_jails.sh
     

   • Create tasks in the Plesk Event Manager with the following parameters:Domain created lowest (0) root /root/add_jails.sh <new_domain_name>

     Default domain (the first domain added to a subscription or webspace) created lowest (0) root /root/add_jails.sh <new_domain_name>

6. Use the instructions above to restart Fail2ban if required.