Web-pages time out

[chris]

I have a probleme with my website.
Browsers can no longer connect, get timeouts.

Website takes to long te respond.

I don't understand why, the server should have enough resources for the site.
top and free, show that CPU and memory are not a problem.

I increased the number of workers in /etc/apache2/mods-enabled/mpm_prefork.conf
But it doesn't solve the problem....

[chris]

find more info in the logs

Code: Select all

tail /var/log/apache2/website.log

Code: Select all

42.187.xxx.xxx - - [15/Oct/2025:18:05:23 +0200] "-" 408 7229 "-" "-"
42.179.xxx.xx - - [15/Oct/2025:18:05:26 +0200] "-" 408 339 "-" "-"
182.34.xx.xx - - [15/Oct/2025:18:05:28 +0200] "-" 408 339 "-" "-"
42.81.xx.xx - - [15/Oct/2025:18:05:31 +0200] "-" 408 7244 "-" "-"
101.47.xx.xxx - - [15/Oct/2025:18:05:31 +0200] "-" 408 339 "-" "-"

Many 408 errors -> Request timeout
and no page in this request, so it looks like a deny of service attack... ( slow HTTP attack)

As 99% is one one of my virtual servers and not the others, makes me think it even more.

This slow Denial-of-service (DoS) attacks, where the attacker send HTTP request in pieces are incomplete. Just to keep the (apache) workers busy waiting for the rest of the data. Until all the workers are used, and can't handle new resuests.

The command (wich also runs slowly now), confirms.

Code: Select all

/sbin/apache2ctl status

All the workers are in R = Reading Request

Code: Select all

  250 requests currently being processed, 0 workers gracefully restarting,
          0 idle workers

RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR

   Scoreboard Key:
   "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
   "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
   "C" Closing connection, "L" Logging, "G" Gracefully finishing,
   "I" Idle cleanup of worker, "." Open slot with no current process

[chris]

I ready many pages.
Possible solutions are change the apache MPM Prefork
to MPM Worker which uses threads to handle request.
or even MPM Event which is more optimize,and can handle even more request.

But it is a new install and you need an update of php, which might breaks things.
So you should test it before putting in production.

Another option is installing a web-application firewall (WAF).
OWASP ModSecurity, shadow daemon, ...
Smart, should do it, but also a new install and need to be configured, so takes time...

[chris]

I think I found a solution.

In /etc/apache2/apache2.conf
I changed Timeout from 300 to 100

And there is a module reqtimeout
If it is not yet enabled you should enable it:

Code: Select all

sudo a2enmod reqtimeout

edit /etc/apache2/mods-enabled/reqtimeout.conf

Code: Select all

# Wait max 20 seconds for the first byte of the request line+headers
# From then, require a minimum data rate of 500 bytes/s, but don't
# wait longer than 40 seconds in total.
# Note: Lower timeouts may make sense on non-ssl virtual hosts but can
# cause problem with ssl enabled virtual hosts: This timeout includes
# the time a browser may need to fetch the CRL for the certificate. If
# the CRL server is not reachable, it may take more than 10 seconds
# until the browser gives up.
RequestReadTimeout header=10-40,minrate=500

# Wait max 10 seconds for the first byte of the request body (if any)
# From then, require a minimum data rate of 500 bytes/s
RequestReadTimeout body=10,minrate=500

You can change the values, i changed the RequestReadTimeout header from 20-40 to 10-40.

and reload apache :

Code: Select all

sudo systemctl reload apache2

For now it solved my problem, but I will keep monitoring.
the workers are now not in all in R ( Reading Request )