Hi guys. Over the last days I see on the logs a few lines like the one below. Do you have any idea what is going on ?



The ip belongs to google !

Thank you for the question. The kernel is just reporting that an application exceed a limit you have configured on your system or that the application set for itself. ASL doesnt set these limits, neither does the kernel, the ASL kernel just has the added capability of reporting when a limited has been exceeded. A vanilla kernel does not have the ability to log this, so you wouldn't know this was happening.

So, short answer, a limit you have configured on your system has been exceeded. Either because of a problem with an application wrongly exceeding them, or your limits (or the applications) are too low.

The default limit most vendors set for the stack (with any kernel) is 8MB. You may want to find out why you are overstepping the default limits. Odds are something is wrong with whatever google is accessing (a buggy web app for example) and its cascading out of control, eating up the stack and being killed off so it wont kill the system (and that action is being logged not caused by the kernel). A stack limit being exceeded by apache is a pretty strange thing to happen, so I'd look at your web logs to see if you have bigger issues like its segfaulting, or something is blinding going outside the stack.

In short, this is the application killing itself, and the kernel is just logging it (not causing it). So theres not anything we can do, its an application error with apache. If I had to guess, you probably have a signal 11 (segfault) happening and thats triggering the RLIMIT_STACK limit.

I see. Well this is good since the system is protecting it self from crashing by killing this process.
I searched the default apache error logs and didn't find anything relevant.
Probably the error has been logged on the error log of a virtualhost which makes impossible for me to find.

No segfaults? If not then its something even more needlesome. Sorry I can't tell you whats causing it, at least you know the time and that its apache. So if it becomes a real issue, you can narrow it down in the apache logs by time stamp - that might be a bit of a chore, but you could do a find -exec grep "time stamp" on your access_logs. It might take some time to process, but eventually you'd find it.

It might be a bug in apache or a supporting library/module, so if you havent already make sure all your patches are installed.

Ok. I now get all the time segfaults in the emails coming from ossec.

I took a look at the error and I found the following. This happens many times.



So when they attack apache gets a seg fault ? Is that normal ?

Hours after an attack? No.

[Sun Nov 06 19:28:12 2011] [error] [client ip.ip.ip.ip] ModSecurity: [file "/etc/httpd/modsecurity.d/20_asl_useragents.conf"] [line "102"] [id "330034"] [rev "8"] [msg "Atomicorp.com WAF Rules: Vulnerability Scanner User agent detected"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Pattern match "(?:n(?:-stealth|sauditor|essus|ikto)|b(?:lack ?widow|rutus|ilbo)|web(?:inspec|roo)t|p(?:mafind|aros)|cgichk|jaascois|\\\\.nasl|metis|webtrends security analyzer|zmeu|w3af\\\\.sourceforge\\\\.net)" at REQUEST_HEADERS:User-Agent. [hostname "ip.ip.ip.ip"] [uri "/MyAdmin/scripts/setup.php"] [unique_id "8GEHF1jGf7IAABsKTe0AAAAg"]
[Mon Nov 07 00:30:01 2011] [error] [client ip.ip.ip.ip] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /w00tw00t.at.ISC.SANS.test0:)
[Mon Nov 07 00:49:06 2011] [error] [client ip.ip.ip.ip] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /w00tw00t.at.ISC.SANS.DFind:)
[Mon Nov 07 02:14:21 2011] [notice] child pid 22728 exit signal Segmentation fault (11)

Those events happened hours before your segfault, so its safe to say they have nothing to do with each other. If you want to find the cause of a segfault, please see the link below. Apaches logs are useless for finding the cause of a segfault.

https://www.atomicorp.com/wiki/index.php/Apache