So there was I thinking "no more bus errors! yay!", as I'd not seen one sent to me by ossec in ages and ages. But they are still happening:



I'm just not getting anything from ossec.

I've checked via the gui, going down to Level 2, and there's nothing in the ASL log either.

In the rule manager, I see that rule 99104 should be doing the trick, is enabled, is Level 14, should log and should email, but it isn't.

I did a quick grep -ri '99104' /var/ossec/etc/rules.d and assuming my syntax was correct, it found nothing. I manually checked 50_asl_apache_rules.xml in that directory and did not find the rule in there, nor in general, nor syslog.

Has it gone missing or am I just looking in the wrong place?

Faris.

Is that log file being monitored by OSSEC?

[EDIT: OK. Didn't read your post correctly....]

I don't know if OSSSEC is looking or not. I would assume so. it is /var/log/apache/error_log
Where are such things defined?

I know it used to be monitored and logged. I used to get regular reports. But no longer.

I have fiddled with certain rules via the rule manager and via local_rules, but I've checked and 99104 is at defaults, reporting and emailing, and my local rules are specifically targeted at some qmail/spamassassin errors.

I just can't find rule 99104 in the actual rule files themselves, which is why I was wondering if it might be missing in action unless I'm looking in the wrong place again.

I just looked, we dont have a rule id 99104. Maybe its something 3rd party/custom youre using?

I was suspcious of the rule number being in the 99 range too. But my custom rules are all in local_rules and have 100 ranges.

I'm not saying it isn't, just saying...

Does the GUI's rule manager get the rule IDs from reading the actual rules then? I don't know how else a custom rule like that would have ended up in it? But if it knows about it, where is it, I wonder?

In any case, as this is an important rule, I'm going to raise a feature request for it in the portal.

For hids rules its all dynamic, the database is updated every time the dbd daemon is restarted. Just hit he "Report false negative" button whenever you have a rule that needs more defining (generally rule id 1002). No need to narrate or anything, its pretty clear most of the time when those come in.

Does that imply that there should be a rule 99104 *somewhere* on my system (and it is just misconfigured and doesn't trigger)?

[I presume the dbd daemon gets restarted when there is an update to the ossec rules, or an asl (minor) version update, and possibly more frequently than that?]

Thats not a rule ID (99104) we use. We dont have anything even close to that high. All our apache rules are in the 3xxxx set. We dont anything for anything in the 9xxxx range.

So that rule wouldnt be one of ours. Anyway, so this is the log event you have:



No there was not a rule for that, but I just added one.

As an aside, for anyone else that reads this thread that error is a different critter from a segfault (just in case anyone is wondering), normally a bus error happens if someone has compiled something for one platform and tries to run it on another. A bad module for example, a CGI, etc. A segfault is a totally different beast, and means you have a memory problem.

Anyway, I just added in a rule for the apache bus error messages as a level 8. If you have any others you want added, just use the False Negative option in the GUI. Thats the best way to let us know. That sends all the debug info we need to see all the event details so we can add in a rule - plus it goes right into the support system so it gets faster attention.

Sorry mike -- I'm always afraid to click on a button without knowing what it does. I misunderstood what you were saying earlier about the use of it.

Because the False Positive button simply sends everything with no further input, I was afriad to go anywhere near the false negative button. I still am, to be honest!

I'm fascinated with this Bus error v segfault difference. Bottom line is that I get bus errors under the same circumstances that I used to get segfaults (and glibc errors), i.e. if I enable "too many" mod_sec rules, I'll get bus errors. The more rules I enable, the more bus errors I get. This change happened about a year ago after a kernel update. I posted about it here: viewtopic.php?f=1&t=4538

This whole thing may just be a peculiarity of Virtuozzo/OpenVZ, however.

Just a reminder - we know it isn't mod_sec or the actual rules that's the root cause. Segfaults were narrowed down to be a problem with apr, I think? But in our case the segfaults/bus errors are most likely some wierd combination or interaction between VZ, php, apache, apr, and for all I know Zend Accelerator/Ioncobe/Suhosin, or more specifically the versions of those that we happen to run, plus some badly written/buggy php/cgi script or other being run either by us or by one of our customers.

And as we can't run mod_whatkilledme or similar tools, I guess we'll never know

Faris.

Actually you can know, just configure core dumps and run a backtrace. You'll see exactly whats causing the bus error:

https://www.atomicorp.com/wiki/index.php/Apache

Nah, been there done that. No httpd-debuginfo for the Centos 4 *centosplus* httpd.

Having said that, I seem to recall someone else had a similar problem recently and got round it - it was in some other package or something?

if you grabbed their src.rpm and rebuilt the package it should** give you a -debuginfo- by default.

** Unless they broke it.

Centos 4 is EOL in a few months and we'll have moved to Centos 6 by then. I think we'll put this on the back burner.