One of the larger efforts for the ASL 3.0 series is to include a more robust vulnerability detection system. With ASL 2.x we focused on more implementation specific vulnerability detection, an area we felt was (and still is!) underserved with standard vulnerability scanning technology. Thats a discussion for another day I think.

Anyway, with the increase in compliance based auditing like PCI DSS we felt it was a good time to start growing out the vulnerability scanner capabilities the same way those outfits use them.  Basically those audits come in two forms, network based scans using tools like Nessus, or commercial web application scanners from companies like HP. The former you may have heard of in the past when it was an open source project. Nessus became a closed source effort with version 3.0 I believe, and during that time the project was forked by the OpenVAS team backed by Greenbone Networks.

Since then the project has continued to be developed heavily, including all sorts of excellent features including a web application vulnerability scanner. This will prove to be an excellent addition to ASL 3.x, given that for the most part ASL is dominated by a web application userbase.

OpenVAS itself is a fairly large project, including native X11 clients, dedicated service daemons for scanning, user administration, management, cli's and now a web based front end. Also given the heavy development schedule it as proved to be an extremely challenging packaging environment given both its complexity and requirements for components not normally found in the enterprise class operating systems.

For us, that means nailing down all of those little integration issues (and some are far from little!) before we can get it integrated into ASL. The good news is that enough of that has been done so far that you can try it out from the atomic channel today. My recommendation for getting the most out of it right away would be to use Fedora 13 as a desktop, since thats what we're using to nail down all the bugs first. Barring that you should be able to run this via web mode once the packages for that are done.

Step 1) Set up the atomic repo:

wget -q -O - http://www.atomicorp.com/installers/atomic |sh

Step 2) Install openvas

yum install openvas-scanner openvas-client

Step 3) Create your user account:

openvas-adduser

Step 4) Launch the openvas client from Applications->Internet->Openvas Client

OSSEC is under heavy development upstream, and we've been helping them out where we can. This week it was getting into the malware detection database / updating the rootkit lists, and today nailing down some issues with OSSEC (and ASL) in agent mode. The current 2.4.1 builds dont handle restarts/reloads if they're deployed as an agent.

Due to the way OSSEC is built we have to make some changes  in order to package both the client and server components. As ASL 2.x has been primarily single-server centric,  we focused more on the server side which meant some features were not fully implemented in  agent mode. That has fortunately been finished now (in version 2.4.1-10), but it leads into the next problem we're waiting on upstream to complete.

OSSEC has merged in cdb database support for datasets, like rules. This is important because it allows you to modify ossec operationally, without having to restart the daemon. The down side is that since we have to work with snapshots for other features we're developing (inotify, etc) other components in 2.4.1-9+ such as the mysql connector are presently non-functional. This will manifest in an error message indicating that rules cannot be found when os_dbd is loaded.

Not to worry, upstream has assured everyone that the changes to complete the os_dbd changes are forthcoming.  I plan on getting that change worked into 2.4.1-11 as soon as it is available.

Today was all about nailing down the 2.6.32.19 kernel update. Upstream** made quite a few changes that believe it or not were effecting ioncube loader from the kernel side. I know I mentioned this before, but this is exactly why building community packages pays off for our security products. Its like the ultimate QA process smoke test.

Anyway, that was cleared up as well as some of the internal changes that need to be made to support dracut in RHEL/CentOS 6, and Fedora 11+. Leaving only one of the sillier problems to deal with, which is how to version it. I'm sure you've probably noticed other packages using "<package name>-<version>-<release>.art.el5.<arch>.rpm" by now, with the "el5" part indicating the distribution it was built under. Until now we've only ever had to build one kernel for all platforms, and we could get away with simple basic naming conventions. Furthermore kernels are more or less agnostic to whatever you have installed in userland, it doesn't matter if you build the kernel on fedora 13 and run it on centos 5. Until now that is...  still pondering the most elegant way to do it. Presumably someone will ask either internally here, or externally so I'll list why we won't be doing it per distro:

• Per distro kernel builds don't work.
• If they did work, they would take 15 times longer than they do now.
• That would equal about 2000 minutes total, or about 33 hours (assuming nothing goes wrong).
• Per distro kernel builds don't work.

Which leads me to other updates, since kernel builds do take so long even with the current design I took a diversion into implementing FPM in the PHP 5.3.3 packages. I looked around fedora to see if they had tried to do this yet without any success. Perhaps its even too new for them. At any rate, the good news is that it was possible to implement as a module for CentOS/RHEL 5 called php-fpm. The bad news is it will not be possible to support this with CentOS/RHEL 4, there are simply too many changes that would need to be made in other packages. There were no issues getting this implemented in Fedora 11, 12, and 13.  We still have no method of evaluating the implementation of this package, so input on what works with this is just as important as knowing what doesn't.

If you'd like to give FPM a shot, you can install it with:

yum --enablerepo=atomic-testing install php-fpm

** Upstream is packager-speak for whoever owns the project code you're working with. Linus Torvalds, Mysql.com, php.net, etc.

In an ongoing effort to reduce overall ASL complexity, we've been working to expand the capabilities of one part or another where they overlap. This summer I gave our intern the project (among others!) to see if he could merge the rkhunter rootkit signatures into the ossec rootkit database.  I'm happy to report that not only did that project succeed, but that we also have a great utility to handle synchronizing rules between the two projects going forward.

So, on to why this is important. For starters  rkhunter is a great project by Michael Boelen, written entirely in shell. As a long time unix guru I thought I had seen everything that could possibly be done in shell before I saw rkhunter first hand (and check out his other project lynis). Its a poem. If you need a shell example cheat sheet, rkhunter is it.

Daniel Cid the author of OSSEC mentioned once that rkhunter was part of his inspiration for his project, but that he needed something more client/server centric to handle a large number of machines.  OSSEC has too many traits to appreciate in this narrative, so I will digress to the bullet form:

• Client/Server system
• Trigger events based on alerts, ie: malware detected in /path/to/file
• Alert suppression, probably not a good idea when we detect the system has been owned. That being said, rkhunter will harass you daily. OSSEC will not :P
• Whitelisting, if for example there is a false positive. You can keep yourself from being locked out.
• The analysis also allows us to look at running processes. Perhaps we can combine file checks with some more advanced process based meta rules. We're still looking into this.
• Ossec supports inotify**

** This is the real reason. Inotify is a kernel subsystem that allows you to trigger an event based on a change to the file system. It moves malware detection into the realm of real-time analysis. Whereas rkhunter is a tool that scans based on a direct userland event (cron, running it manually, etc), inotify events are immediate. This means that we only look at the specific file when it changes, resulting in both a vast reduction in overhead  and an immediate response.

So what does this mean for rkhunter in ASL? For now business as usual, but if we can duplicate the capabilities completely in OSSEC then it would eliminate that in our design.

I'm also happy to report that the OSSEC project has accepted our rootkit updates, and it will be showing up in future releases.