Podcast: What is OSSEC and Why People Use It
OSSEC, which is short for open source security, was founded in 2004. It is an open source project for cybersecurity and delivers the most robust endpoint detection and response (EDR) capabilities available to enterprises today. Scott Shinn, OSSEC project manager, introduced its most recent update to 3.0 at the OSSEC conference this past April.
In this week’s episode of the Linux Security Podcast, he goes into further detail. He discusses the history of the project, why companies from Apple to Netflix choose to work with OSSEC and how it differs from commercial security software currently in use. He also goes into detail about the five key areas of OSSEC, from log-based intrusion detection and file integrity monitoring to active response, and why he chose to be a part of the project. To listen, click below.
Atomicorp provides unified workload security for cloud, data center or hybrid platforms. Built on OSSEC, the World’s Leading Open Source Server Protection Platform. See our products.
What is OSSEC and Why People Use It – Podcast Transcript
Bret Kinsella: [00:00:00] Episode 7 of the Linux Security Podcast. Today’s topic the open source project OSSEC.
Bret Kinsella: [00:00:16] Welcome back to the Linux Security Podcast. I’m here today with Scott Shinn. CTO of Atomicorp. He is the author of Troubleshooting Linux Security Firewalls. He was a former interim ISSO at the Department of Interior. He is more than 20 years in the security industry. And today we’re going to talk about OSSEC. And Scott also holds the distinction of being the project manager for about the last four years of the OSSEC open source project. So I thought the best place to start, Scott, would be to tell the listeners what is OSSEC and why do people use it.
Scott Shinn: [00:00:50] So OSSEC which is I found out recently having been involved with this project for over 15 years a short for open source security and I didn’t know that. But OSSEC is an open source host based intrusion detection system. And what that means is we’ve got an open source platform that runs on literally everything, Linux, Windows, runs on AX and Solaris and HQ X and even weird stuff like TiVo. What it brings to the table is the five areas that I think are key for anything that’s host based that’s log based intrusion detection, File Integrity Monitoring, auditing, malware detection and active response.
Bret Kinsella: [00:01:32] Okay. So who’s using this today?
Scott Shinn: [00:01:36] So it’s used by… I am constantly surprised by how many it is used by… I know probably one of their biggest deployments out there that I’ve been told of is Apple. But certainly Netflix and and Facebook at one point I believe was using it, in financial industry and certainly in the government. This is one of those open source projects that is a security product specifically that is super ubiquitous like I found it in every industry. You know railroads and finance and medical systems and you know governments. So it’s it’s just really, really popular I think.
Bret Kinsella: [00:02:14] So why is it so broadly used?
Scott Shinn: [00:02:16] I think probably the key part of this project is it. It sort of fits in that realm of ease of deployment and how much value it brings back quickly. When I first got involved in this project, I was a developer on an open source SIEM project and I remember working constantly on building a great SIEM for myself and I was putting an hour after hour day after day and getting you know was for me fun results. But in the end it was it was a lot of a lot of heavy lifting to implement. And when I got introduced to the second project I remember it took me like an hour the first time I installed it. This is 15 years ago we had to compile it and everything in about an hour I was able to get really good like… really useful capability like in an hour and I just thought “Man this is this is the project for me” because I did that intersection of utility and capability and maturity. Here we have 15 years later but even back then it was pretty mature so it was… I had to ask myself what did I want to do today. On a weekend I want to spend the next eight hours tinkering with this SIEM project or do I want to go do something else. Right. So, that was sort of sort of where it came from whenever that epiphany hit me right.
Bret Kinsella: [00:03:43] And over time it’s become even easier to install and it’s actually you do you and the other contributors to the project to understand have created connectors to all sorts of other systems.
Scott Shinn: [00:03:55] It’s gotten really… I mean it’s there is a lot of community support behind it. A lot of a lot of those rough edges that and they weren’t even that rough in 2005 and I got involved with this but that whole process has gotten down to the point where we’ve seen these things get deployed on thousands of systems in a day. You know and it could scale easily much higher than that but it it’s like I said a lot of people that have contributed changes to this to make it really, really easy… even easier than it was before to deploy and to use it. For example we’ve got this down to like one one line you could install this a system in one line whether it’s a Linux system or Windows system or whatever.
Bret Kinsella: [00:04:38] And you know if we if we look at it in terms of how people use it, are people using all the five areas that you mentioned or most people using one area or two… two of them. How is it implemented in the field?
Scott Shinn: [00:04:53] I’d say the most people focus on just two areas in it when they’re deploying it you know and in scale and that would be a log based intrusion detection component and then the file integrity monitoring component. The other three I think they don’t get they don’t get enough. I think they’re awesome. I don’t think they get enough. I don’t know people really appreciate how much they bring to the table. But you know in in many cases this is software that gets deployed in enterprise right. And so usually an enterprise is trying to speak to a specific problem or trying to solve whether that’s analyzing logs or implementing File Integrity Monitoring. And I get that. That’s where they’re their pain is to solve it. When you’re talking about something like active response that’s something that can change a system. Right. It’s going to change firewalls. It’s going to disable user accounts it’s going to do self healing. I understand the reluctance of somebody who’s implementing something like this that maybe they don’t they’re not positioned to use that or feel comfortable using that whereas me I’m a paranoid security guy that likes automation. I’m all about making this defend itself. So I don’t have to spend my time you know effecting those changes myself but that’s a different. You know I think it’s a different use case. It’s just so.
Bret Kinsella: [00:06:16] So most of the users in your opinion or in your experience are using this for the detection capabilities as opposed to the prevention protection capability.
Scott Shinn: [00:06:25] That’s that’s exactly right. Yeah I mean there are there are a lot of pressures I think, regulatory pressures for those kinds of those kinds of capabilities that have a direct… it’s very easy to measure the impact of not doing it right. If you’re not doing some of these things you might either get fees in a case of like processing card industry. It’s really easy to say what was the cost of not doing this. Well it was you know 2 percent of every transaction. OK. That’s easy right. Right. Versus active response. What’s the cost of doing active response and not doing it. I think there’s a lot more tolerance for it you know for a compromise sadly than there is for you know creating an outage based on something that’s made a change. Right. On a system. So.
Bret Kinsella: [00:07:17] So that’s the risk of active response is that it blocks something that you would want to have come through.
Scott Shinn: [00:07:22] Yeah. And there could be up there could there’s far more I think there could be in an organization if you’re you’re running you know you’re a security engineer and there’s far more social impact to you for that versus as an organization that you’re dealing with a compromise than it’s you know the impact socially on you is blunted by the fact that you’re just one of many in the organization. Right. So that that I appreciate everybody who stuck with that situation. I think you should use all five. But usually in a big deployment it’s you know logs or FIM and sometimes both.
Bret Kinsella: [00:07:56] Well the detection gambit right is it part of your security strategy is universal now like everybody… there’s a big focus on right.
Scott Shinn: [00:08:07] And there’s no risk of detection like there’s no risk that you’re going to expose yourself through detection of causing like an impact on the organization. All you’re doing is analyzing.
Bret Kinsella: [00:08:18] But the return is really much higher on the prevention side, the protection side because you’re actually eliminating threats before they take hold in the enterprise.
Scott Shinn: [00:08:27] Yeah. And you’re getting out ahead of them and in that regard I mean there’s certain there’s certain threats that that you know they move so quickly that no human being could ever respond to them and you know a detection model is you know reactive. Right I mean so there’s no way you can use a detection strategy on anything other than really effectively it’s a tool of Incident Response. Right. Now you’ve identified an issue. Now you were going to put a human being in the process of doing something about it and if you’re dealing with a really, really rapid attack and I think that’s just pretty common these days when you’re dealing with either a malware outbreak or an extortion scam or something like that I mean you just don’t know no human being could ever possibly scale to something on that and you’re you know people just can’t fight automation right.
Bret Kinsella: [00:09:21] That’s right. But seems like the pendulum is starting to swing to an all the above strategy where we’ve had this era of detection that really followed an era of protection. Right. So we really focused on protecting systems for a while and then more recently it’s been on detection you know Gartner talks about EDR capabilities and those types of things. It seems like more and more people are now saying well I need to have a nice balance here of detection and protection and response. And I need to use all of them.
Scott Shinn: [00:09:50] I also think that the nature of computing in general has it shifting a lot where you know before we had a model of you know self-contained data centers and perimeters where we can establish where the perimeter is. Right. We can we can say that this is the edge of my network you know all of my data is behind you know these enclaves, these firewalls these DMZs and now it’s a lot more spread out. So you know even we’re way past just pure cloud computing and Amazon.
Scott Shinn: [00:10:18] We’re now into things like Salesforce and Slack and all these great tools… you know… your data now doesn’t live behind that on cloud. It lives up in G-suite. It now lives up in these other components all of your sensitive stuff is out in these environments and these parameters have really changed. Right. And so when you’re dealing with something that maybe is an environment without a perimeter, the pure detection model, it’s made it even harder. Right. It’s made a lot harder. Like you already didn’t have people to do this and now you’ve made it even harder. So the people that you do have are now even working longer days, longer hours getting more worn out, more alert fatigue that kind of thing to deal with a problem that just continues to get more and more complicated.
Bret Kinsella: [00:11:09] Ok. So for OSSEC the active response capabilities in particular create automated protection, maybe relieve the burden on the analysts. They can focus on the things that really need analysts attention. Let’s talk just quickly about the detection pieces. So I understand a lot of people are starting to use or have been using OSSEC for FIM. And so why is it a good tool for that compared to things that people already familiar with like TripWire the market.
Scott Shinn: [00:11:37] So I think it get again it gets back to the nature of the speed more than anything else that OSSEC is a real time FIM and so other tools that are out there that are referenced are like aid or Trip Wire or even a really old one called cops. These were scheduled scans. So they would detect changes 24 hours, a week after they’ve happened. And so we’ve put in place probably about 10 years ago the ability to do real time changes and so in that regard you do have real time notification. You can say this specific file maybe a key file has changed and you can notify somebody that’s happened immediately. Personally I’d like to marry that with active response. So if something is changed I wanted to do something when it changes and that could be… that might just be notification or that might be something a little more specific like “hey you know what. Maybe we shut the service down”. Usually when you’re making a change on the system I would hope that you know what it’s going to be when you’re going to do it and if it’s outside of a window you would want to arrest that in its tracks. And that’s where I would use the two together.
Scott Shinn: [00:12:54] The other half of that is that tools the traditional tools in that space of FIM just tell you that I’ve changed right. They don’t tell you what changed or who changed it. And in both regards we’re in a position to be able to say this is exactly what changed. You know it’s a text file. We can say it’s you know these 10 lines changed and then we can also marry that with data to say well this is you know the process I.D. whatever that actually made the change.
Bret Kinsella: [00:13:21] So for File Integrity Monitoring it ostensibly has all the capabilities of the traditional if I am tools like TripWire. But it has more because it does the real time change, it tells you what the change was, you can tie to active response. Is that the right way to think about it?
Scott Shinn: [00:13:37] That’s exactly right. Yeah that’s exactly right. Let’s start gluing these siloed pieces like hey let’s do detect changes with something that can do something about it or some place where we can add extra value right. I mean we can we can say this file changed and then we can take that change file and put it in some kind of analyzer. Right. And you could easily… if its source code your change could trigger a source code analysis piece if it’s a you know a part of a website or maybe a file on there you can run it through some sort of malware scanning engine or whatnot so we’re definitely moving past a reactive instant response model of saying something changed and I think a lot of those those players in that space have always just said “Well our job is just to tell you that it changed and then I’m going to leave it up to you to figure out what to do with it.” And I was never satisfied with that and neither was anybody else on the project. We always thought you know how can we do this. You know in a much more advanced way.
Bret Kinsella: [00:14:34] Makes total sense. So the other aspect you talked about on the detection side was the log-base intrusion detection. So most people think of log aggregation, they’re thinking of SIEM. So how is OSEC similar or different from SIEM.
Scott Shinn: [00:14:49] So I think it’s it is it fits very much in a in a big part of what the true value of a SIEM is which is which is to take a whole bunch of data and I call it fish guts and turn that those fish guts into filet mignon. Right. So the way that SIEMS do this traditionally is you know you have a framework. Right. It’s usually some sort of graphical framework that allows you to build rules and tie a bunch of data from all these disparate parts together to analyze it. And come up with the with some sort of a result. So that’s sort of a before how I got involved with this I was working on tools that did that. And what I found in that space is that the expectation was its framework and you need to write. You need to create the logic around it and they made they made the use of generic stuff. But when I found was OSSEC what I found immediately was that we could get that GUI framework that which is fantastic and cool to play with is really necessary for for getting most of the way there. You would you really needed was that engine to be able to take all this and just all this data turned into some kind of useful object and then perform analysis on it. And the IDS world we have a way we break this down into atomic events… which is great because it’s also named our company… single atomic… meaning single… and composite events where we take multiple atomic events and we turn it into something filet mignon. Right. So multiple atomic events add up to like a brute force attack for compliance out of compliance or whatever. And I always wanted to be able to implement that quickly and not have to write a ton of logic around it to create that kind of stuff. The irony of course is a developer in the project guess what I do all the time. I write that logic right. So you know I didn’t really achieve that goal for myself but I did find I think a much faster, better, cheaper and approachable way to do this.
Bret Kinsella: [00:17:05] We created it for other people. I created for other people. So you’ve done it so they don’t have to.
Scott Shinn: [00:17:09] Exactly right. Right.
Bret Kinsella: [00:17:10] And so does OSSEC from a log standpoint does it replaces a SIEM? Does it work with a SIEM? How do people marry the two up if at all?
Scott Shinn: [00:17:23] So it absolutely works with SIEMs there’s no there’s no reason why you couldn’t use this with a SIEM and leverage things that a SIEM can do that. That that it’s really good at right. You know visualization reporting you know SIEMs will also tie into tools that are not that are that are that are like more proactive like vulnerability scanners. Right. So you know that is a or network management tools. Right. So those are those are all great pieces. There are fantastic reasons to use OSSEC with the SIEM mainly because it reduces a huge amount of just the noise that ends up in this. Right. When you’re running in anything through an IDS system it’s turning atomic events into composite events. And that is by virtue of the fact that I’ve turned 15 events into one event, I’m massively reducing what goes up to the analyst in the unit I’ve created it a robot so to speak to do to be the analyst I now the analyst is working with much better data. So adding it to SIEM is absolutely beneficial in every way shape and form it just dropping random junk in there.
Bret Kinsella: [00:18:34] Well sounds like it would increase the efficiency of the SOC analysts.
Scott Shinn: [00:18:36] Absolutely. And I know there are plenty of people I know that are getting rid of SIMs and they’re leveraging tools like this. And I know there are a great many visualization tools out there that are just fantastic that are that are great at it. So for that matter I see those exact same tools they’re used with just raw sort of OSSEC IDS events also being used by people who run SIEM software. Right. So you might be running ArcSight or key rate or any vault or something with that but you’ll probably also be using some kind of big data visualization platform like elastic search for the analysts. So now even the analysts are using the SIEM for the analyst part they’re using these other tools.
[00:19:17] So I just I see this as a you know as it is is a great way to make a SIEM better and if the same is not meeting your goals. I have a friend of mine who’s a CSO at an energy company and he told me the reason why he got rid of their SIEM was because he felt that his engineers relied on it too much. So if they didn’t find problem in the SIEM, you know well didn’t exist. And you know him from his perspective he’s performance driven right. So he didn’t care if it wasn’t in the same. He just wants to know if something bad was going on. And I appreciate that I mean you know as an engineer if I’ve got a tool that’s doing that I think right maybe I’m not. Maybe I’m not the guy who built the framework. I think it’s right. And I’m working with it every day and it’s making pretty pictures and charts and stuff for me then sure it’s so much easier to communicate with a pretty picture and a chart than it is with a great big text report. But that said you know I’m with them on that. But those tools abstract away a lot of information that might lead you to a false sense of security.
Bret Kinsella: [00:20:32] Yeah. Fair enough. So I guess the bottom line there is OSSEC can work with a SIEM to compliment it and make your SIEM more effective in your SOC more effective. It could potentially in some instances replace it. But OSSEC overall is being used by a lot of companies. And it has a lot of capabilities so if listeners to this podcast want to get involved in the OSSEC open source project learn more about it. Where do they go?
Scott Shinn: [00:20:59] We are on GitHub so you can just look for us as O S S E C… OSSEC on github and we have a Slack channel. It’s OSSEC.slack.com
Bret Kinsella: [00:21:12] Great. Thanks a lot Scott.
Scott Shinn: [00:21:14] Thank you.
Atomicorp provides unified workload security for cloud, data center or hybrid platforms. Built on OSSEC, the World’s Leading Open Source Server Protection Platform. See our products.