Always Check the Spark Plug and Other Lessons in Cybersecurity
Rule 1: First check the spark plugs! It’s a lesson my brother, Scott, and I learned as young men decades ago in high school. It’s something we even painted on the wall of our parents’ garage. And it’s an idea that’s been applicable in our work in software development and cybersecurity ever since.
The principle of checking the spark plugs is simple: don’t overthink the problem, start with simple things first. Back then, like many youths in the 1980s, we bought used (sometimes very used) cars. Although inexpensive, these used cars came with maintenance problems galore that we had to fix ourselves.
A lot of things can go wrong with a car, but it turns out that most of them aren’t that complicated. What’s important is the vast majority of the things that can go wrong take very little time to check. Spark plugs were a common one.
That fateful day when we created the rule, we’d torn a carburetor apart, and rebuilt it, messed with the timing, changed the jets (again), swapped out a fuel pump, and made countless other changes all to no avail. We went back to square one.
What about the gap on the spark plugs we’d put in? We pulled one out, checked the gap and sure enough it was way way off. A few minutes and 8 re-gapped plugs later, we had one 1972 GTO running like a champ.
We’d overthought the problem, and wasted a lot of time not enjoying our cars! We needed a problem-solving process. Start simple, work to complex later. To make the point we memorialized this on the garage wall: “Rule 1 of Wrenching: always check the spark plugs!”
It turns out this idea isn’t really new. Occam’s razor (also called the law of parsimony and attributed to William of Ockham), overly simplified, is when you have multiple explanations for the same thing, the one with the least assumptions is probably the right one. When applied to problem solving, start with simple explanations, and rule them out first. Don’t move onto complex problems until you do that.
Without a process like this, the practice of cybersecurity can involve similar guesswork and lead to failed detection, SIEM infoglut, too many response tickets, paralysis, and vulnerability to that knockout blow. I recently had the opportunity to discuss these and other cybersecurity challenges with Joe Saunders, CEO of RunSafe Security, and host of the podcast series Lessons from the School of Cyber Hard Knocks.
View the Podcast Featuring:
Michael Shinn, CEO, Atomicorp
Joe Saunders, CEO RunSafe Security
Joe asked me a number of great questions about compliance and security, and I want to reinforce three big takeaways from the interview.
1) The need for speed
One of the toughest challenges on the cyber battlefield is the ability of security software updates to be able to keep up with both business-oriented rapid software development and the rapid pace at which adversaries find new vulnerabilities to exploit. You have got to get these new products and features ready for your customers and to keep pace with your competitors, while quickly finding vulnerabilities, then developing and releasing patches, in line with customer expectations where things ship really fast (and continue to work properly). All while staying ahead of the bad guys.
The resulting reality is that no matter how hard we try, we won’t always be able to keep up through our best efforts… vulnerabilities will still be there. Every piece of software and hardware almost certainly has some unknown, undiscovered vulnerability, probably many of them. The best we can do is to produce software that has less vulnerabilities in it, and build ‘defense in depth’ into whatever computer architecture we’re deploying. Assume the worst case: there’s a critical vulnerability there and you won’t patch it in time. So now what? Build in defense-in-depth, layered independent obstacles and other defense mechanisms to disrupt the attack chain. If the bad guy gets on the system, you need to detect that, stop the spread and escalation, remove the malware or malicious presence, and so on. You can’t patch your way out of it.
This detection and response capability must be active everywhere, from your endpoints to the cloud. An open source cloud workload protection platform like Atomic OSSEC empowers organizations to apply cybersecurity everywhere. From your endpoints, to your legacy systems, and into environments like AWS, containerized environments, Kubernetes, OpenShift, GCP, and Azure, you’ll be able to engineer security into all your systems and containers wherever and whenever they are deployed.
So how do you ensure security in this rapid DevOps environment? Even perfect software runs the risk of the environment being misconfigured or the password being set as easy as 12345. That’s not even a bug, that’s a configuration issue. So everything is collapsing into this thing called DevSecOps, with operations and development and security getting together as a team instead of separate groups doing their little part. It’s all got to be a whole. The sooner ideas are transformed into running code, the faster a business can see the results. But the reality is there is still a lot of specialization, especially in larger organizations. For example, one group is doing network security and another host-based security, which isn’t really an integrated team. So you have to figure out a way to knock out some of these vulnerabilities as code ships quickly.
IT and OT teams need more than knowledge, they need aptitude, which is further reason to tear these siloes down, and bring your security people into the development process, because most developers aren’t cybersecurity people.
2) Being compliant is not the same thing as being secure
Security and compliance are often talked about together, but there is an important difference between the two.
Compliance is about meeting a regulation or standard (similar to putting a fence around a public water tower in accordance with a statute). Compliance is about meeting the letter of the law. Too often, however, it gets conflated to mean secure, or worse a requirement is too prescriptive, preventing you from using your own creativity and understanding of your unique environments to secure your environment. Understand that compliance is not security. It is its own thing. You will still have to assess your security separately to ensure your security measures are actually effective for your environment and the threats it is exposed to. No external entity will understand that better than you can, because no external entity can know your environment and processes better than you can.
It’s also important to remember that in cybersecurity in particular, we’re faced with defending against other human beings, who are also creative and motivated to defeat any security measures we put in place. The pace of change is rapid, and the limitation of compliance standards and regulatory frameworks is that they will always have trouble keeping up to date with whatever the state of the art is, including new attack methods by bad actors and the latest defense technologies.
Regulations would have to be very nimble and that’s very difficult for government organizations to do. They have to often build consensus, navigate political realities and stay within the limits of their regulatory authority. It may simply be legally impossible for a government entity to dictate all of the things an organization should do to be secure, as some of it may be outside the scope of its regulatory authority. Keep in mind the scope of the framework or regulation you must comply with, and determine if it includes all the risks you would like to protect yourself from.
For example, I really like NIST (National Institute of Standards and Technology), which provides excellent guidance and frameworks. But NIST, as excellent as it is, is still at the mercy of how quickly it can both produce new guidance, and get that published. It will always be playing catch up. I really like performance-based regulations rather than prescriptive ones. Tell the industry participants what the goal is. What is it you need to demonstrate you can do, what they need to protect against, and then allow them to innovate the means to do that. Measure compliance through performance – for example, through things like Pen Tests, aka simulated and realistic test attacks as opposed to just assessing if prescribed controls are implemented as dictated. For example, is it better to have a 10-foot-tall fence, a moat filled with alligators, or a bunch of armed guards to protect a building? That all depends on how and what you’re trying to defend against, your building, and how you operate. If you can stop the bad guys and prevent whatever you want to prevent from happening, or recover from it without causing unacceptable damage, then you’re doing a good job.
Simply checking a box but still getting compromised may mean you aren’t in violation of a standard or regulation, but you’re still going to suffer from the compromise, maybe even catastrophically. Again, compliance and security are not the same thing.
What the difference between compliance and security really boils down to is, if tomorrow your adversary invents a death ray, the organization can’t just point to the regulation and say, “Hey, the regulation didn’t say we had to defend against death rays.” Security, not compliance, embodies imagination, and accounts for and seeks to protect against contingencies that weren’t written into law or industry standard months or even years ago, that weren’t YET imagined, invented or unfurled. But when they are, you must adapt.
3) Go on offense: use imagination in defense
Cyber defense is hard. There’s the old saying that bad guys only have to get it right once, the defenders have to get it right all the time. I don’t think you can be all that successful on the defensive side unless you have experience on the offensive side. And that goes back to the biggest challenge in cybersecurity being a lack of imagination. If you don’t believe in the various threats, it is impossible to stop them, and you’ll be hard-pressed to detect them at all, and very likely you will be overwhelmed when they occur and unable to effectively respond until well after it’s too late. An important, but unsung hero role is being able to get buy-in and money from people in the organization. The tech stuff is super cool, but without resources or buy-in you won’t even be able to start. This is a pretty tough thing to do if purse-holders in the organization don’t understand what the adversary can do. This is where white hat promoters come in, people who are going to be trusted, and have the technical knowledge to understand what is possible. Communicating with and educating leadership is absolutely vital.
The thing that keeps me up at night from a security perspective is how this lack of imagination leaves organizations unprepared for today’s and tomorrow’s attacks. Adversaries aren’t just kids in the basement any more; cyber crime is big business. State actors hacking into U.S. private networks is a real threat. Most recently, the decentralized financial network, the Poly Network, got hit for 500 to 600 million dollars in stolen crypto currencies. And when you start piling on IoT, industrial control systems, and SCADA, you start moving from bits to direct physical effects that will kill people. These are the types of things that keep me up at night. Driverless cars, pacemakers controlled by WiFi – these new vulnerabilities move us from theft to cyberterrorism where you can kill people from a thousand miles away.
Start with the simple stuff, most of the time most problems you’re trying to solve are not super complicated. And then, if that’s not the root of the problem, be able to expertly dig through the coding.
About Atomicorp and Atomic OSSEC
Open source security (OSSEC) is a widely used open source software that’s been around 17 plus years. We run the OSSEC program and include OSSEC in our Atomicorp stack, which includes host-based intrusion detection, file integrity monitoring, malware detection, vulnerability detection, active response, and more. We have a free version of OSSEC and a version with features that midsize and large enterprises have asked us for.
Listen to the entire “Lessons from the School of Cyber Hard Knocks” Podcast.
Learn more about our Web application security solutions, Atomic ModSecurity Rules, as well as Atomic WAF.