|Written by Michael Shinn|
|Wednesday, 27 April 2011 14:06|
Virtual patching is an invaluable tool for immediate remediation to fix vulnerabilities in web applications. Atomic Secured Linux and the Atomicorp.com/Gotroot.com modsecurity rules contain thousands of Virtual Patches which we update everyday.
Sometimes you may need to patch a vulnerability in an application that we are not aware of, such as with a custom application. This paper outlines exactly where and when Virtual Patching is appropriate, how it can be integrated into the Incident Response process, and how it can be integrated into the incident response process, and the proper steps for creating and testing real-world examples.
This article outlines exactly where and when Virtual Patching is appropriate, how it can be integrated into the Incident Response process, and how it can be integrated into the incident response process, and the proper steps for creating and testing real-world examples.
1. Commonsense tip one: Speed!
Don't get bogged down, a virtual patch is about time. You need to get that patch in place ASAP, otherwise, what’s the point?
2. If you can get the exploit, that’s a great way to test the patch for effectiveness - your application might still work, but did you fix the vulnerability?
If you can't get the exploit, you will need a lot of detail about how the attack works. If you discover a new vulnerability yourself, make sure you can test it against your new patch. And if you have nothing, then you will need to know how your app works. Get as much information as you can.
3. Exploit! = Vulnerability.
Just because the exploit didn't work doesn't mean your application is now safe. Try to understand what’s going on with your app. If two variables are vulnerable to a SQL injection attack, you may have a pattern with this in other parts of the application. It might be time to start writing some new patches for other problems. Where there is one hole, there are usually more. So be paranoid. Also, some products share the same code base. Take a look at your apps to see if the share similar variables, directory structures, libraries, etc. I've seen some shared PHP code, for example, that lead to the same remote PHP code inclusion vulnerability in well over 100 applications. Stay alert to the fact that a vulnerability in another application might be the advanced warning you need to patch yours. And, that brings me to the next tip, number 4.
4. Write multiple patches for different things
Don't try to cram it all into one regexp unless it’s easy for you and others to maintain. Yes, if you run thousands and thousands of regexp's you might experience performance issues but modsecurity 2.5.x and higher is very fast so don't get too hung up on performance. Get the rules in place, and don't feel like you have to jam all your patches into one big obfuscated patch. Unless you are a regexp god, or you have serious performance problems, you really should stick with simple regular expressions and multiple rules and patches. There is always a need to evaluate your own organization’s priorities for performance and security. Sometimes performance is more important, other times it is security.
5. If you don't know how to write regular expressions, use a tool or find an expert
If you don't know how to write regular expressions, use tools like RegEx Coach, Expresso and others to help you debug and understand what’s going on with your regexps. Your best bet is to seek the help of someone that does understand regular expressions to make sure your rules are doing what you want.
6. To that end, ‘Keep it Simple!’
Keep your patch simple and work your way up to complex. It’s much harder debugging a 5 line, chained, back referenced, nested regexp than a simple set of patterns. Start simple, and work your way up. Worry about speed, efficiency and perfecting your rule after you get your patch working. So, if it works for you, if your performance is acceptable, and you like it - then it’s a good patch. Don't get caught trying to make your patch "perfect". Make it good enough.
7. Make your patches easy for other people to understand and maintain.
Comment them so someone can know what the patch fixes (or does not fix), give it a Unique ID, version number and revision, and make sure you log when it fires so you can debug if it breaks something. Particularly when trying to go fast, there is a risk of overwriting changes without backing those changes up. Quick and dirty change tracking can be very useful in the process of rapidly developing a defense.
8. When writing a patch, decide what your goal is.
Do you want to define the correct behavior of the application, the behavior of the vulnerability, or both? Remember, only bite off what you can chew. If you don't have the data to define the normal behavior of your app, just stick with the vulnerability. If you don't have any information about the vulnerability, except some vague report of a SQL injection vulnerability in your products search function - it may be time to define the allowed behavior of that function. And sometimes, you need both.
9. The easiest type of patch to write is to define the behavior of the vulnerability.
It’s also a lot less likely to create a false positive and upset your users. The most secure type of patch defines the correct behavior of your application, and in the wild it’s best to do both. So, do both. Always remember, these two types of rules are not mutually exclusive, defense in depth is your friend - write more rules and write them for both cases (positive and negative rules,if you will).
10. Here’s the good news.
The vast majority of unstructured attacks stick with the script, literally. We see a lot of automated attacks that follow the published exploits to the letter. You can get a lot of mileage out of patches that just cover the known exploits for that threat.
(see tip 12 for how you can use this)
For that case, you can write a very effective and powerful patch by simply answering some basic questions:
a) What’s the URL to that app on your box?
b) What variables does it affect?
c) What’s the payload of the attack?
d) What’s the normal payload for the variable?
For instance, you have an app bar.asp, it has a SQL injection hole in the "id" argument that’s triggered with a ', and that “id” only accepts integers. With that basic information, you can write a simple patch for the vulnerability vector, such as:
SecRule REQUEST_URI "$/foo/bar\\.asp^" "chain, id:400000, msg: 'Attack on my app'"
SecRule ARGS:search "'"
And, you can define the normal behavior for the application like this:
SecRule REQUEST_URI "$/foo/bar\\.asp^" "chain, id:400001, msg: 'Attack on my app'"
SecRule ARGS:search "![0-9]+"
And, if you set up a response to this (see tip 11), for instance a firewall rule triggered by OSSEC, you can block this attacker from further mischief.
11. Perfect is the opposite of good enough.
Don't try to make your patch anything other than one that works for you. If it works for you, it’s good enough. If you want to publish it, please do - but don't let that stop you from writing an "ugly" patch. Any patch that works is worth something, anything else is useless. Also, patches do not have to be one size fits all. If you have to tweak the patch for a box that’s otherwise supposed to be identical, tweak it. Worry about why the boxes are not actually identical later.
12. Sometimes, the best you can do is to write a rule that’s just a tripwire.
For example, maybe you can't write a patch that works - it won't stop the attacks, or it breaks your app. What you can do is write some rules that detect the behavior or actions of the attacker before they exploit your app, such as rules that detect a recon probe for your vulnerable application (i.e., multiple attempts to find myphpadmin in multiple directories) or that detect a general type of attack (PHP remote code inclusion).
You can use tripwire rules to trigger other events, like a firewall rule to block your attacker before they can attack your vulnerable application. You can also write tripwires to fire on other vulnerabilities and use that information to block your attacker. For example, the attacker tries to find a phpbb vulnerability, but you aren't running phpbb. That’s fine; just write a quick phpbb rule:
SecRule REQUEST_URI "(posting|users|other_phpbb_apps|etc)\\.php"
It won't hurt your box to detect that; anyone that tries to access anything associated with phpbb gets blocked by your firewall, and now all their other attacks fail (see tip 14). Is this method perfect? No, nothing ever is, but you would be surprised how many new attacks you end up blocking by simply setting some tripwires out there for older ones or general attack patterns, like PHP code inclusion attacks. (See the gotroot.com rules and the modsecurity core rules for examples.)
13. Test your patch for both cases.
That means you have to test for both the vulnerability and whether your application still works. If you can't fix the hole, then the patch is just wasting cycles.
14. Evolve your signatures and rules.
Don't try to make them perfect, if you think a sig, rule or patch is weak, add a better version of it after your old tried and true patch – and run them both until you can prove you don’t need the older rule. Sometimes you need to take a few stabs at a vulnerability to get it right. Don't be afraid to have overlapping and duplicate or similar patches/rules/sigs. Remember defense in depth, use it to your advantage. And remember, humans make mistakes, especially us defenders, so be careful that you don't remove rules that worked before and may appear to not be working again. Its much wiser to add a newer better rule and to leave the older rule still in place. If you see the older rule still tripping (make sure you put your newer rules first), you'll know your newer better rule maybe isn't as good as you thought.
15. Check your tripwires for new unknown vulnerabilities.
You may find some new thing that your attacker has discovered which you are temporarily blocking because of their carelessness in hitting your tripwires. Take that information and craft new patches.
Additional thoughts for MSSPs:
MSSPs often have little to no influence on the customer environment other than the security equipment we manage for them. Most of the time this is network oriented equipment such as IPS and firewall. Of course, MSSPs can (and do) advise customers to use layered defense. Unfortunately, security and server management departments within companies are usually several organizational layers apart.
What MSSPs can do is block things on a network level via the IPSs we manage for these customers. However, as most will know this is indeed limited (hence the effort with mod security). The following guidance is to help MSSPs setup a reverse proxy method to provide an additional layer of security.
And finally, purely technical notes: