How to Deconstruct and Reverse Engineer Malware Using Ghidra - Atomicorp - Unified Security Built on OSSEC

Combining Ghidra Reverse Engineering and OSSEC Protection

By Scott Shinn

Ghidra is an open source reverse engineering tool developed by the NSA. Four years mature, Ghidra is a solid tool that can be used to disable attacks at the root, enabling you to take the malware apart and build immunities. It’s reverse engineering via an open source tool today.

At the February 2023 Atomic OSSEC Conference, I demonstrated a lightweight, feature-rich set of open source security rules – a host-based intrusion detection and response system to support your open source reverse engineering program. 

Watch how the combined capabilities of Ghidra and Atomic OSSEC dove deep into malware reverse engineering at the recent Atomic OSSEC Conference 2023.


Detect, Detain, Analyze, and Reverse Engineer Malware Using OSSEC and Ghidra

In the video, I’m on a VM (Fedora 37 virtual machine desktop) running Ghidra  in an Atomic OSSEC environment. 


I’m gonna talk about two major mechanisms that we employ here. 

One is open source-based security detection, prevention and compliance (Atomic OSSEC). The second is Ghidra’s binary reverse engineering capabilities. 

You want to be able to detect, detain, analyze. Be able to look beyond the code for clues such as compressed files and unusual system behavior or patterns, and filter and block anomalous text strings before you accept something into your sandboxed or live environment. You are, in essence, safeguarding workflows, using an overarching lightweight HIDS and WAF that sniffs IP traffic and payload, identifies patterns, and blocks and immobilizes malware and malicious strings of text earlier. This is malware we’re dealing with, and we want to isolate it while we’re analyzing it. Sometimes that means we need to look at how it communicates on the network.

I’m heavily utilizing other malware scanning tools like ClamAV, rootcheck, and ssdeep. Also, I’m using syscheck (FIM) that enables real-time detection of files created in the OS. I always use this on any part of a system that has an externally reachable attack surface, like a web server. 

The Atomic OSSEC host-based intrusion detection system (HIDS) provides endpoint and cloud workload protection, and a deep and effective open-source-security-based environment for collecting data for your Ghidra malware reverse engineering program.

The Atomic OSSEC system can:

  • Inspect the content of a file or registry 
  • Test processes
  • Examine directories
  • Read binary or ASCII files 
  • Read Windows registries 
  • Look into files for anomalies 
  • Integrate more advanced open-source-based solutions that are about prevention.
    • Fine-tune ClamAV and ModSecurity, all in one tool. Be able to include SELinux and Fanotify, the latter which is a file system IPS. 
  • Enhance IT service performance, malware detection, application identification, proactive response, and compliance. In sophisticated attacks, the malware hides and can delete itself but Atomic OSSEC captures a record.
  • Address security and privacy regulations and standards that require artifacts for compliance auditing. 
  • Repel brute force attacks on a machine, on a shell, with brute force repellents built in.

How Atomic OSSEC works in a Ghidra application

The Atomic OSSEC framework does all the hard work of providing an environment for multiple detection methods, from simple string analysis to locality sensitive hashing. It can detect using signatures, or more broadly by implementing a real-time forensic versioning file system. 

Atomic OSSEC can be orchestrated via a command line interface (CLI) (See Figure 1) and also includes a management and analyst GUI (see Figure 2).

Figure 1: Capture, Isolate and Take Malware Apart at the Command Line

Capture, Isolate and Take Malware Apart at the Command Line

Figure 2: FIM GUI Dashboard for Intrusion Detection and Compliance

FIM GUI Dashboard for Intrusion Detection and Compliance

In Figure 2 above, we used real-time FIM to capture all files created by the web server user. This allowed us to get a copy of the malware, even when the code deletes itself after running.    

So we’ve captured a variant, and now let’s take it apart using Ghidra (see Figure 3).


Figure 3: Atomic OSSEC captures malware and we take it apart in Ghidra. 

Atomic OSSEC captures malware and we take it apart in Ghidra.

You will be able to look at what was captured and analyze the malware safely, comparing the alerted with the original via a FIM-highlighted view of what changed. Look for strings in files, and be able to search the log right down to the character. Where it’s from, what development language they’re using, what troll servers they’re using.


Getting Into Ghidra Malware Deconstruction at Atomic OSSEC Conference 2023

Block, filter, and maintain the integrity of your data environments. See how we take apart malware in Ghidra, and use the forensics to orchestrate even better security. Watch my presentation

Listen to the Atomic OSSEC conference’s graphical UI presentation recording to learn how to:

    • Practice malware reverse engineering with Ghidra. Reinforce a Ghidra project in a sandboxed environment or real-time scenario.
    • Block binary and PHP malware. Integrate a security response earlier against threats and vulnerabilities.
    • Perform forensics on malware. 
    • Control what the application sees from DNS.
    • Look at payload beyond SSL.
    • Engineer breaks in your code processing to stop hostile programming. 
    • Be able to detect and extract malware riding inside the software supply chain, in a network filtering / traffic shaping environment such as Linux eBPF, or a programming language such as Log4J . . . in operating systems like Windows and Linux, and legacy C languages such as AIX, and across cloud and web APIs. 
    • Look at strings. Get into a highlighted compressed file. Is malware trying to hide in files as small as 25k and 35k? The answer is yes, and sometimes even smaller. Ghidra and Atomic OSSEC provides the ability to decompress and look at the code securely. Discover what programming language is used, country of origin, IP server, and be able to block sources and reverse engineer with Ghidra. 
    • Get granular at the hash and caret level. Be able to look at small tight bits of malicious code. See the variables they’re using through preconfigured alerts and search capabilities.
    • Filter both input and output (I/O) and be able to block and deconstruct for compliance and risk. 
    • Turn the malware back into the software before it was compromised with Decompiler. However, keep in mind, part of the fun of malware reverse engineering is that you can look at original source code and use the attacker’s same function libraries. 
    • Go beyond patching. Atomic OSSEC backs up and analyzes the log and uses threat intelligence to engineer future security rules into the detection and response engine. Block it and be able to reverse-engineer protection into the source software, mitigating the risk of the attack reaching authorized customers and system users. 
    • Proactively defend the ChatGPT attack angle. Developed by OpenAI, ChatGPT is a groundbreaking app but already has proven as pregnable. Leverage deep malware detection and extraction across file, system and application threats and vulnerabilities.
    • Build detection into your web hosting platforms like Plesk or cPanel. You may have run into situations where you have to scan the whole web application tree and this can be time-consuming. The Atomic OSSEC file integrity monitoring (FIM) and HIDS engine provides a smart and efficient defense.


    Atomic OSSEC for Malware Detection and Removal

    Atomicorp is a U.S.-based security and compliance provider specializing in endpoint and cloud workload protection. Atomicorp leverages the power of the world’s most widely used open source security solution, OSSEC, to detect malicious activity and protect business-critical workloads, regardless of where they reside. Atomicorp overcomes the challenges of ensuring security and compliance on processing across multiple cloud and data center environments via a lightweight, DevOps-friendly platform that automates a wide range of functional requirements, including file integrity monitoring (FIM).

    Atomic OSSEC is an easy and evolving open source based security system to pop in, to protect servers, VMs, web and cloud APIs, and container and cluster environments. It’s been around since the late 1990s. Your risk of implementing it is low, it’s inexpensive and comes in versatile agented and non-agented service models, including software as a service (SaaS). 

    Atomic OSSEC will help you to:

    • Respond to threats on your IT, through open-source and Atomicorp global threat intelligence and automated rules response. 
    • Detect and manage vulnerabilities, active response, and audit and accountability controls across SIEM and for compliance requirements such as NIST 800-171 and PCI DSS 11.5. 
    • Implement in a way that is generic and lightweight enough so you are not impacting performance. 
    • Block suspicious packets and unauthorized command languages. Be able to configure nonblocking for authentic traffic or forensic analysis. 
    • And much more.

    Learn how to create better open-source security rules using Ghidra. Watch my full presentation from Atomic OSSEC Conference 2023.


    Get an Atomic OSSEC demonstration. Check out the full array of videos from Atomic OSSEC Conference 2023.


    We support the open source security community and OSSEC projects, contribute, and integrate and enhance developments in our free and enterprise-grade offerings. Join us on Slack.


    Learn more about Atomic OSSEC endpoint and cloud workload protection for extended detection and response (XDR) and standards and regulatory compliance. Visit the page.