MITRE recently published the detailed results of their second round of tests. This test pitted APT29 malware and methods against 21 cybersecurity vendors. The MITRE testing is an excellent benchmark for comprehensively exercising Endpoint Protection (EPP) and Endpoint Detection & Response (EDR) tools in real-world scenarios where organizations find themselves under attack by Advanced Persistent Threats (APTs). MITRE describes the environments, methodology, and operation flow of their testing regime in great detail here. The raw results are available for review, and they have created a tool to help with data analysis.
Many of the participating vendors have analyzed and posted their interpretations of this latest round of testing. In general, each vendor’s internal analysis tends to focus on their areas in which they do well. In this article, we will call out the most serious of the commonly missed detections of specific procedures in the framework.
There was a wide range in the success rates experienced by vendors, with higher success rates equating to low values for missed detections. As expected, there is a fairly strong correlation between a vendor’s product having specific techniques for detection of procedures and steps in the ATT&CK framework and catching a would-be malicious actor utilizing those actions.
The ATT&CK framework features a matrix of 11 tactics and 171 techniques that adversaries use to conduct a cyber-attack. MITRE has just released an updated version of the ATT&CK framework which contains sub-techniques. The techniques and sub-techniques depicted generally relate to application or system level API and/or procedure calls. Legitimate software including operating systems use these API and system calls to accomplish tasks on behalf of users. This is what makes the job of anti-malware and other security tools difficult: they must distinguish the “intent” of the calling code. For example, encrypting files is most often a legitimate operation to protect data. However, ransomware also attempts to encrypt files. How does anti-malware decide to allow some operations to proceed and stop others?
What is the code’s intent?
Intent can be discerned by context. Context can be probabilistically inferred by looking at the calling process: which user account launched it, how long has it been running, have other processes injected it, what other calls precede or follow it in the queue, etc. Security vendors build detection models using analysis of code and memory enhanced by Machine Learning (ML) to make these kinds of determinations. Security tools are also informed by collections of threat intelligence and Indicators of Compromise (IOCs).
The following list of techniques are those which many vendor products were not able to detect in the latest round of MITRE testing, and those which we consider the most consequential if exploited.
Missed detections
- Sandbox evasion techniques such as checking BIOS, manufacturer, model, version
- Attempts to use PowerShell to get system, domain, logged-on username, running processes, anti-malware presence, and endpoint firewall status
- Reconnaissance of lateral assets using PowerShell
- Manipulation of access token for PowerShell
- Enumerating files in directories commonly used for user data; enumerating media files
- Enumerating registry keys
- Capturing screenshots and clipboard contents (lesser instances of keylogging)
- Decryption of Chrome browser username and password databases
- Creation of pre-exfiltration staging directories
- Simple attempts to encrypt C2 traffic using standard crypto techniques
- Reading user files, while connected to C2
- Encryption of user data files
- Exfiltration over non-standard TCP ports
- Exfiltration over HTTPS
- Establishment of persistence via javavmtsup
Recommendations for vendors
Threat actors are constantly improving and adapting to innovations in security products. Thus, security products with deficiencies in detecting items on the preceding list should focus on:
- Anti-evasion: malware detonation sandboxes are standard features in EPP and NDR products, but they can’t help if malware is designed to lay dormant if it can figure out that it’s in a sandbox. Sandbox evasion used to be limited to the most sophisticated malware, but it has become more of a commodity feature.
- PowerShell scrutiny: PowerShell is a powerful and useful tool, as its name implies. However, given the massive proliferation of file-less malware that uses PowerShell (as a means to avoid detection by signature-based and static file scanners), security solutions must scrutinize every invocation by pulling more context for deeper examination.
- Recon and lateral movement: APT actors learned to go low and slow to avoid setting off SIEM alerts. Extending data retention periods can make it easier to group what may seem to be outliers into suspicious incidents. For example, seemingly random enumeration requests over long periods may point to a concerted effort to map resources.
- Secure the browser username/password database. Users often rely on browsers for generating, storing, and protecting credentials. Besides data exfiltration by APTs, insecure browser password databases can lead to Account Takeovers (ATOs) for users’ personal information.
- Stop staging: Users don’t often move and rename data files, then package it all up to be sent elsewhere.
- Lock down comms: Applications sending traffic on non-standard ports and/or to suspicious IPs should send up red flags, even if the traffic volume is very low. Continuously updated high quality threat intelligence can mitigate this risk.
- Encryption scrutiny: Ransomware is still rampant. While many garden varieties can be interdicted by most security tools, ransomware authors are also evolving their tools to defeat ransomware protection. For example, some ransomware groups use APT tactics of making an initial silent compromise, surveying enterprise assets, staging the malicious payloads in various places around the target environment, then simultaneously detonating them to limit the effectiveness of anti-ransomware prevention and detection techniques. Some security tools need to view ransomware in the same way as APTs and combine detection and prevention methods to help at-risk clients.
The good news is that several endpoint and network security vendors did very well in this run of the MITRE ATT&CK evaluation. We have noted this in our latest research: Market Compass Endpoint Protection Detection & Response. While the MITRE test is significant in that it covers EPP and EDR, it is only a single test. There are other independent test regimes that should be considered when evaluating endpoint security solutions. If your organization is considering upgrading or replacing EPP or EDR, or adding Network Detection & Response (NDR), KuppingerCole can assist with reviewing your requirements and crafting an RFP based on our up-to-date research. Also, for cybersecurity vendors that need help in prioritizing their product or service roadmaps, we can provide tailored advisory services as well. For more information, visit https://www.kuppingercole.com/advisory.
