Jan Miller is CTO of Threat Analysis at OPSWAT. He shared with us that his journey in malware analysis started with a passion for reverse engineering and low-level programming, which led him to co-found sandbox-focused startups like Payload Security, which was acquired by CrowdStrike, and FileScan.io, which is now part of OPSWAT. These ventures taught him the importance of scalability, adaptability, and transparency in sandboxing technologies.
At OPSWAT, Jan focusses on advancing emulation-based sandboxes that leverage adaptive dynamic analysis to counter modern threats. As a lead author and active contributor to AMTSO’s Sandbox Evaluation Framework, he helps define transparent, real-world testing methodologies to ensure fair evaluations across diverse sandbox use cases.
Jan, you, as a representative of OPSWAT have been very active in AMTSO’s Sandbox Working Group, and provided the draft of the Sandbox Evaluation Framework. What unique perspectives do you have on Sandbox testing?
Sandbox testing is not easy, as there are dozens of vendors and hundreds of features not needed by every use case. This led to the idea of creating a sandbox evaluation/testing framework that is outcome driven and focuses on testing weighted feature sets. For example, not every organization needs exhaustive TTP and IOC extraction. Many prioritize perimeter defense, requiring sandboxes integrated inline, such as with ICAP, for high-speed detection at scale. On the other hand, if you are working in a SOC and performing incident response, a more detailed behavioral insight will be your focus instead of the detection of unknown attack vectors. What performance is expected from a sandbox? Will it be deployed in the cloud or maybe even on premise in an air-gapped secret environment? All of these requirements shape a different testing methodology.
Our sandbox solutions at OPSWAT use emulation-based adaptive dynamic analysis, which dynamically alters malware execution paths to unmask evasive threats. This perspective shaped the framework to emphasize use case-driven metrics, from speed and scalability to depth of analysis and evasion resistance.
What are the key aspects of sandbox technology that you believe should be emphasized in testing guidelines or standards?
I believe in the following key aspects, which should align with the latest AMTSO Sandbox Evaluation Framework draft:
First, testing should be outcome-driven, focussing on what a sandbox achieves—its detection capability, scalability, and insights—rather than how it achieves these objectives.
Second, testing must reflect diverse use case-based evaluations, and align with diverse scenarios, such as:
- Inline Protection: Delivering low-latency real-time detection (e.g., ICAP).
- Threat Triage: Balancing speed with deeper analysis for EDR alerts or phishing detection.
- Advanced Forensics: Extracting rich IOCs and TTPs for comprehensive threat analysis.
Additionally, weighted feature sets are essential. Labs should transparently define and weight the criteria relevant to each use case, ensuring fair evaluations that match real-world needs.
Another critical factor is transparency in methodology. Clear disclosure of sample sets, methodologies, and scoring is critical to building trust and enabling informed decisions.
Moreover, evasion resistance should be a focus, measuring a sandbox’s ability to detect advanced threats, such as those employing virtualization checks or staged execution.
Finally, scalability and speed should be considered. Guidelines should evaluate both high-throughput inline performance and deeper, more resource-intensive forensic capabilities separately.
By emphasizing outcomes, aligning with use cases, and ensuring transparency, testing can help organizations select sandbox solutions that best meet their security goals.
What are your goals that OPSWAT is pursuing as part of the Sandbox working group?
Our primary goal is to create a use case and outcome-driven evaluation framework. Sandboxes serve diverse needs—some focus on rapid inline detection to protect the perimeter, while others are tailored for forensic analysis and extracting rich intelligence.
The framework aims to define clear, weighted criteria for evaluating sandbox performance across these scenarios. By doing so, we help customers select the right sandbox for their specific goals. We also advocate for transparency in testing, enabling organizations to trust the results and build confidence in their selected solutions.
What are some of the most significant challenges in sandbox-based malware detection and analysis today, and how is OPSWAT addressing them?
The two biggest challenges are evasion resistance and scalability: Evasion Resistance is a challenge as malware authors increasingly design threats to detect sandbox environments or require specific execution conditions. OPSWAT uses emulation-based adaptive dynamic analysis, allowing our sandboxes to dynamically manipulate execution paths and reveal hidden malicious behaviors.
Scalability is another critical challenge as inline deployments, such as ICAP integrations, demand high-speed processing for millions of files daily. By leveraging emulation instead of traditional VM-based architectures, our sandbox provides near real-time results while reducing resource overhead.
From OPSWAT’s perspective, what are the most pressing trends or emerging threats that cybersecurity testers need to address?
In addition to the previously mentioned challenges, I believe we see the following trends:
Fileless and Multi-Stage Malware: These threats activate under specific conditions, making it critical for sandboxes to detect partial or incomplete behaviors.
Living-off-the-Land Attacks: Threat actors use legitimate tools, like PowerShell or WMI, to mask malicious activities, posing unique detection challenges.
Supply Chain Attacks: Compromised pipelines and trusted software updates are becoming common attack vectors.
URL-Based Attacks: Increasingly, attackers exploit URLs to deliver malware or conduct phishing campaigns. These attacks often evade traditional antivirus solutions that lack near real-time threat intelligence, making sandboxes essential for detecting dynamic behaviors tied to malicious URLs.
This evolving threat landscape underscores the importance of sandbox solutions that combine advanced detection techniques, threat intelligence, and real-time adaptability to address these challenges effectively.
How does OPSWAT approach innovation in cybersecurity tools, particularly in sandboxing and malware analysis?
We focus on leveraging emulation as a foundation for adaptive dynamic analysis, which allows our sandbox to dynamically influence and adapt to malware execution in real time. This approach reduces reliance on costly VM farms, improves scalability, and enhances detection accuracy.
At OPSWAT, we maintain a startup mindset, prioritizing rapid iteration, experimentation, and close collaboration with customers to refine our solutions. Our sandbox seamlessly integrates with our broader platform, which includes multi-scanners, Deep CDR, and AI, delivering a layered and adaptive security approach.
Where do you see the field of sandbox technology and malware analysis heading in the next five years?
I believe sandboxes will evolve into comprehensive threat detection platforms, combining multiple technologies and approaches to address diverse use cases:
A key development will be the integration of threat intelligence, where platforms will deliver tactical IOCs alongside high-fidelity intelligence, providing context and actionable insights.Emulation-based sandboxing will also play a pivotal role, acting as a fast-pass dynamic analysis layer, enabling near real-time detection for perimeter defense.
At the same time, AI-driven, and context-based detection will become increasingly essential. Advanced AI models will cluster and contextualize threats, improving efficiency and reducing analyst fatigue.
Despite these advancements, traditional sandboxes will continue to be indispensable for forensic analysis and incident response. While emulation will dominate inline use cases, traditional sandboxes will remain critical for forensic analysis and incident response.
Finally, Automation and Orchestration will be important. Threat detection platforms will integrate seamlessly with EDR/XDR solutions, automating workflows and enabling faster threat mitigation.
This convergence will transform sandboxes from standalone tools into essential components of modern security ecosystems.
Why do you think collaboration through organizations like AMTSO is essential for advancing the field?
Attackers collaborate freely, and defenders must do the same to stay ahead. Organizations like AMTSO provide a platform for vendors, researchers, and testers to establish transparent, standardized methodologies that reflect real-world needs.
By focusing on use case-driven testing, AMTSO ensures that sandboxes can be evaluated fairly across diverse requirements, from high-speed inline detection to forensic analysis. This collaboration fosters trust, drives innovation, and raises the overall security standard for the entire industry.
Thanks so much for these great insights, Jan