VPN Performance Testing Guidelines tailored to enhance fair privacy and security assessments
San Francisco, California – February 19, 2025 – AMTSO, the cybersecurity industry’s testing standard community, today announced the release of the first in a series of VPN Testing Guidelines, focusing on performance assessment. This publication marks a major step forward in providing standardized methodologies for evaluating VPN services. The new guidelines, developed by AMTSO’s VPN working group which includes VPN vendors and testers, offer comprehensive testing recommendations to ensure fair and effective assessments of VPN performance, security, and reliability.
As VPNs have long become an essential tool for online privacy and security, evaluating their effectiveness has become increasingly important. AMTSO’s VPN Performance Testing Guidelines cover key aspects such as:
- Launch on-boot and auto-connection testing – ensuring VPNs activate automatically to protect users from the moment their device starts.
- Internet kill switch tests – verifying that no data leaks occur if a VPN connection drops unexpectedly.
- Leak prevention controls – assessing whether VPNs effectively block DNS, IP, and WebRTC leaks.
- Split tunnel testing – checking whether users can exclude specific applications from VPN encryption without unintentional leaks.
- Speed and latency tests – evaluating the impact of VPN services on internet performance under different network conditions.
- Website accessibility and resource consumption – measuring how well VPNs maintain access to websites and optimize device resources.
“This first release is an important milestone in our mission to provide testing guidance in the world of privacy and security,” said John Hawes, COO of AMTSO. “By setting clear benchmarks, we enable fair comparisons and help users make informed choices when selecting a VPN provider.”
AMTSO invites independent testers and VPN vendors to adopt these guidelines in their assessments and join the ongoing efforts to refine VPN testing methodologies. Moving forward, the AMTSO VPN Working Group will expand its focus to include malware testing within VPN environments and additional security feature evaluations.
The guidelines were created by AMTSO’s VPN Working Group, with contributors including AV-Test, SE Labs and Nord Security.
The AMTSO VPN Performance Testing Guidelines are available at: https://www.amtso.org/documents/.
About AMTSO:
AMTSO is the cybersecurity industry’s testing standard community, consisting of over 50 security and testing member companies from around the world. The organization offers a platform for knowledge-sharing and collaboration on objective standards and best practices for anti-malware testing and assessment of other cybersecurity products. The AMTSO standard raises the bar for cybersecurity tests, contributing to more fairness in the industry, and creating transparency for consumers and businesses looking for the best digital protection. www.amtso.org