AMTSO Members vote to move forward with Standards development | AMTSO Standards and Guidelines | Forum
Kraków meeting sees in-depth debate and ultimate approval of Draft Standards.
They were there to discuss one main issue – the ongoing process of adopting the AMTSO Testing Standards. Discussion was lengthy and in-depth, occasionally heated, concluding in overwhelming support for continuing the process to adopt the Draft Standards and move to the operationalization stage.
The Draft Standards are the result of over a year of dedicated development, and the culmination of everything AMTSO has done since its foundation in 2008. Having produced a wealth of guideline documents, tools and resources to help and guide testers and anyone who uses test data, in early 2016 the AMTSO membership decided to start building a set of clear and helpful standards to define and describe everything required to run a fair and helpful test.
The Standards Working Group (SWG) was formed to carry out this task, made up of volunteer representatives from several major testing labs and a number of expert security product developers. At several stages during the process the SWG opened up their work to comment from both the AMTSO membership and publicly, receiving detailed feedback. A pilot process was run in March and April, trialling the latest version of the Standards against several existing test programs.
The latest feedback round collected over 200 individual comments which were reviewed by the SWG and the AMTSO Board of Directors, to produce the final Draft which went forward to the membership vote last week.
The Draft Standards, now approved by members, include requirements for public notification of upcoming tests, the information testers should provide to ensure test participants and test report users can properly understand the significance and relevance of test results, and provisions to ensure vendors play fair and do not try to misrepresent their products in tests.
To support the Standards a range of ancillary services will now start development, including an official Contact List allowing all testers and vendor companies to communicate, a commentary system enabling test participants to provide public feedback and opinion on the design and implementation of a test, and a change process to ensure the Standards evolve over time to keep up with changes with the malware threat and the methods used to counter it. A range of supporting templates and examples are already in preparation, and will be shared with AMTSO member testers for review and feedback within the next few days.
In recent years the organization has grown its membership considerably, thanks in part to diversification in the security industry and a growing recognition of the need to properly measure the capabilities of different approaches to malware prevention. With a new leadership team in place and several staff positions filled, AMTSO is in a position to demonstrate its ability to implement and operate a major project.
Other areas discussed in depth included the benefits testers will derive from the Standards, and the amount of additional work that will be involved, for both testers and vendor participants, in ensuring compliance.
In between sessions debating the Standards, a number of other subjects were covered at the Kraków meeting, with presentations from members and guests ranging from sample sharing approaches and techniques to vulnerabilities in security software, and from acceptance criteria for security apps on the Google Play Store to advanced machine learning and how it might be tested.
The next AMTSO member meeting will take place in Beijing, China in November.
Most Users Ever Online: 29
Currently Browsing this Page:
Tommi Uhlemann: 1
Guest Posters: 0
Administrators: AMTSO Admin, Thomas Wegele, Peter Stelzhammer, Righard Zwienenberg, JohnHawes, Brad Albrecht
Moderators: MarkKennedy, luis_corrons, Eddy Willems, dodiglenn