Virus Bulletin VBSpam Q4 2024

“The test plan doesn’t mention a minimum required number of mails in each category. Having only 26 newsletters, a single misclassified newsletter will disqualify you from the best category “SPAM+”. We would appreciate either to remove this criteria or to have a minimum sample size for each category.”

“As a vendor, we put much higher importance on blocking phishing and malwares, compared to properly classifying newsletters. Unfortunately, the scoring mechanism doesn’t make a difference if you let through an “annoyance” spam or a malware that would infiltrate the whole company. We wish malwares and phishes were weighted differently in the catch rate and that the cutoff of maximum newsletters FPs were higher or even removed.”

Response from test lab:

“Virus Bulletin investigated this issue and confirmed that significantly fewer newsletter emails than usual were included in this test. This resulted in a reduced resolution for this segment of the data. The decline in numbers was caused by an unexpected but natural drop in the number of newsletters received from ‘in-the-wild’ sources. Virus Bulletin acknowledges this vendor feedback and will take it into consideration moving forward.”