Malware detection list updated, yup, again [updated]
It’s been a long and hard few days for my poor Mac Pro, running a dozen virtual machines simultaneously for hours on end testing AV products and only getting some rest when I was sleeping (the AC had to run a bit extra too because the office felt like a sauna!) But it’s done. I’ve created brand new images, thoroughly re-infected all of them, re-installed all of the products and re-tested them all against the full sample set. The results can be found here. I expect the next update to be shortly after OS X 10.9 Mavericks is released, giving the AV companies some time to update their products, I’ll then upgrade the VM’s and re-test. Here are some of the notes and changes:
UPDATE: I forgot to upload the new PDF, just did this now. Apologies to those that were driven insane thinking their browser cache was causing the old PDF to download every time! The page that has the link to the PDF will now also show the date and time it was last updated, I’ll adjust this every time áfter I upload the PDF.
After adding the new samples VirusBarrier still performs really well and detects all of them right away. This gives me hope because I mentioned in a previous comment it looked like they just added all the signatures in this list to look good in the test. While this may still have been the case, the fact that all new samples were detected before this list went live shows they have done their homework. VirusBarrier 2013 now dominates first place and the older X6 version a very close second place.
Avast has been updated to version 8 and has dropped down the ranks to place #4
FortiClient has been updated to version 5.0.6.131 but the results are the same. This leaves FortiClient in the “excluded from future testing” list. We’ll dust it off and take it for another spin when they become serious about protecting their Mac customers.
Dr.Web was updated to version 9.0.0 but all efforts to get it up and running failed. The installer had to be downloaded via an FTP link found on startpage.com after trying to download it from the site directly failed every time. Once installed attempts to generate a trial license failed. “Unknown Error. Please contact support” and “License key file generation error, please try the website”. No one would waste more than 10 minutes on stuff like this so I think I was ok to stop trying after 30. So for now the results reflect version 6.
Small corrections in Avira’s results. Genieo was marked as detected but this was incorrect.
ClamXav took a pretty big hit knocking it out of the 60-80% detection rate and was moved to the 60% or lower list. After testing the product for this long and seeing no improvement there is no reason to expect improvement anytime soon.
Unable to test Trend Micro Titanium. Even from a clean VM and over a VPN it insists the newly started trial has expired.
All McAfee products use the same database and none of the new or previously missed samples were detected. Apparently no work has been done for the Mac users at all. As a result all of them fell below 55% detection rate. I have moved both business products to the “excluded from future testing list. The consumer version (Internet Security) will be sporadically tested. Until I see a noticeable improvement in results for the consumer version there is no reason to believe the business products are worth testing (or spending money on).
3 of the 8 IceFog samples were blocked by XProtect, all of them compromised applications. As all other samples are Mach-O files and one non-functional application XProtect does indeed do it’s job this time around to keep OS X users safe from this particular malware. Those that have been infected in the two weeks it took Apple to update XProtect are not so lucky, the files installed by IceFog are not blocked by XProtect so the backdoor can still launch itself after every restart. Genieo and most Yontoo samples are still allowed to install but Leverage samples were blocked.