Antivirus detection rate results update

Alright, it’s been a few months but I finally had some time to update the test.

A few changes have been made to the test environment:
– As the majority of Mac users now use 10.9 Mavericks the virtual machines were all rebuilt from scratch using the latest version of Mavericks (10.9.4). Upgrading the existing VM’s proved problematic and I was not happy with the results so starting fresh was the best option.
– In re-infecting the new VM’s I had a good chance to test Gatekeeper in it’s default setting too (Mac App Store & Identified Developers only). The results were added in a separate column next to XProtect. It shows that OS X does a decent job at blocking malware, 40% of all samples, but since it can easily be bypassed and malware has been seen signed with a valid developer ID Mac users should not rely on Gatekeeper to stay safe. The same goes for XProtect of course which does a lousy job in general.
– Flash Player, Java, Firefox, Chrome and Opera were installed and will be kept up to date with every test.
– VM resources remained the same. 4 CPU Cores, 4 GB RAM and ample drive space on a dedicated SSD.
– Little Snitch is no longer being used in the virtual environments as it may impact the behavior of certain malware. Instead VM’s now use their own ethernet cable that leads to a Mac with internet sharing enabled. On that Mac Little Snitch is active so connection attempts can still be monitored. As far as the VM knows it is connected to the internet and has no monitoring software present but on the other Mac I can still see exactly which types of connections are being made and where to. So far in testing this has worked well. For a more detailed analysis, if needed, this setup also allows me to utilize other tools without impacting the virtual environment.
– The older 10.8 virtual machines were updated with the latest samples and software and will be kept around if needed for testing.
– The applications that have a detection rate of 90% or higher has doubled since the test started. I felt this was a good time to make this the new standard. Whereas previous tests showed the top as being 80% or higher AV, this has now been raised to 90%. I might even make the top performing category 95% and better soon. Why should we not expect the best of applications that claim they protect us, right?

Some observations in this test:
– Avast kept running the virtual machine into the ground as soon as the installer was finished so I used the old 10.8 VM instead. This was also completely unresponsive and crashed after the Avast install. I used an archived installer from the beginning of the year, this installed without issues. From there I could update to their new version 9 and run the test. I don’t know if this is because of the virtual environment or if their latest installer behaves the same on actual Macs. Use caution.

– I’m very happy to see F-Secure finally released an actual application, it seems they take Mac users seriously now. Their previous products were not very stable and definitely did not run well in virtual environments, this has changed. The application has preferences that reside in the System Preferences window, the scan results are clear and the interface is neat. There are no options when it comes to scan settings. Whatever is found is trashed immediately, no questions asked, or labeled as riskware and left for you to clean. Real-time scanning can not be disabled. Apart from a few minor issues with the interface like the scan being completed but the progress bar being stuck at 98% the application performed well.

– Something I liked a lot about ESET version 6 was it’s notifications the operating system was out of date. The Vm did not have the latest iTunes update installed and, while not critical for the OS, this is a great feature to have. I have not seen this from any of the previous ESET products.

– MacKeeper was not willing to provide me with a trial license (needed to update virus definitions). A supervisor will let me know within a week if I can get one, they will also let me know if I should exclude them from the test going forward. I’m certainly not postponing the test for a week to wait for that license so their results were not updated.

– I was unable to get BitDefender (app store version) to scan. It downloaded ok, definitions updates and the app launched fine. However when I clicked any of the scan buttons the app would just sit there and do nothing. Reboots, reinstalls, fresh VM and even an actual Mac running OS X 10.9.4 all had the same result. As the app was last updated two years ago it may simply no longer be compatible with Mavericks. I’ll test some more in the near future. If I can not get it to work I will revert back to the 10.8 VM for this particular app and continue it’s testing.

– After ClamXav’s last sudden improvements I had high hopes for this test. Sadly it did not improve as much as I had hoped.

Other notes:
I’m considering making the trace detection results count towards the final percentage. Trace files are present on systems that are already infected and the original file that caused the infection may be long gone (an installer or downloaded file of some kind).

I will be working on updating the rest of the application results in the upcoming week or two.

The results can be found here.