Aleieldin Salem
Skip Abstract Section
Abstract
The malware analysis and detection research community relies on the online platform VirusTotal to label Android apps based on the scan results of around 60 antiviral scanners. Unfortunately, there are no standards on how to best interpret the scan results acquired from VirusTotal, which leads to the utilization of different threshold-based labeling strategies (e.g., if 10 or more scanners deem an app malicious, it is considered malicious). While some of the utilized thresholds may be able to accurately approximate the ground truths of apps, the fact that VirusTotal changes the set and versions of the scanners it uses makes such thresholds unsustainable over time. We implemented a method, Maat, that tackles these issues of standardization and sustainability by automatically generating a Machine Learning (ML)-based labeling scheme, which outperforms threshold-based labeling strategies. Using the VirusTotal scan reports of 53K Android apps that span 1 year, we evaluated the applicability of Maat’s Machine Learning (ML)-based labeling strategies by comparing their performance against threshold-based strategies. We found that such ML-based strategies (a) can accurately and consistently label apps based on their VirusTotal scan reports, and (b) contribute to training ML-based detection methods that are more effective at classifying out-of-sample apps than their threshold-based counterparts.
- 2019. K-9 Mail—Advanced Email for Android. Retrieved on June 2019 from http://tiny.cc/1fbb7y.Google Scholar
- Kevin Allix, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon. 2016. Androzoo: Collecting millions of android apps for the research community. In Proceedings of the 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR’16). IEEE, 468–471.Google ScholarDigital Library
- Daniel Arp, Michael Spreitzenbarth, Malte Hubner, Hugo Gascon, and Konrad Rieck. 2014. DREBIN: Effective and explainable detection of android malware in your pocket. In NDSS.Google Scholar
- Saba Arshad, Munam Ali Shah, Abid Khan, and Mansoor Ahmed. 2016. Android malware detection and protection: A survey. International Journal of Advanced Computer Science and Applications 7 (2016), 463–475.Google ScholarCross Ref
- Davide Chicco and Giuseppe Jurman. 2020. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21, 1 (2020), 6.Google ScholarCross Ref
- Ken Dunham, Shane Hartman, Manu Quintans, Jose Andre Morales, and Tim Strazzere. 2014. Android Malware and Analysis. Auerbach Publications.Google ScholarDigital Library
- Roberto Jordaney Johannes Kinder Feargus Pendlebury, Fabio Pierazzi and Lorenzo Cavallaro. 2019. TESSERACT: Eliminating experimental bias in malware classification across space and time. In 28th USENIX Security Symposium. USENIX Association, Santa Clara, CA.Google Scholar
- Ali Feizollah, Nor Badrul Anuar, Rosli Salleh, and Ainuddin Wahid Abdul Wahab. 2015. A review on feature selection in mobile malware detection. Digital Investigation 13 (2015), 22–37.Google ScholarDigital Library
- Médéric Hurier, Kevin Allix, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon. 2016. On the lack of consensus in anti-virus decisions: Metrics and insights on building ground truths of android malware. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 142–162.Google ScholarDigital Library
- Médéric Hurier, Guillermo Suarez-Tangil, Santanu Kumar Dash, Tegawendé F. Bissyandé, Yves Le Traon, Jacques Klein, and Lorenzo Cavallaro. 2017. Euphony: Harmonious unification of cacophonous anti-virus vendor labels for android malware. In Proceedings of the 14th International Conference on Mining Software Repositories. IEEE Press, 425–435.Google ScholarDigital Library
- AV-Test: The Independent IT-Security Institute. 2019. The best antivirus software for Android. Retrieved on June 2019 from http://tiny.cc/1k66az.Google Scholar
- Alex Kantchelian, Michael Carl Tschantz, Sadia Afroz, Brad Miller, Vaishaal Shankar, Rekha Bachwani, Anthony D. Joseph, and J. Doug Tygar. 2015. Better malware ground truth: Techniques for weighting anti-virus vendor labels. In Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security. ACM, 45–56.Google Scholar
- Tom Kelchner. 2010. The (in) consistent naming of malcode. Computer Fraud & Security 2010, 2 (2010), 5–7.Google ScholarCross Ref
- Li Li, Daoyuan Li, Tegawendé F. Bissyandé, Jacques Klein, Yves Le Traon, David Lo, and Lorenzo Cavallaro. 2017. Understanding android app piggybacking: A systematic study of malicious code grafting. IEEE Transactions on Information Forensics and Security 12, 6 (2017), 1269–1284.Google ScholarDigital Library
- Symphony Luo and Peter Yan. 2014. Fake Apps: Feigning Legitimacy. Retrieved on April 2019 from https://goo.gl/hzZBiH.Google Scholar
- PC Magazine. 2008. BitDefender Free Edition. Retrieved on June 2019 from http://tiny.cc/8vx9az.Google Scholar
- Federico Maggi, Andrea Bellini, Guido Salvaneschi, and Stefano Zanero. 2011. Finding non-trivial malware naming inconsistencies. In International Conference on Information Systems Security. Springer, 144–159.Google ScholarDigital Library
- Brad Miller, Alex Kantchelian, Michael Carl Tschantz, Sadia Afroz, Rekha Bachwani, Riyaz Faizullabhoy, Ling Huang, Vaishaal Shankar, Tony Wu, George Yiu, et al. 2016. Reviewer integration and performance measurement for malware detection. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 122–141.Google ScholarDigital Library
- Aziz Mohaisen and Omar Alrawi. 2014. AV-meter: An evaluation of antivirus scans and labels. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 112–131.Google ScholarCross Ref
- Aziz Mohaisen, Omar Alrawi, Matt Larson, and Danny McPherson. 2013. Towards a methodical evaluation of antivirus scans and labels. In International Workshop on Information Security Applications. Springer, 231–241.Google Scholar
- Jon Oberheide and Charlie Miller. 2012. Dissecting the android bouncer. SummerCon2012, New York.Google Scholar
- Feargus Pendlebury, Fabio Pierazzi, Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. 2018. Enabling fair ML evaluations for security. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2264–2266.Google ScholarDigital Library
- Peng Peng, Limin Yang, Linhai Song, and Gang Wang. 2019. Opening the blackbox of virustotal: Analyzing online phishing scan engines. In Proceedings of the Internet Measurement Conference. ACM, 478–485.Google ScholarDigital Library
- Roberto Perdisci et al. 2012. VAMO: Towards a fully automated malware clustering validity analysis. In Proceedings of the 28th Annual Computer Security Applications Conference. ACM, 329–338.Google Scholar
- Neil J. Rubenking. 2015. False Positives Sink Antivirus Ratings. Retrieved on June 2019 from http://tiny.cc/eyh2uz.Google Scholar
- Shefali Sachdeva, Romuald Jolivot, and Worawat Choensawat. 2018. Android malware classification based on mobile security framework. IAENG International Journal of Computer Science 45, 4 (2018).Google Scholar
- Aleieldin Salem and Alexander Pretschner. 2018. Poking the bear: Lessons learned from probing three android malware datasets. In Proceedings of the 1st International Workshop on Advances in Mobile App Analysis. ACM, 19–24.Google ScholarDigital Library
- Hillary Sanders and Joshua Saxe. 2017. Garbage in, garbage out: How purportedly great ML models can be screwed up by bad data. Technical Report.Google Scholar
- Borja Sanz, Igor Santos, Carlos Laorden, Xabier Ugarte-Pedrero, and Pablo Garcia Bringas. 2012. On the automatic categorisation of android applications. In Proceedings of the 2012 IEEE Consumer Communications and Networking Conference (CCNC’12). IEEE, 149–153.Google ScholarCross Ref
- Borja Sanz, Igor Santos, Carlos Laorden, Xabier Ugarte-Pedrero, Pablo Garcia Bringas, and Gonzalo Álvarez. 2013. Puma: Permission usage to detect malware in android. In International Joint Conference CISIS’12-ICEUTE’12-SOCO’12 Special Sessions. Springer, 289–298.Google ScholarCross Ref
- Ryo Sato, Daiki Chiba, and Shigeki Goto. 2013. Detecting android malware by analyzing manifest files. Proceedings of the Asia-Pacific Advanced Network 36 (2013), 23–31.Google ScholarCross Ref
- scikit learn. 2019. Feature Selection. Retrieved on October 2019 from http://tiny.cc/2d997y.Google Scholar
- scikit learn. 2019. GridSearchCV. Retrieved on October 2019 from http://tiny.cc/mquy9y.Google Scholar
- Marcos Sebastián, Richard Rivera, Platon Kotzias, and Juan Caballero. 2016. AVclass: A tool for massive malware labeling. In International Symposium on Research in Attacks, Intrusions, and Defenses. Springer, 230–253.Google ScholarCross Ref
- Tim Stazzere. 2016. Detecting pirated and malicious android apps with apkid. Retrieved from https://goo.gl/Na2ZAX.Google Scholar
- Guillermo Suarez-Tangil, Santanu Kumar Dash, Mansour Ahmadi, Johannes Kinder, Giorgio Giacinto, and Lorenzo Cavallaro. 2017. DroidSieve: Fast and accurate classification of obfuscated android malware. In Proceedings of the 7th ACM Conference on Data and Application Security and Privacy. ACM, 309–320.Google ScholarDigital Library
- Kimberly Tam, Ali Feizollah, Nor Badrul Anuar, Rosli Salleh, and Lorenzo Cavallaro. 2017. The evolution of android malware and android analysis techniques. ACM Computing Surveys (CSUR) 49, 4 (2017), 76.Google ScholarDigital Library
- VirusTotal. 2019. VirusTotal. Retrieved on September 2019 from http://tiny.cc/xjbb7y.Google Scholar
- Haoyu Wang, Zhe Liu, Jingyue Liang, Narseo Vallina-Rodriguez, Yao Guo, Li Li, Juan Tapiador, Jingcun Cao, and Guoai Xu. 2018. Beyond Google Play: A large-scale comparative study of Chinese android app markets. In Proceedings of the Internet Measurement Conference 2018. ACM, 293–307.Google ScholarDigital Library
- Ting Wang, Shicong Meng, Wei Gao, and Xin Hu. 2014. Rebuilding the tower of babel: Towards cross-system malware information sharing. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. ACM, 1239–1248.Google ScholarDigital Library
- Fengguo Wei, Yuping Li, Sankardas Roy, Xinming Ou, and Wu Zhou. 2017. Deep ground truth analysis of current android malware. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 252–276.Google ScholarCross Ref
- Dong-Jie Wu, Ching-Hao Mao, Te-En Wei, Hahn-Ming Lee, and Kuo-Ping Wu. 2012. Droidmat: Android malware detection through manifest and api calls tracing. In Proceedings of the 2012 7th Asia Joint Conference on Information Security (Asia JCIS’12). IEEE, 62–69.Google ScholarDigital Library
- Wei Yang, Deguang Kong, Tao Xie, and Carl A. Gunter. 2017. Malware detection in adversarial settings: Exploiting feature evolutions and confusions in android apps. In Proceedings of the 33rd Annual Computer Security Applications Conference. ACM, 288–302.Google Scholar
- Wu Zhou, Yajin Zhou, Michael Grace, Xuxian Jiang, and Shihong Zou. 2013. Fast, scalable detection of piggybacked mobile applications. In Proceedings of the 3rd ACM Conference on Data and Application Security and Privacy. ACM, 185–196.Google ScholarDigital Library
- Yajin Zhou and Xuxian Jiang. 2012. Dissecting android malware: Characterization and evolution. In 2012 IEEE Symposium on Security and Privacy (SP’12). IEEE, 95–109.Google ScholarDigital Library
- Shuofei Zhu, Jianjun Shi, Limin Yang, Boqin Qin, Ziyi Zhang, Linhai Song, and Gang Wang. 2020. Measuring and modeling the label dynamics of online anti-malware engines. In 29th USENIX Security Symposium (USENIX Security 20).Google Scholar
Index Terms
- Maat: Automatically Analyzing VirusTotal for Accurate Labeling and Effective Malware Detection
Recommendations
Combining Crowd Contributions with Machine Learning to Detect Malicious Mobile Apps
Internetware '15: Proceedings of the 7th Asia-Pacific Symposium on InternetwareAndroid is undoubtedly becoming the most popular smartphone platform. The popularity of Android, unfortunately, has also made the devices become the target of malware. Most of existing malicious mobile apps feature stealthy operations such as collecting ...
Malware detection using adaptive data compression
AISec '08: Proceedings of the 1st ACM workshop on Workshop on AISecA popular approach in current commercial anti-malware software detects malicious programs by searching in the code of programs for scan strings that are byte sequences indicative of malicious code. The scan strings, also known as the signatures of ...
An in-depth review of machine learning based Android malware detection
AbstractIt is estimated that around 70% of mobile phone users have an Android device. Due to this popularity, the Android operating system attracts a lot of malware attacks. The sensitive nature of data present on smartphones means that it is ...
Comments