Abstract
Increasing interest in securing the Android ecosystem has spawned numerous efforts to assist app developers in building secure apps. These efforts have resulted in tools and techniques capable of detecting vulnerabilities and malicious behaviors in apps. However, there has been no evaluation of the effectiveness of these tools and techniques in detecting known vulnerabilities. The absence of such evaluations puts app developers at a disadvantage when choosing security analysis tools to secure their apps. In this regard, we evaluated the effectiveness of vulnerability detection tools for Android apps. We reviewed 64 tools and empirically evaluated 14 vulnerability detection tools against 42 known unique vulnerabilities captured by Ghera benchmarks, which are composed of both vulnerable and secure apps. Of the 20 observations from the evaluation, the main observation is existing vulnerability detection tools for Android apps are very limited in their ability to detect known vulnerabilities — all of the evaluated tools together could only detect 30 of the 42 known unique vulnerabilities. More effort is required if security analysis tools are to help developers build secure apps. We hope the observations from this evaluation will help app developers choose appropriate security analysis tools and persuade tool developers and researchers to identify and address limitations in their tools and techniques. We also hope this evaluation will catalyze or spark a conversation in the software engineering and security communities to require a more rigorous and explicit evaluation of security analysis tools and techniques.
Similar content being viewed by others
Notes
We use the terms benign and vulnerable interchangeably.
dex_date may not correspond to the release date of the app.
API level 20 was excluded because it is API level 19 with wearable extensions and Ghera benchmarks focus on vulnerabilities in Android apps running on mobile devices.
In the rest of this manuscript, “API levels 19 thru 25” means API levels 19 thru 25 excluding 20.
We chose API level 14 as the cut-off for minimum API level as the number of apps targeting API level 19 peaked at minimum API level 14.
The raw data supporting these observations is available as part of publicly available evaluation artifacts; see Section 6.
These numbers will only be higher when lower p-value thresholds are considered.
In the rest of this manuscript, for brevity, we say “a tool is applicable to a benchmark” if the tool is designed or claims to detect the vulnerability captured by the benchmark. Likewise, “a tool is applicable to a set of benchmarks” if the tool is applicable to at least one benchmark in the set.
While Ghera did not have benchmarks that were applicable to some of the rejected tools at the time of this evaluation, it currently has such benchmarks that can be used in subsequent iterations of this evaluation.
Refer to Appendix A for a brief description of these benchmarks.
AndroBugs, Marvin-SA, and MobSF were the exceptions.
We considered all variations of a tool as one tool, e.g., JAADAS. We did not count FixDroid as we did not evaluate it on secure apps in Ghera.
TP, FP, FN, and TN denote the number of true positive, false positive, false negative, and true negative verdicts, respectively.
We considered TP-only (FN-only) APIs along with common APIs as vulnerabilities may depend on the combination of TP-only (FN-only) APIs and common APIs.
While evaluating the tools against all vulnerabilities seems like a solution to address this threat, we recommend against it as it would be unfair (as mentioned in Consider only applicable categories of vulnerabilities paragraph in Section 4.3.1).
The corresponding GitHub ticket can be found at https://github.com/AndroBugs/AndroBugs_Framework/issues/16.
Refer to Sections 2 and 3.4 of the work by Mitra and Ranganath (2017a) for a detailed description of characteristics of Ghera.
References
Abraham A, Schelecht D, Dobrushin M (2015) Mobile Security Framework. https://github.com/MobSF/Mobile-Security-Framework-MobSF, Accessed: 21-Nov-2017
Agrawal A (2015) Mobile Security Wiki. https://mobilesecuritywiki.com/, Accessed: 01-May-2018
Allen B (2018) AppCritique: Online Vulnerability Detection Tool. https://appcritique.boozallen.com/, Accessed: 21-Nov-2017
Allix K, Bissyande TF, Klein J, Traon YL (2016) Androzoo: Collecting millions of android apps for the research community. In: Proceedings of the 13th International Conference on Mining Software Repositories. ACM, pp 468–471
Anand S, Naik M, Harrold MJ, Yang H (2012) Automated concolic testing of smartphone apps. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM, pp 59:1–59:11
Antunes N, Vieira M (2010) Benchmarking vulnerability detection tools for web services. In: 2010 IEEE International Conference on Web Services, pp 203–210
App-Ray (2015) AppRay. http://app-ray.co/, Accessed: 04-Jun-2018
Arzt S, Rasthofer S, Fritz C, Bodden E, Bartel A, Klein J, Le Traon Y, Octeau D, McDaniel P (2014) Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. In: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, pp 259–269. https://github.com/secure-software-engineering/FlowDroid, Accessed: 21-Nov-2017
Au KWY, Zhou YF, Huang Z, Lie D (2012) Pscout: Analyzing the android permission specification. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security. ACM, pp 217–228, http://pscout.csl.toronto.edu/, Accessed: 21-Nov-2017
Backes SRT GmbH (2014) SRT:AppGuard. http://www.srt-appguard.com/en/, Accessed: 04-Jun-2018
Bagheri H, Sadeghi A, Garcia J, Malek S (2015) Covert: Compositional analysis of android inter-app permission leakage. IEEE Transactions on Software Engineering. pp, 866–886. https://seal.ics.uci.edu/projects/covert/index.html, Accessed: 21-May-2018
Bhatia A (2014) A collection of android security related resources. https://github.com/ashishb/android-security-awesome, Accessed: 01-May-2018
Bosu A (2017a) DIALDroidBench. https://tool865110240.wordpress.com/dialdroidbench/, Accessed: 12-Sep-2018
Bosu A, Liu F, Yao DD, Wang G (2017b) Collusive data leak and more: Large-scale threat analysis of inter-app communications. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. ACM, New York, pp 71–85. https://github.com/dialdroid-android, Accessed: 05-May-2018
Bugiel S, Heuser S, Sadeghi AR (2013) Flexible and fine-grained mandatory access control on android for diverse security and privacy policies. In: 22nd USENIX Security Symposium. USENIX. http://www.flaskdroid.org/index.html04-Jun-2018
Calzavara S, Grishchenko I, Maffei M (2016) Horndroid: Practical and sound static analysis of android applications by SMT solving. In: 2016 IEEE European Symposium on Security and Privacy, pp 47–62. https://github.com/ylya/horndroid, Accessed: 05-May-2018
Chin E, Felt A P, Greenwood K, Wagner D (2011) Analyzing inter-application communication in android. In: Proceedings of the 9th International Conference on Mobile Systems, Applications, and Services. ACM, pp 239–252
Chin E, Wagner DA (2013) Bifocals: Analyzing webview vulnerabilities in android applications. In: WISA. Springer, pp 138–159
Cuckoo (2015) CuckooDroid: Automated Android Malware Analysis. https://github.com/idanr1986/cuckoo-droid, Accessed: 01-May-2018
Debize T (2012) AndroWarn : Yet Another Static Code Analyzer for malicious Android applications. https://github.com/maaaaz/androwarn/, Accessed: 21-Nov-2017
Deshotels L, Notani V, Lakhotia A (2014) Droidlegacy: Automated familial classification of android malware. In: Proceedings of ACM SIGPLAN on Program Protection and Reverse Engineering Workshop 2014. ACM, pp 3:1–3:12
DevKnox (2016) DevKnox - Security Plugin for Android Studio. https://devknox.io/, Accessed: 21-Nov-2017
DroidBench (2013) DroidBench: A micro-benchmark suite to assess the stability of taint-analysis tools for Android. https://github.com/secure-software-engineering/DroidBench, Accessed: 01-June-2018
Egele M, Brumley D, Fratantonio Y, Kruegel C (2013) An empirical study of cryptographic misuse in android applications. In: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security. ACM, pp 73–84
Enck W, Gilbert P, Chun BG, Cox LP, Jung J, McDaniel P, Sheth AN (2010) Taintdroid: An information-flow tracking system for realtime privacy monitoring on smartphones. In: Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, USENIX, pp 393–407
Ernst MD, Just R, Millstein S, Dietl W, Pernsteiner S, Roesner F, Koscher K, Barros PB, Bhoraskar R, Han S, Vines P, Wu EX (2014) Collaborative verification of information flow for a high-assurance app store. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. ACM, pp 1092–1104. https://www.cs.washington.edu/sparta, Accessed : 01-June-2018
Fahl S, Harbach M, Muders T, Baumgärtner L, Freisleben B, Smith M (2012) Why eve and mallory love android: An analysis of android ssl (in)security. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security. ACM, pp 50–61. https://github.com/sfahl/mallodroid, Accessed: 15-Apr-2018
Fuchs A, Chaudhuri A, Foster JS (2009) Scandroid: Automated security certification of android applications. Tech. rep., University of Maryland. https://github.com/SCanDroid/SCanDroid, Accessed: 04-Jun-2018
Garcia J, Hammad M, Ghorbani N, Malek S (2017) Automatic generation of inter-component communication exploits for android applications. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, pp, 661–671. http://seal.ics.uci.edu/projects/letterbomb/, Accessed: 24-Apr-2018
Google Inc (2017) Android Intent with Chrome. https://developer.chrome.com/multidevice/android/intents, Accessed: 29-May-2018
Google Inc (2018a) Android developer documentation - Binder. https://developer.android.com/reference/android/os/Binder.html#getCallingPid(), Accessed: 01-Jun-2018
Google Inc (2018b) Android developer documentation - Content Provider. https://developer.android.com/reference/android/content/ContentProvider.html#call(java.lang.String,%20java.lang.String,%20android.os.Bundle), Accessed: 07-Mar-2018
Google Inc (2018c) Android Security Tips. https://developer.android.com/training/articles/security-tips, Accessed: 01-Jun-2017
Gordon MI, Kim D, Perkins J, Gilham L, Nguyen N, Rinard M (2015) Information-flow analysis of Android applications in DroidSafe. In: Proceedings of the 22nd Annual Network and Distributed System Security Symposium (NDSS’15). https://github.com/MIT-PAC/droidsafe-src, Accessed: 21-Apr-2018
Gray J (1992) Benchmark Handbook: For Database and Transaction Processing Systems. Morgan Kaufmann Publishers Inc., San Mateo
Green M, Smith M (2016) Developers are not the enemy!: The need for usable security apis. IEEE Secur Priv 14(5):40–46
IBM (2018) IBM App Scan. https://www.ibm.com/us-en/marketplace/ibm-appscan-source, Accessed: 01-June-2018
Jia YJ, Chen QA, Lin Y, Kong C, Mao ZM (2017) Open doors for bob and mallory: Open port usage in android apps and security implications. In: EuroS & P. IEEE, pp 190–203
Keen T (2016) Joint Advanced Application Defect Assessment for Android Application (JAADAS). https://github.com/flankerhqd/JAADAS, Accessed: 21-Nov-2017
Klieber W, Flynn L, Bhosale A, Jia L, Bauer L (2014) Android taint flow analysis for app sets. In: Proceedings of the 3rd ACM SIGPLAN International Workshop on the State of the Art in Java Program Analysis. ACM. https://www.cert.org/secure-coding/tools/didfail.cfm, Accessed: 21-Apr-2018
Li L, Bartel A, Bissyandé TF, Klein J, Le Traon Y, Arzt S, Rasthofer S, Bodden E, Octeau D, McDaniel P (2015) Iccta: Detecting inter-component privacy leaks in android apps. In: Proceedings of the 37th International Conference on Software Engineering - Volume 1. IEEE Press, pp 280–291. https://github.com/lilicoding/soot-infoflow-android-iccta, Accessed: 05-May-2018
Lin YC (2015) AndroBugs Framework. https://github.com/AndroBugs/AndroBugs_Framework, Accessed: 21-Nov-2017
LinkedIn (2015) Quick Android Review Kit. https://github.com/linkedin/qark/, Accessed: 21-Nov-2017
Mitra J, Ranganath V (2017a) Ghera: A repository of android app vulnerability benchmarks. In: Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering. ACM, pp 43–52. https://bitbucket.org/secure-it-i/android-app-vulnerability-benchmarks/, Accessed: 21-Nov-2017
Mitre Corporation (2017b) Common vulnerabilities and exposures. https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=Android, Accessed: 08-Jun-2017
MWR Labs (2012) Drozer. https://github.com/mwrlabs/drozer/, Accessed: 20-Apr-2018
Nadkarni A, Andow B, Enck W, Jha S (2016) Practical DIFC enforcement on android. In: 25th USENIX Security Symposium (USENIX Security 16). USENIX, pp 1119–1136. https://wspr.csc.ncsu.edu/aquifer/, Accessed : 01-June-2018
Nguyen DC, Wermke D, Acar Y, Backes M, Weir C, Fahl S (2017) A stitch in time: Supporting android developers in writing secure code. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, pp 1065–1077. https://plugins.jetbrains.com/plugin/9497-fixdroid, Accessed: 21-Apr-2018
Pauck F, Bodden E, Wehrheim H (2018) Do android taint analysis tools keep their promises? In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 331–341
Powers DMW (2011) Evaluation: From precision, recall and f-measure to roc., informedness, markedness & correlation. J Mach Learn Technol 2(1):37–63
Reaves B, Bowers J, Gorski III SA, Anise O, Bobhate R, Cho R, Das H, Hussain S, Karachiwala H, Scaife N, Wright B, Butler K, Enck W, Traynor P (2016) *droid: Assessment and evaluation of android application analysis tools. ACM Comput Surv 49(3):55:1–55:30
Ren C, Zhang Y, Xue H, Wei T, Liu P (2015) Towards discovering and understanding task hijacking in android. In: 24th USENIX Security Symposium (USENIX Security 15). USENIX, pp 945–959
Rinaudo J, Heguiabehere J (2016) Marvin Static Analyzer. https://github.com/programa-stic/Marvin-static-Analyzer, Accessed: 21-Nov-2017
Rogue Wave Software (2017) Klocwork. https://www.roguewave.com/products-services/klocwork/detection/android, Accessed: 01-June-2018
Ryohei K (2014) CVE-2014-5319: Directory traversal vulnerability. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-5319, Accessed: 29-May-2018
Ryoji T (2014) CVE-2014-0806: Geolocation disclosure. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0806, Accessed: 29-May-2018
Sadeghi A, Bagheri H, Garcia J, Malek S (2017) A taxonomy and qualitative comparison of program analysis techniques for security assessment of android software. IEEE Trans Softw Eng 43(6):492–530
Satoru T (2014) CVE-2014-1977: Weak Permissions. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-1977, Accessed: 29-May-2018
Shen F, Vishnubhotla N, Todarka C, Arora M, Dhandapani B, Lehner EJ, Ko SY, Ziarek L (2014) Information flows as a permission mechanism. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering. ACM, pp 515–526. http://blueseal.cse.buffalo.edu/multiflow.html, Accessed : 01-June-2018
Sounthiraraj D, Sahs J, Greenwood G, Lin Z, Khan L (2014) Smv-hunter: Large scale, automated detection of ssl/tls man-in-the-middle vulnerabilities in android apps. In: Inproceedings of the 21st Annual Network and Distributed System Security Symposium (NDSS’14). https://github.com/utds3lab/SMVHunter, Accessed: 10-Jun-2018
Sufatrio, Tan DJJ, Chua TW, Thing VLL (2015) Securing android: a survey, taxonomy, and challenges. ACM Comput Surv 47(4):58:1–58:45
Tendulkar V, Enck W (2014) An application package configuration approach to mitigating android SSL vulnerabilities. CoRR arXiv:http://arXiv.org/abs/1410.7745
van der Veen V, Rossow C (2013) Tracedroid. http://tracedroid.few.vu.nl/, Accessed: 01-May-2018
Wang T (2014a) CVE-2014-8507: SQL Injection vulnerability. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-8507, Accessed: 29-May-2018
Wang T (2014b) CVE-2014-8609: Android Settings application privilege leakage vulnerability. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-8609, Accessed: 29-May-2018
Wei F, Roy S, Robby OX (2014) Amandroid: A precise and general inter-component data flow analysis framework for security vetting of android apps. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. ACM, pp 1329–1341. http://pag.arguslab.org/argus-saf, Accessed: 05-May-2018
Wei F (2017) ICC-Bench. https://github.com/fgwei/ICC-Bench, Accessed: 12-Sep-2018
Wei F, Roy S, Ou X (2018) Robby Amandroid: A precise and general inter-component data flow analysis framework for security vetting of android apps. ACM Trans Privacy Secur 21(3):14:1–14:32
Xu R, Saïdi H, Anderson R (2012) Aurasium: Practical policy enforcement for android applications. In: Proceedings of the 21st USENIX Conference on Security Symposium. USENIX, pp 27–27, https://github.com/xurubin/aurasium, Accessed : 01-June-2018
Yu D (2014) CVE-2014-1566: Leak information to SD card. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-1566, Accessed: 29-May-2018
Zhauniarovich Y, Ahmad M, Gadyatskaya O, Crispo B, Massacci F (2015) Stadyna: Addressing the problem of dynamic code updates in the security analysis of android applications. In: Proceedings of the 5th ACM Conference on Data and Application Security and Privacy. ACM, pp 37–48. https://github.com/zyrikby/StaDynA, Accessed: 11-Jun-2018
Zhou Y, Jiang X (2012) Dissecting android malware: Characterization and evolution. In: Proceedings of the 2012 IEEE Symposium on Security and Privacy. IEEE Computer Society, pp 95–109
Acknowledgments
We thank the readers and reviewers for their feedback to improve this manuscript.
We thank Aditya Narkar and Nasik Muhammad Nafi for their help in implementing 17 new benchmarks that are cataloged as Ghera benchmarks for the first time in this paper and were used in the evaluations described in this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Jeffrey C. Carver
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix: A Catalog of Considered Vulnerabilities
Appendix: A Catalog of Considered Vulnerabilities
In this catalog, we briefly describe the 42 vulnerabilities captured in Ghera (along with their canonical references) that were used in this evaluation. Few vulnerabilities have generic references as they were discovered by Ghera authors while reading the security guidelines available as part of Android documentation (Google Inc 2018c). Please refer the work by Mitra and Ranganath (2017a) for details about the repository and the initial set of vulnerabilities.
1.1 A.1 Crypto
Crypto APIs enable Android apps to encrypt and decrypt information and manage cryptographic keys.
- C1
The result of encrypting a message twice using Block Cipher algorithm in ECB mode is the message itself. So, apps using Block Cipher algorithm in ECB mode (explicitly or due to default on Android platform) can leak information (Egele et al. 2013).
- C2
Encryption using Block Cipher algorithm in CBC mode with a constant Initialization Vector (IV) can be broken by recovering the constant IV using plain text attack. So, apps using such encryption can leak information (Egele et al. 2013).
- C3
Password-based encryption (PBE) uses a salt to generate a password-based encryption key. If the salt is constant, the encryption key can be recovered with knowledge about the password. Hence, apps using PBE with constant salt can leak information (Egele et al. 2013).
- C4
Cipher APIs rely on unique keys to encrypt information. If such keys are embedded in the app’s code, then attackers can recover such keys from the app’s code. Hence, such apps are susceptible to both information leak and data injection (Egele et al. 2013).
1.2 A.2 Inter Component Communication (ICC)
Android apps are composed of four basic kinds of components: 1) Activity components display user interfaces, 2) Service components perform background operations, 3) Broadcast Receiver components receive event notifications and act on those notifications, and 4) Content Provider components manage app data. Communication between components in an app and in different apps (e.g., to perform specific actions, share information) is facilitated via exchange of Intent s. Components specify their ability to process specific kinds of intents by using intent-filters.
- I1
Android apps can dynamically register broadcast receivers at runtime. Such receivers are automatically exported without any access restrictions and, hence, can be accessed via ICC and exploited to perform unintended actions (Chin et al. 2011).
- I2
A component can use a pending intent to allow another component to act on its behalf. When a pending intent is empty (i.e., does not specify an action), it can be seeded (via interception) with an unintended action to be executed on behalf of the originating component (Wang 2014b).
- I3
To perform an action (e.g., send email), users choose an activity/app from a list of activities ordered by priority. By using appropriate priorities, activities can gain unintended privilege over other activities (Chin et al. 2011).
- I4
Implicit intents are processed by any qualifying service determined by intent filters as opposed to a specific explicitly named service. In case of multiple qualifying services, the service with the highest priority processes the intent. By registering appropriate intent filters and by using appropriate priorities, services can gain unintended access to implicit intents (Chin et al. 2011).
- I5
Pending intents can contain implicit intents. Hence, services processing implicit intents contained in pending intents are vulnerable like in I4 (Wang 2014b).
- I6
Apps can use path-permissions to control access to data exposed by content provider. These permissions control access to a folder and not to its subfolders/descendants and their contents. Incorrectly assuming the extent of these permissions can lead to information leak (read) and data injection (write) (Google Inc 2018c).
- I7
Components that process implicit intents are by default exported without any access restrictions. If such a component processes intents without verifying their authenticity (e.g., source) and handles sensitive information or performs sensitive operations in response to implicit intents, then it can leak information or perform unintended actions (Chin et al. 2011).
- I8
Broadcast receivers registered for system intents (e.g., low memory) from Android platform are by default exported without any access restrictions. If such a receiver services intents without verifying the authenticity of intents (e.g., requested action), then it may be vulnerable like the components in I7 (Chin et al. 2011).
- I9
Broadcast receivers respond to ordered broadcasts in the order of priority. By using appropriate priorities, receivers can modify such broadcasts to perform unintended actions (Chin et al. 2011).
- I10
Sticky broadcast intents are delivered to every registered receiver and saved in the system to be delivered to receivers that register in the future. When such an intent is re-broadcasted with modification, it replaces the saved intent in the system, which can lead to information leak and data injection (Chin et al. 2011).
- I11
Every activity is launched in a task, a collection (stack) of activities. An activity can declare its affinity to be started in a specific task under certain conditions. When the user navigates away from an activity X via the back button, the activity below X on X’s task is displayed. This behavior along with task affinity and specific start order — malicious activity started before benign activity — can be used to mount a phishing attack (Ren et al. 2015).
- I12
When an activity from a task in the background (i.e., none of its activities are being displayed) is resumed, the activity at the top of the task (and not the resumed activity) is displayed. This behavior along with task affinity and specific start order — malicious activity started after benign activity — can be used to mount a phishing attack (Ren et al. 2015).
- I13
When a launcher activity is started, its task is created and it is added as the first activity of the task. If the task of a launcher activity already exists with other activities in it, then its task is brought to the foreground, but the launcher activity is not started. This behavior along with task affinity and specific start order — malicious activity started before benign activity — can be used to mount a phishing attack (Ren et al. 2015).
- I14
In addition to declaring task affinity, an activity can request that it be moved to the affine task when the task is created or moved to the foreground (known as task reparenting). This behavior along with task affinity and specific start order — malicious activity started before benign activity — can be used to mount a denial-of-service or a phishing attack (Ren et al. 2015).
- I15
Apps can request permission to perform privileged operations (e.g., send SMS) and offer interfaces (e.g., broadcast receiver) thru which these privileged operations can be triggered. Unless appropriately protected, these interfaces can be exploited to perform an operation without sufficient permissions (Chin et al. 2011).
- I16
The call method of Content Provider API can be used to invoke any provider-defined method. With reference to a content provider, this method can be invoked without any restriction leading to both information leak and data injection (Google Inc 2018b).
1.3 A.3 Networking
Networking APIs allow Android apps to communicate over the network via multiple protocols.
- N1
Apps can open server sockets to listen to connections from clients; typically, remote servers. If such sockets are not appropriately protected, then they can lead to information leak and data injection (Jia et al. 2017).
- N2
Apps can communicate with remote servers via insecure TCP/IP connections. Such scenarios are susceptible to MitM attacks (Jia et al. 2017).
1.4 A.4 Permissions
In addition to system-defined permissions, Android apps can create and use custom permissions. These permissions can be combined with the available four protection levels (i.e., normal, dangerous, signature, signatureOrSystem) to control access to various features and services.
- P1
Permissions with normal protection level are automatically granted to requesting apps during installation. Consequently, any component or its interface protected by such “normal” permissions will be accessible to every installed app (Satoru 2014).
1.5 A.5 Storage
Android provides two basic options to store app data.
- 1.
Internal Storage is best suited to store files private to apps. Every time an app is uninstalled, its internal storage is purged. Starting with Android 7.0 (API 24), files stored in internal storage cannot be shared with and accessed by other apps (Google Inc 2018c).
- 2.
External Storage is best suited to store files that are to be shared with other apps or persisted even after an app is uninstalled. While public directories are accessible by all apps, app-specific directories are accessible only by corresponding apps or other apps with appropriate permission (Google Inc 2018c).
- S1
Files stored in public directories on external storage can be accessed by an app with appropriate permission to access external storage. This aspect can be used to tamper data via data injection (Google Inc 2018c).
- S2
The same aspect from S1 can lead to information leak (Google Inc 2018c).
- S3
Apps can accept paths to files in the external storage from external sources and use them without sanitizing them. A well-crafted file path can be used to read, write, or execute files in the app’s private directory on external storage (directory traversal attack) (Ryohei 2014).
- S4
Apps can copy data from internal storage to external storage. As a result, information could be leaked if such apps accept input from untrusted sources to determine the data to be copied (Yu 2014).
- S5
SQLiteDatabase.rawQuery() method can be used by apps to serve data queries. If such uses rely on external inputs and use non-parameterized SQL queries, then they are susceptible to sql injection attacks (Wang 2014a).
- S6
Content Provider API support selectionArgs parameter in various data access operations to separate selection criteria and selection parameters. App that do not use this parameter are be susceptible to sql injection attacks (Wang 2014a).
1.6 A.6 System
System APIs enable Android apps to access low-level features of the Android platform like process management, thread management, runtime permissions, etc.
Every Android app runs in its own process with a unique Process ID (PID) and a User ID (UID). All components in an app run in the same process. A permission can be granted to an app at installation time or at run time. All components inherit the permissions granted to the containing app at installation time. If a component in an app is protected by a permission, only components that have been granted this permission can communicate with the protected component.
- Y1
During IPC, checkCallingOrSelfPermission method can be used to check if the calling process or the called process has permission P. If a component with permission P uses this method to check if the calling component has permission P, then improper use of this method can leak privilege when the calling component does not have permission P (Google Inc 2018a).
- Y2
checkPermission method can be used to check if the given permission is allowed for the given PID and UID pair. getCallingPID and getCallingUID methods of Binder API can be used to retrieve the PID and UID of the calling process. In certain situations, they return PID/UID of the called process. So, improper use of these methods by a called component with given permission can leak privilege (Google Inc 2018a).
- Y3
During IPC, enforceCallingOrSelfPermission method can be used to check if the calling process or the called process has permission P. Like in Y1, improper use of this method can leak privilege (Google Inc 2018a).
- Y4
enforcePermission method can be used to check if the given permission is allowed for the given PID and UID pair. Like in Y2, improper use of this method along with getCallingPID and getCallingUID can leak privilege (Google Inc 2018a).
1.7 A.7 Web
Web APIs allow Android apps to interact with web servers both with and without SSL/TLS, display web content through WebView widget, and control navigation between web pages via WebViewClient class.
- W1
Apps connecting to remote servers via HTTP (as opposed to HTTPS) are susceptible to information theft via Man-in-the-Middle (MitM) attacks (Tendulkar and Enck 2014).
- W2
Apps can employ HostnameVerifier interface to perform custom checks on hostname when using SSL/TLS for secure communication. If these checks are incorrect, apps can end up connecting to malicious servers and be targets of malicious actions (Tendulkar and Enck 2014).
- W3
In secure communication, apps employ TrustManager interface to check the validity and trustworthiness of presented certificates. Like in W1, if these checks are incorrect, apps can end up trusting certificates from malicious servers and be targets of malicious actions (Tendulkar and Enck 2014).
- W4
Intents can be embedded in URIs. Apps that do not handle such intents safely (e.g., check intended app) can leak information (Google Inc 2017).
- W5
Web pages can access information local to the device (e.g., GPS location). Apps that allow such access without explicit user permission can leak information (Ryoji 2014).
- W6
When WebView is used to display web content, JavaScript code executed as part of the web content is executed with the permissions of the host app. Without proper checks, malicious JavaScript code can get access to the app’s resources, e.g. private files (Chin and Wagner 2013).
- W7
When loading content over a secure connection via WebView, host app is notified of SSL errors via WebViewClient. Apps ignoring such errors can enable MitM attacks (Tendulkar and Enck 2014).
- W8
When a web resource (e.g., CSS file) is loaded in WebView, the load request can be validated in shouldInterceptRequest method of WebViewClient. Apps failing to validate such requests can allow loading of malicious content (Chin and Wagner 2013).
- W9
When a web page is loaded into WebView, the load request can be validated in shouldOverridUrlLoading method of WebViewClient. Apps failing to validate such requests can allow loading of malicious content (Chin and Wagner 2013).
Rights and permissions
About this article
Cite this article
Ranganath, VP., Mitra, J. Are free Android app security analysis tools effective in detecting known vulnerabilities?. Empir Software Eng 25, 178–219 (2020). https://doi.org/10.1007/s10664-019-09749-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-019-09749-y