A STUDY OF SYSTEM VULNERABILITY AND MALWARE
ON ANDROID

Huang, Heqing

A STUDY OF SYSTEM VULNERABILITY AND MALWARE ON ANDROID

Open Access

Author:

Huang, Heqing

Graduate Program:

Computer Science and Engineering

Degree:

Doctor of Philosophy

Document Type:

Dissertation

Date of Defense:

February 23, 2016

Committee Members:

Sencun Zhu, Dissertation Advisor
Peng Liu, Committee Chair
Thomas La Porta, Committee Member
David Jonathan Miller, Committee Member

Keywords:

Android Security
Vulnerability
Malware

Abstract:

The increasing popularity of mobile devices (e.g., Android, iOS and etc.) attracts both normal users and malware writers. In this dissertation, we conduct research on three important aspects of security problems in Android, which has a lion share (about 80%) of the current mobile market. In the application-level, we perform a comprehensive analysis on the design of top 30 antivirus detectors (AVDs) tailored for Android. One latest comparison of Android AVDs from the independent lab AV-TEST reports that the AVDs have around 95% malware detection rate. This only indicates that current AVDs on Android have good malware signature databases. When the AVDs are deployed on the fast-evolving mobile system, their effectiveness should also be measured on their runtime behavior. Our new understanding of the AVDs’ design leads us to discover the hazards in adopting AVD solutions for Android. First, we measure the seriousness of the discovered hazard in the malware scan operations by developing evasion techniques, which work even under the assumption that the AVDs are equipped with “complete” virus definition files. Second, we discover that, during the engine update operations, the Android system surprisingly nullifies all types of protection of the AVDs and exposes the system to high risks. We design and develop a model checker to confirm the presence of this vulnerable program logic in all versions of Google Android source code and other vendor customized system images. We then report the findings to AVD vendors across 16 countries. In the system-level, we identify and mitigate the system vulnerabilities in Android, which cause serious denial of service (DoS). The System Server (SS) process is considered as the heart of Android, as it contains most of the Android system services in the Android framework, which provides the essential functionalities for applications (apps). However, due to the complicated design of the SS and the easily-accessible nature of its system services (e.g., through Android APIs), we conjecture that the SS may face serious DoS attacks. Through source code analysis, we have discovered a general design pattern in the concurrency control mechanism of the SS that could lead to deadly DoS attacks. As the SS plays the anchor role in Android, these DoS attacks could cause single-point-of-failure in Android. We name it Android Stroke Vulnerability (ASV), as the SS, encounters downtime when the ASV is exploited. We then design an ASV-Hunter to rank the risk level of methods in the SS to cost-efficiently discover four unknown ASVs in critical services of SS. Our further threat analysis result is daunting: by easily writing a loop to invoke Android APIs in an app, an attacker can prevent the user from patching vulnerable banking apps, reboot the device at mission critical moments (e.g., making phone calls). The ASVs can be easily leveraged to design ransomware by putting the device into repeated freezing/rebooting loops or help equip malware with anti-removal capability. Google confirmed our findings immediately after sending them a report. We also proposed defenses to secure the SS. After identifying vulnerabilities in both critical apps and system components of Android, we consider that the vulnerable and fast evolving Android system may be the next target of malware writers. Hence, we are trying to uncover the current status of Android malware development in the real world. We suspect that, during the malware development and testing phase, some Android malware writers are continuously using public scanning services (e.g., VirusTotal “VT”) for testing the evasion capability of their malware samples, which we name Android malware development (AMD) cases. In this work, we designed an AMD hunting system in the context of VT to identify AMD cases and reveal new threats from Android malware development. Our system was implemented and used in a leading security company for four months. It has processed 58 million of Android sample submissions on VT, and identified 1,623 AMD cases with 13,855 samples from 83 countries. We then perform malware analysis and case studies on 890 samples selected from the identified AMD cases. Our case study reveals lots of new malware threats, including fake system app development, new phishing development, new rooting cases, new evasive techniques and etc. Besides raising the awareness of the existence of AMD cases, more importantly, our research provides a generic and scalable framework for the systematic study of AMD cases on malware submission platforms. The relevant samples that we identified will become a fresh Android malware source for the research community.