Advanced Software Obfuscation Techniques and Applications

Open Access
Author:
Wang, Pei
Graduate Program:
Information Sciences and Technology
Degree:
Doctor of Philosophy
Document Type:
Dissertation
Date of Defense:
May 10, 2018
Committee Members:
  • Dinghao Wu, Dissertation Advisor
  • Dinghao Wu, Committee Chair
  • Peng Liu, Committee Member
  • Sencun Zhu, Committee Member
  • Danfeng Zhang, Outside Member
Keywords:
  • obfuscation
  • software protection
  • reverse engineering
  • empirical study
  • mobile apps
Abstract:
Obfuscation is an important software protection technique that prevents automated or human analyses from revealing the internal design and implementation details of software. There has been a strong demand for advanced obfuscation techniques from software vendors to confront threats like intellectual property thefts and cybersecurity attacks. This dissertations approaches the software protection problem through obfuscation in three different aspects, i.e., techniques, applications, and experiences. The dissertation first introduces translingual obfuscation, a novel software obfuscation technique that makes programs obscure by “misusing” certain features of programming languages derived from highly abstract computation theories. For programs written in imperative languages, which are popular but relatively easy to reverse engineer, translingual obfuscation translates part of a program to another language which has a much more complicated programming paradigm and execution model, thus increasing program complexity. The evaluation shows that this advanced obfuscation technique is suitable for protecting software in desktop and server computation environments. It provides effective and stealthy obfuscation effects with only modest performance cost, compared to one of the most popular commercial obfuscators on the market. As for mobile software, its development, deployment, and execution are significantly different from those of traditional desktop software, while must less is known about the practice of software protection on this emerging platform. Therefore, the dissertation takes a first step to systematically studying the applications of software obfuscation techniques in mobile app development. With the help of an automated but coarse-grained method, we computed the likelihood of an app being obfuscated for over a million app samples crawled from Apple App Store. We then inspected the top 6600 most likely obfuscated instances and managed to identify 601 obfuscated versions of 539 iOS apps. By analyzing this sample set with intensive manual effort, we made various observations that help reveal the status quo of mobile obfuscation in the real world. As such, the dissertation can provide insights into understanding and improving the situation of software protection on mobile platforms. Finally, the dissertation reports field experience of applying obfuscation to multiple commercial mobile apps, each of which serves millions of users. In this case study, we leveraged the knowledge learned from the empirical study. The dissertation discusses the challenges of software obfuscation on the iOS platform and our efforts in overcoming these obstacles. This report can benefit many stakeholders in the mobile ecosystem, including developers, security service providers, and administrators of mobile software ecosystems such as Apple and Google.