Open Access
Wang, Yan
Graduate Program:
Information Sciences and Technology
Master of Science
Document Type:
Master Thesis
Date of Defense:
June 26, 2017
Committee Members:
  • Dinghao Wu, Thesis Advisor
  • Software Security
  • Control Flow Obfuscation
  • Turing Machine
Obfuscation is an important technique to protect software from adversary analysis. Control flow obfuscation effectively prevents attackers from understanding the program structure, hence impeding a broad set of reverse engineering activities. In this thesis, a novel control flow obfuscation method is proposed. It employs Turing machines to simulate the computation of branch conditions. By weaving the original program with Turing machine components, program control flow graph and call graph would become more complex. Moreover, due to the computation complexity of a Turing machine, program execution flow would become much more complicated and resilient to advanced reverse engineering approaches through symbolic execution and concolic testing. A prototype tool based on the proposed technique is implemented. Comparing with previous work, the proposed control flow obfuscation technique bears three distinct advantages. 1). Complexity: the complicated implementation of a Turing machine makes it hard for attackers to understand the program control flow struc- ture. 2). Universality: theoretically, Turing machines can encode any computation. The obfuscation is built on top of the LLVM intermediate representation so the application scope is broadened to almost every language with an LLVM front-end compiler. 3). Resiliency: our obfuscation is shown to be very resilient to advanced analysis tools. We have evaluated the method in terms of functionality correct- ness, potency, resilience, stealth, and cost, respectively. The experimental results show that the proposed technique can obfuscate programs in stealth with good performance and robustness.