Enforcing Execution Integrity for Software Systems
Open Access
- Author:
- Ge, Xinyang
- Graduate Program:
- Computer Science and Engineering
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- August 01, 2016
- Committee Members:
- Trent Jaeger, Dissertation Advisor/Co-Advisor
Trent Jaeger, Committee Chair/Co-Chair
Anand Sivasubramaniam, Committee Member
Gang Tan, Committee Member
Peng Liu, Outside Member
Danfeng Zhang, Committee Member
Mathias Payer, Special Member - Keywords:
- Security
Operating System - Abstract:
- Memory corruption bugs have been the most common cause of security vulnerabilities for decades. Adversaries exploit such program flaws to launch code-injection and code-reuse attacks. Code-injection attacks execute instructions that are injected by an adversary at runtime while code-reuse attacks redirect the execution to existing code of her choice. Both attacks can lead to severe security breaches because they often enable arbitrary code execution. To prevent adversaries from exploiting such vulnerabilities, researchers have proposed mitigation techinques such as data execution prevention (DEP), which prohibits execution over data memory hence injected code, and control-flow integrity (CFI), which limits adversaries’ choices when reusing existing code. Despite the effectiveness of these defenses, they can be circumvented by a compromised operating system kernel, which has full privileges over the running system and is capable of negating the deployed defenses. In this dissertation, we explore solutions to protecting an entire software system with both DEP and CFI, which we refer as execution integrity. We take three steps to approach this goal. First, we propose a lightweight system to enforce DEP for the operating system kernel from an isolated computation environment. We mediate the kernel’s memory management operations and ensure they always comply with a set of general principles for lifetime code integrity. Second, we demonstrate a systematic approach to enforce fine-grained CFI for the operating system kernel comprehensively and efficiently. We find that fine-grained CFI can be enforced as efficiently as (or even outperform) comparable coarse-grained CFI. Third, we examine the effectiveness of applying recent hardware mechanisms that log complete control-flow traces to enforce CFI system-wide. We construct an online CFI enforcement system that is capable of enforcing various types of CFI policies over all running, unmodified user-space applications. We intensively optimize the implementation to maximize the performance using the current hardware, and study alternative hardware logging schemes for further performance improvements. The dissertation shows that execution integrity can be enforced efficiently, flexibly and comprehensively, thus is practical to mitigate memory corruption attacks for the whole software system.