Protecting Programs During Resource Access

Open Access
Vijayakumar, Hayawardh
Graduate Program:
Computer Science and Engineering
Doctor of Philosophy
Document Type:
Date of Defense:
February 06, 2014
Committee Members:
  • Trent Ray Jaeger, Dissertation Advisor
  • Patrick Drew Mcdaniel, Committee Chair
  • Bhuvan Urgaonkar, Committee Member
  • Constantino Manuel Lagoa, Special Member
  • Operating Systems
  • Security
  • Resource Access
With the emergence of targeted malware such as Stuxnet and the continued prevalence of spyware and other types of malicious software, host security has become a critical issue. Attackers break into systems through vulnerabilities in network daemons, malicious insiders, or social engineering, and then attempt to escalate privileges to the administrator to gain complete control of the system by exploiting local vulnerabilities. Thus far, such local vulnerabilities have received little attention, and it has been taken for granted that any attacker break-in can easily be escalated to full control. In this dissertation, we identify a class of previously disjoint local vulnerability attack classes that we call resource access attacks, and provide a framework to detect and defend against them. Programs have to fetch resources, such as files from the operating system (OS) to function. However, local adversaries such as spyware also share this namespace of resources, and can trick programs into retrieving an unintended resource using a variety of resource access attacks that make up 10-15% of vulnerabilities reported each year. Such attacks are challenging to defend for a few reasons. First, program checks to defend against such attacks cause a performance overhead, so programmers have an incentive to omit checks altogether. Second, there is a disconnect between the parties involved in resource access. On the one hand, due to this overhead, programmers omit checks under the expectation that the deployment’s access control policy will protect a subset of resources from adversaries. On the other hand, access control policies are framed by OS distributors and system administrators, who in turn have little idea about programmer expectations, causing mismatches with programmer expectations. Third, even when programmers check resource access, such checks are difficult to get right due to inherent races in the system call API. Previous work handles a subset of resource access attacks but in ad-hoc ways. This dissertation takes several steps to address resource access attacks. First, we present a technique for automated evaluation of a program attack surface in its system deployment, where checks for resource access are required. Second, we present a technique that uses this attack surface to detect a subset of resource access attacks. We found more than 25 previously-unknown vulnerabilities across a variety of both mature and new programs in the widely-used Fedora and Ubuntu Linux distributions, proving the prevalence of such vulnerabilities. Third, we present the Process Firewall, a system to defend against resource access attacks in an efficient manner without requiring program code change. Fourth, we propose a technique to automatically derive the programmer-expected attack surface of a program, and generate Process Firewall rules to enforce that the only adversary-controlled resource accesses in the deployment are part of the expected attack surface. The work in this dissertation thus provides a principled starting point to protect programs during resource access, thus reducing the vectors adversaries have to compromise a computer system.