Home
Resource
Automated detection and containment of stealth attacks on the operating system kernel
PDF
PDF format is widely accepted and good for printing.
Plug-in required
PDF-1(1.12 MB)
Citation & Export
View Usage Statistics
Staff View

Citation & Export
Hide

Simple citation

Baliga, Arati. Automated detection and containment of stealth attacks on the operating system kernel. Retrieved from https://doi.org/doi:10.7282/T33B60FK

Export

Click here for information about Citation Management Tools at Rutgers.

Statistics
Hide

Description

TitleAutomated detection and containment of stealth attacks on the operating system kernel
NameBaliga, Arati (author); Iftode, Liviu (chair); Ganapathy, Vinod (internal member); Trappe, Wade (internal member); Jaeger, Trent (outside member); Rutgers University; Graduate School - New Brunswick
Date Created
Other Date2009-01 (degree)
SubjectComputer Science, Computer security, Data protection
Extentxii, 105 p. : ill.
DescriptionThe operating system kernel serves as the root of trust for all applications running on the computer system. A compromised system can be exploited by remote attackers stealthily, such as exfiltration of sensitive information, wasteful usage of the system's resources, or involving the system in malicious activities without the user's knowledge or permission. The lack of appropriate detection tools allows such systems to stealthily lie within the attackers realm for indefinite periods of time. Stealth attacks on the kernel are carried out by malware commonly known as rootkits. The goal of the rootkit is to conceal the presence of the attacker on the victim system. Conventionally, kernel rootkits modified the kernel to achieve stealth, while most functionality was provided by accompanying user space programs. The newer kernel rootkits achieve the malice and stealth solely by modifying kernel data. This dissertation explores the threat posed by both types of kernel rootkits and proposes novel automated techniques for their detection and containment. Our first contribution is an automated containment technique built using the virtualization architecture. This technique counters the ongoing damage done to the system by the conventional kernel rootkits. It is well suited for attacks that employ kernel or user mode stealth but provide most of the malicious functionality as user space programs. Our second contribution is to identify a new class of stealth attacks on the kernel, which do not exhibit explicit hiding behavior but are stealthy by design. They achieve their malicious objectives by solely modifying data within the kernel. These attacks demonstrate that the threat posed to kernel data is systemic requiring comprehensive protection.
Our final contribution is a novel automated technique that can be used for detection of such stealth data-centric attacks. The key idea behind this technique is to automatically identify and extract invariants exhibited by kernel data structures during a training phase. These invariants are used as specifications of data structure integrity and are enforced during runtime. Our technique could successfully detect all rootkits that were publicly available. It could also detect more recent stealth attacks developed by us or proposed by
other recent research literature.
NotePh.D.
NoteIncludes bibliographical references (p. 99-103)
Noteby Arati Baliga
Genretheses, ETD doctoral
Persistent URLhttps://doi.org/doi:10.7282/T33B60FK
Languageeng
CollectionGraduate School - New Brunswick Electronic Theses and Dissertations
Organization NameNjNbRU
RightsThe author owns the copyright to this work.
Version 8.5.5
Rutgers University Libraries - Copyright ©2024