Publications

Operating System Security, Malware Analysis, and Memory Forensics


Chariacterizing Kernel Malware Behavior with Kernel Data Access Patterns

Junghwan Rhee, Zhiqiang Lin, Dongyan Xu

To appear in Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security (ASIACCS'11), Hong Kong, March 2011 (Acceptance Rate 16.1%)

Abstract

Characterizing malware’s behavior using its sequential code information faces several challenges, such as obfuscation techniques and the variations in the observed malware behavior due to dynamic runtime execution. We introduce here a new approach that characterizes kernel malware’s behavior by using kernel data access patterns unique to the malware. This approach does not use malware’s control flow, and the code information specific to the malware is generalized. Thus, the malware signature generated by using such data access patterns is resilient in matching malware variants. To evaluate the effectiveness of this approach, we generated the signatures of three classic rootkits using their data access patterns and matched them with a group of benign kernel execution instances (kernel runs) and malicious kernel runs with 16 different rootkits. The signatures did not trigger any false positives in benign kernel runs; however, all kernel runs compromised by rootkits were detected due to the data access patterns shared with the compared signature(s). We further observed high similarities in the signatures of the tested rootkits and exposed popular rootkit attack operations by ranking and listing common data behavior across rootkits. Our experiments show that this approach can be effective not only to detect the malware whose signature is available, but also to determine a variety of rootkits which share kernel data access patterns.

Paper: [pdf]

Slides: [slides]

Bibtex: [bibtex]

[HOME] [BACK]

Kernel Malware Analysis with Un-tampered and Temporal Views of Dynamic Kernel Memory

Junghwan Rhee, Ryan Riley, Dongyan Xu, and Xuxian Jiang

In Proceedings of the 13th International Symposium of Recent Advances in Intrusion Detection (RAID'10) , Ottawa, Canada, September 2010 (Acceptance rate 23.1%)

Abstract

Dynamic kernel memory has been a popular target of recent kernel malware due to the difficulty of determining the status of volatile dynamic kernel objects. Some existing approaches use kernel memory mapping to identify dynamic kernel objects and check kernel integrity. The snapshot-based memory maps generated by these approaches are based on the kernel memory which may have been manipulated by kernel malware. In addition, because the snapshot only reflects the memory status at a single time instance, its usage is limited in temporal kernel execution analysis. We introduce a new runtime kernel memory mapping scheme called allocation-driven mapping, which systematically identifies dynamic kernel objects, including their types and lifetimes. The scheme works by capturing kernel object allocation and deallocation events. Our system provides a number of unique benefits to kernel malware analysis: (1) an un-tampered view wherein the mapping of kernel data is unaffected by the manipulation of kernel memory and (2) a temporal view of kernel objects to be used in temporal analysis of kernel execution. We demonstrate the effectiveness of allocation-driven mapping in two usage scenarios. First, we build a hidden kernel object detector that uses an un-tampered view to detect the data hiding attacks of 10 kernel rootkits that directly manipulate kernel objects (DKOM). Second, we develop a temporal malware behavior monitor that tracks and visualizes malware behavior triggered by the manipulation of dynamic kernel objects. Allocation-driven mapping enables a reliable analysis of such behavior by guiding the inspection only to the events relevant to the attack.

Paper: [pdf]

Slides: [slides]

Bibtex: [bibtex]

Demo Video (YouTube): Please maximize the screen for a clearer view.

Demo Video (Downloadable): [avi]

[HOME] [BACK]

Defeating Dynamic Data Kernel Rootkit Attacks via VMM-based Guest Transparent Monitoring

Junghwan Rhee, Ryan Riley, Xuxian Jiang, and Dongyan Xu

In Proceedings of the 4th International Conference on Availability, Reliability and Security (ARES'09), March 2009 (Acceptance rate 25%)

Abstract Targeting the operating system kernel, the core of trust in a system, kernel rootkits are able to compromise the entire system, placing it under malicious control, while eluding detection efforts. Within the realm of kernel rootkits, dynamic data rootkits are particularly elusive due to the fact that they attack only data targets. Dynamic data rootkits avoid code injection and instead use existing kernel code to manipulate kernel data. Because they do not execute any new code, they are able to complete their attacks without violating kernel code integrity. We propose a prevention solution that blocks dynamic data kernel rootkit attacks by monitoring kernel memory access using virtual machine monitor (VMM) policies. Although the VMM is an external monitor, our system preemptively detects changes to monitored kernel data states and enables fine-grained inspection of memory accesses on dynamically changing kernel data. In addition, readable and writable kernel data can be protected by exposing the illegal use of existing code by dynamic data kernel rootkits. We have implemented a prototype of our system using the QEMU VMM. Our experiments show that it successfully defeats synthesized dynamic data kernel rootkits in real-time, demonstrating its effectiveness and practicality.

Paper: [pdf]

Bibtex: [bibtex]

[HOME] [BACK]

SigGraph: Brute Force Scanning of Kernel Data Structure Instances Using Graph-based Signatures

Zhiqiang Lin, Junghwan Rhee, Xiangyu Zhang, Dongyan Xu, and Xuxian Jiang

To appear in Proceedings of the 18th Network and Distributed System Security Symposium (NDSS'11), San Diego, CA, February 2011 (Acceptance rate 20.1%)

Abstract

Brute force scanning of kernel memory images for finding kernel data structure instances is an important function in many computer security and forensics applications. Brute force scanning requires effective, robust signatures of kernel data structures. Existing approaches often use the value invariants of certain fields as data structure signatures. However, they do not fully exploit the rich points-to relations between kernel data structures. In this paper, we show that such points-to relations can be leveraged to generate graph-based structural invariant signatures. More specifically, we develop SigGraph, a framework that systematically generates non-isomorphic signatures for data structures in an OS kernel. Each signature is a graph rooted at a subject data structure with its edges reflecting the points-to relations with other data structures. Our experiments with a range of Linux kernels show that SigGraph-based signatures achieve high accuracy in recognizing kernel data structure instances via brute force scanning. We further show that SigGraph achieves better robustness against pointer value anomalies and corruptions, without requiring global memory mapping and object reachability. We demonstrate that SigGraph can be applied to kernel memory forensics, kernel rootkit detection, and kernel version inference.

Paper: [pdf]

Bibtex: [bibtex]

Demo (YouTube): Please maximize the screen for a clearer view.

Demo (Downloadable): [avi]

[HOME] [BACK]



Virtualization, Cloud Computing, and System Reliability


DeskBench: Flexible Virtual Desktop Benchmarking Toolkit

Junghwan Rhee, Andrzej Kochut, and Kirk Beaty

In Proceedings of the 11th IFIP/IEEE International Symposium on Integrated Network Management (Mini-conference) (IM'09), June 2009

Abstract

The thin-client computing model has been recently regaining popularity in a new form known as the virtual desktop. That is where the desktop is hosted on a virtualized platform. Even though the interest in this computing paradigm is broad there are relatively few tools and methods for benchmarking virtual client infrastructures. We believe that developing such tools and approaches is crucial for the future success of virtual client deployments and also for objective evaluation of existing and new algorithms, communication protocols, and technologies. We present DeskBench, a virtual desktop benchmarking tool, that allows for fast and easy creation of benchmarks by simple recording of the user's activity. It also allows for replaying the recorded actions in a synchronized manner at maximum possible speeds without compromising the correctness of the replay. The proposed approach relies only on the basic primitives of mouse and keyboard events as well as screen region updates which are common in window manager systems. We have implemented a prototype of the system and also conducted a series of experiments measuring responsiveness of virtual machine based desktops under various load conditions and network latencies. The experiments illustrate the flexibility and accuracy of the proposed method and also give some interesting insights into the scalability of virtual machine based desktops.

Paper: [pdf]

Bibtex: [bibtex]

[HOME] [BACK]

Autonomic Live Adaptation of Virtual Computational Environments in a Multi-Domain Infrastructure

Paul Ruth, Junghwan Rhee, Dongyan Xu, Rick Kennell, and Sebastien Goasguen

In Proceedings of the 3rd IEEE International Conference on Autonomic Computing (ICAC'06), June 2006 (Acceptance rate 21.6%)

Abstract

A shared distributed infrastructure is formed by federating computation resources from multiple domains. Such shared infrastructures are increasing in popularity and are providing massive amounts of aggregated computation resources to large numbers of users. Meanwhile, virtualization technologies, at machine and network levels, are maturing and enabling mutually isolated virtual computation environments for executing arbitrary parallel/distributed applications on top of such a shared physical infrastructure. In this paper, we go one step further by supporting autonomic adaptation of virtual computation environments as active, integrated entities. More specifically, driven by both dynamic availability of infrastructure resources and dynamic application resource demand, a virtual computation environment is able to automatically relocate itself across the infrastructure and scale its share of infrastructural resources. Such autonomic adaptation is transparent to both users of virtual environments and administrators of infrastructures, maintaining the look and feel of a stable, dedicated environment for the user. As our proofof- concept, we present the design, implementation, and evaluation of a system called VIOLIN, which is composed of a virtual network of virtual machines capable of live migration across a multi-domain physical infrastructure.

Paper: [pdf]

Bibtex: [bibtex]

[HOME] [BACK]

Short Paper: Autonomic Adaptation of Virtual Distributed Environments in a Multi-Domain Infrastructure

Dongyan Xu, Paul Ruth, Junghwan Rhee, Rick Kennell, and Sebastien Goasguen

In Proceedings of the 15th IEEE International Symposium on High Performance Distributed Computing (HPDC'06), June 2006 (Acceptance rate 14%)

Abstract

By federating resources from multiple domains, a shared infrastructure provides aggregated computation resources to a large number of users. With rapid advances in virtualization technologies, we propose the concept of virtual distributed environments as a new sharing paradigm for a multi-domain shared infrastructure. Such virtual environments provide users with confined, customized platforms to execute legacy parallel/ distributed applications. Furthermore, we propose to support autonomic adaptation of virtual distributed environments, driven by both dynamic availability of infrastructure resources and dynamic application resource demand. We identify new research challenges and describe our on-going work and preliminary results.

Paper: [pdf]

Bibtex: [bibtex]

[HOME] [BACK]

Mondrix: Memory Isolation for Linux using Mondriaan Memory Protection

Emmett Witchel, Junghwan Rhee, and Krste Asanović

In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05), October 2005 (Acceptance rate 13%)

Abstract

This paper presents the design and an evaluation of Mondrix, a version of the Linux kernel with Mondriaan Memory Protection (MMP). MMP is a combination of hardware and software that provides efficient fine-grained memory protection between multiple protection domains sharing a linear address space. Mondrix uses MMP to enforce isolation between kernel modules which helps detect bugs, limits their damage, and improves kernel robustness and maintainability. During development, MMP exposed two kernel bugs in common, heavily-tested code, and during fault injection experiments, it prevented three of five file system corruptions. The Mondrix implementation demonstrates how MMP can bring memory isolation to modules that already exist in a large software application. It shows the benefit of isolation for robustness and error detection and prevention, while validating previous claims that the protection abstractions MMP offers are a good fit for software. This paper describes the design of the memory supervisor, the kernel module which implements permissions policy. We present an evaluation of Mondrix using full-system simulation of large kernel-intensive workloads. Experiments with several benchmarks where MMP was used extensively indicate the additional space taken by the MMP data structures reduce the kernel’s free memory by less than 10%, and the kernel’s runtime increases less than 15% relative to an unmodified kernel.

Paper: [pdf]

Bibtex: [bibtex]

[HOME] [BACK]

TraceBack: First-Fault Diagnosis by Reconstruction of Distributed Control Flow

A. Ayers, R. Schooler, A. Agarwal, C. Metcalf, J. Rhee, and E. Witchel

In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI'05), Chicago, IL, June 2005 (Acceptance rate 21%)

Abstract

Faults that occur in production systems are the most important faults to fix, but most production systems lack the debugging facilities present in development environments. TraceBack provides debugging information for production systems by providing execution history data about program problems (such as crashes, hangs, and exceptions). TraceBack supports features commonly found in production environments such as multiple threads, dynamically loaded modules, multiple source languages (e.g., Java applications running with JNI modules written in C++), and distributed execution across multiple computers. TraceBack supports first fault diagnosis—discovering what went wrong the first time a fault is encountered. The user can see how the program reached the fault state without having to re-run the computation; in effect enabling a limited form of a debugger in production code. TraceBack uses static, binary program analysis to inject lowoverhead runtime instrumentation at control-flow block granularity. Post-facto reconstruction of the records written by the instrumentation code produces a source-statement trace for user diagnosis. The trace shows the dynamic instruction sequence leading up to the fault state, even when the program took exceptions or terminated abruptly (e.g., kill -9). We have implemented TraceBack on a variety of architectures and operating systems, and present examples from a variety of platforms. Performance overhead is variable, from 5% for Apache running SPECweb99, to 16%–25% for the Java SPECJbb benchmark, to 60% average for SPECint2000. We show examples of TraceBack’s cross-language and cross-machine abilities, and report its use in diagnosing problems in production software.

Paper: [pdf]

Bibtex: [bibtex]

[HOME] [BACK]