Xuxian Jiang, Xinyuan Wang, Dongyan Xu

CERIAS TR 2007-80

article

An alarming trend in malware attacks is that they are armed with stealthy techniques to detect, evade, and subvert malware detection facilities of the victim. On the defensive side, a fundamental lim- itation of traditional host-based anti-malware systems is that they run inside the very hosts they are protecting (“in the box”), making them vulnerable to counter-detection and subversion by malware. To address this limitation, recent solutions based on virtual ma- chine (VM) technologies advocate placing the malware detection facilities outside of the protected VM (“out of the box”). However, they gain tamper resistance at the cost of losing the native, seman- tic view of the host which is enjoyed by the “in the box” approach, thus leading to a technical challenge known as the semantic gap. In this paper, we present the design, implementation, and evalua- tion of VMwatcher – an “out-of-the-box” approach that overcomes the semantic gap challenge. A new technique called guest view casting is developed to systematically reconstruct internal seman- tic views (e.g., files, processes, and kernel modules) of a VM from the outside in a non-intrusive manner. Specifically, the new tech- nique casts semantic definitions of guest OS data structures and functions on virtual machine monitor (VMM)-level VM states, so that the semantic view can be reconstructed. With the semantic gap bridged, we identify two unique malware detection capabilities: (1) view comparison-based malware detection and its demonstration in rootkit detection and (2) “out-of-the-box” deployment of host- based anti-malware software with improved detection accuracy and tamper-resistance. We have implemented a proof-of-concept pro- totype on both Linux and Windows platforms and our experimen- tal results with real-world malware, including elusiverootkits, demonstrate its practicality and effectiveness.

PDF

Jiang

2001-01-01