Detecting a malicious executable without prior knowledge of its patterns

Dongming Michael Cai; James Theiler; Maya Gokhale

doi:10.1117/12.603174

28 March 2005 Detecting a malicious executable without prior knowledge of its patterns

Dongming Michael Cai, James Theiler, Maya Gokhale

Proceedings Volume 5812, Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security 2005; (2005) https://doi.org/10.1117/12.603174
Event: Defense and Security, 2005, Orlando, Florida, United States

Abstract

To detect malicious executables, often spread as email attachments, two types of algorithms are usually applied under instance-based statistical learning paradigms: (1) Signature-based template matching, which finds unique tell-tale characteristics of a malicious executable and thus is capable of matching those with known signatures; (2) Two-class supervised learning, which determines a set of features that allow benign and malicious patterns to occupy a disjoint regions in a feature vector space and thus probabilistically identifies malicious executables with the similar features. Nevertheless, given the huge potential variety of malicious executables, we cannot be confident that existing training sets adequately represent the class as a whole. In this study, we investigated the use of byte sequence frequencies to profile only benign data. The malicious executables are identified as outliers or anomalies that significantly deviate from the normal profile. A multivariate Gaussian likelihood model, fit with a Principal Component Analysis (PCA), was compared with a one-class Support Vector Machine (SVM) model for characterizing the benign executables. We found that the Gaussian model substantially outperformed the one-class SVM in its ability to distinguish malicious from benign files. Complementing to the capabilities in reliably detecting those malicious files with known or similar features using two aforementioned methods, the one-class unsupervised approach may provide another layer of safeguard in identifying those novel computer viruses.

Citation Download Citation

Dongming Michael Cai, James Theiler, and Maya Gokhale "Detecting a malicious executable without prior knowledge of its patterns", Proc. SPIE 5812, Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security 2005, (28 March 2005); https://doi.org/10.1117/12.603174

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available