Malware analysis using reverse engineering and data mining tools

TL;DR: A case study of the well-known Nugache worm is presented using existing reverse engineering tools to collect data from malwares running in a closed-lab environment and the proposed approach can be used for the study of malware behaviors in a safe and pedagogical environment.

Abstract: One challenge in malware analysis involves collecting useful data without risking experimenters' machines or systems. Static analysis of malware codebases is valuable in providing insights on malware development mechanisms, however, it cannot provide understanding in dynamic profiling of executable codes. In this paper, we present a case study of the well-known Nugache worm using existing reverse engineering tools to collect data from malwares running in a closed-lab environment. Useful dynamic patterns of malwares are generated by using a rough set based machine learning tool. The proposed approach can be used for the study of malware behaviors in a safe and pedagogical environment. The dynamic patterns generated by data mining tools may provide insights for specifying similarity measures used by network level Intrusion Detection Systems.