Ghost turns Zombie: Exploring the Life Cycle of Web-based Malware

from LEET 08, a paper that i found entirely disappointing. all it is is a collection of numbers without much interpretation. the field is significantly more complex than you can allow in 8 pages, but the authors make no attempt to dig into why they get the results they get.

While the web provides information and services that enrich our lives in many ways, it has also become the primary vehicle for delivering malware. Once infected with web-based malware, an unsuspecting user’s machine is converted into a productive member of the Internet underground. In this work, we explore the life cycle of web-based malware by employing light-weight responders to capture the network profile of infected machines. Our results indicate that web-based malware provides a cornerstone for large scale electronic fraud. It is used to exfiltrate address books of compromised machines creating databases of hundred millions of email addresses, to form spamming botnets responsible for a significant fraction of spam currently seen on the Internet, and also to steal login credentials that can be directly monetized or leveraged to turn more web servers into malware delivery vectors.

We support our findings by providing a broad overview of the post-infection network behavior of web-based malware, as well as in-depth examinations of the botnets and leaked information we found during the course of our study.

source: Ghost turns Zombie: Exploring the Life Cycle of Web-based Malware,Michalis Polychronakis, Panayiotis Mavrommatis, Niels Provos.

Related articles

• Botnet sics zombie soldiers on gimpy websites [via Zemanta]
• Research on Malware Distribution [via Zemanta]

Anti-Unpacker Tricks

from the CARO workshop i attended a week or so ago, this gem from Peter Ferrie. Peter's collected a lot of useful info for anyone working with Win32 protected EXEs.

Unpackers are as old as the packers themselves, but anti-unpacking tricks are a more recent development. These anti-unpacking tricks have developed quickly in number and, in some cases, complexity. In this paper, we will describe some of the most common anti-unpacking tricks, along with some countermeasures.

source: Anti-unpacker Tricks, Peter Ferrie.

Why Writing Your Own Search Engine is Hard

found this in my ACM Queue pile this morning. there's no single silver bullet here, but lots of good principles learned over a few years of working with boatloads of data. i think she's right in most cases based on my (limited) experience with piles of data i wish to search and handle.

There must be 4,000 programmers typing away in their basements trying to build the next "world's most scalable" search engine. It has been done only a few times. It has never been done by a big group; always one to four people did the core work, and the big team came on to build the elaborations and the production infrastructure. Why is it so hard? We are going to delve a bit into the various issues to consider when writing a search engine. This article is aimed at those individuals or small groups that are considering this endeavor for their Web site or intranet. It is fun, but a word of caution: not only is it difficult, but you need two commodities in short supply—time and patience.

source: Why Writing Your Own Search Engine is Hard, from Enterprise Search Vol. 2, No. 2 - April 2004. Anna Patterson, Stanford University.

Data Reduction in Intrusion Alert Correlation

been thinking a lot about data fusion and correlation for years, but a lot more lately. sadly this paper did not help. pretty weak presentation, not a very well developed approach, nor is it very novel.

Network intrusion detection sensors are usually built around low level models of network traffic. This means that their output is of a similarly low level and as a consequence, is difficult to analyze. Intrusion alert correlation is the task of automating some of this analysis by grouping related alerts together. Attack graphs provide an intuitive model for such analysis. Unfortunately alert flooding attacks can still cause a loss of service on sensors, and when performing attack graph correlation, there can be a large number of extraneous alerts included in the output graph. This obscures the fine structure of genuine attacks and makes them more difficult for human operators to discern. This paper explores modified correlation algorithms which attempt to minimize the impact of this attack.

Source: Data Reduction in Intrusion Alert Correlation, Tedesco Gianni, Aickelin Uwe.

Behind Phishing: An Examination of Phisher Modi Operandi

LEET 08 papers are now visible as the workshop is on. i went looking for a few papers in domains i study, hoping to get some more insight. sadly, this paper (the second of the bunch i read, thorsten's was first) really let me down. pretty weak data, very cursory analysis, very lame conclusions. some folks are disconnected - quite visibly disconnected - from the very real world problems they're studying.

Phishing costs Internet users billions of dollars a year. Using various data sets collected in real-time, this paper analyzes various aspects of phisher modi operandi. We examine the anatomy of phishing URLs and domains, registration of phishing domains and time to activation, and the machines used to host the phishing sites. Our findings can be used as heuristics in filtering phishing-related e-mails and in identifying suspicious domain registrations.

Source: Behind Phishing: An Examination of Phisher Modi Operandi, D. Kevin McGrath, Minaxi Gupta.

Measurements and Mitigation of Peer-to-Peer-based Botnets: A Case Study on StormWorm

one of the papers i read this week. i was frustrated that the authors didn't get more pages to discuss their work and instead had to fit more background material in there.

Botnets, i.e., networks of compromised machines under a common
control infrastructure, are commonly controlled by an attacker
with the help of a central server: all compromised machines
connect to the central server and wait for commands.

However, the first botnets that use peer-to-peer (P2P) networks
for remote control of the compromised machines appeared
in the wild recently. In this paper, we introduce a
methodology to analyze and mitigate P2P botnets. In a case
study, we examine in detail the Storm Worm botnet, the most
wide-spread P2P botnet currently propagating in the wild. We
were able to infiltrate and analyze in-depth the botnet, which allows
us to estimate the total number of compromised machines.
Furthermore, we present two different ways to disrupt the communication
channel between controller and compromised machines
in order to mitigate the botnet and evaluate the effectiveness
of these mechanisms.

Source: Measurements and Mitigation of Peer-to-Peer-based Botnets: A Case Study on StormWorm, Thorsten Holz, Moritz Steinery, Frederic Dahl, Ernst Biersacky, Felix Freiling, from a paper to appear at LEET 08.

Characterizing Residential Broadband Networks

with comcast and bittorrent shaping in the news, this study from last year's Internet Measurement Conference workshop is pretty interesting. lots of good traffic measurements about actual bandwidth, its stability, etc.

A large and rapidly growing proportion of users connect to
the Internet via residential broadband networks such as Dig-
ital Subscriber Lines (DSL) and cable. Residential networks
are often the bottleneck in the last mile of today’s Internet.
Their characteristics critically affect Internet applications,
including voice-over-IP, online games, and peer-to-peer con-
tent sharing/delivery systems. However, to date, few studies
have investigated commercial broadband deployments, and
rigorous measurement data that characterize these networks
at scale are lacking.
In this paper, we present the first large-scale measurement
study of ma jor cable and DSL providers in North America
and Europe. We describe and evaluate the measurement
tools we developed for this purpose. Our study character-
izes several properties of broadband networks, including link
capacities, packet round-trip times and jitter, packet loss
rates, queue lengths, and queue drop policies. Our analysis
reveals important ways in which residential networks differ
from how the Internet is conventionally thought to operate.
We also discuss the implications of our findings for many
emerging protocols and systems, including delay-based con-
gestion control (e.g., PCP) and network coordinate systems
(e.g., Vivaldi).

source: Characterizing Residential Broadband Networks, Marcel Dischinger, MPI for Software Systems; Andreas Haeberlen, MPI for Software Systems and Rice University; Krishna P. Gummadi, MPI for Software Systems; Stefan Saroiu, University of Toronto.

Manitou: A Layer-Below Approach to Fighting Malware

i'm skeptical this would work in reality for a variety of reasons. this also smells like hammers (hypervisor-based solutions, memory page tracking) looking for nails (malcode detection).

Unbeknownst to many computer users, their machines are running malware. Others are aware that strange software inhabits their machine, but cannot get rid of it. In this paper, we present Manitou, a system that provides users with the ability to assign, track and revoke execution privileges for code, regardless of the integrity and type of operating system the machine is using.

Manitou is implemented within a hypervisor and uses the per-page permission bits to ensure that any code contained in an executable page corresponds to authorized code. Manitou authenticates code by taking a cryptographic hash of the content of a page right before executing code contained in that page. Our system guarantees that only authorized code can be run on the system.

Source: Manitou: A Layer-Below Approach to Fighting Malware, Lionel Litty and David Lie, in ASID’06 (October 21, 2006, San Jose,
California, USA).

Python Object Sharing (POSH)

i got to looking at concurrency in Python, and this was the most useful thing i could find for my needs.

Python Object Sharing, or POSH for short, is an extension module to Python that allows objects to be placed in shared memory. Objects in shared memory can be accessed transparently, and most types of objects, including instances of user-defined classes, can be shared. POSH allows concurrent processes to communicate simply by assigning objects to shared container objects. On multiprocessor architectures, multi-process applications using POSH can significantly outperform similar multi-threaded applications, since Python threads don't scale to take advantage of multiple processors. Even so, POSH lends itself to a programming model very similar to threads.

From a master's thesis by Steffen Viken Valvåg.

Efficient Sequence Alignment of Network Traffic.

ahh .. sequence alignment. something near and dear to my heart, especially when applied to network payloads. sadly, this isn't quite what i had in mind, but it's a step closer. the code's out, too, via the BRO IDS.

String comparison algorithms, inspired by methods used in bioinformatics, have recently gained popularity in network applications. In this paper we demonstrate the need for careful selection of alignment models if such algorithms are to yield the desired results when applied to network traffic. We introduce a novel variant of the Jacobson-Vo algorithm employing a flexible gap-minimising alignment model suitable for network traffic, and find that our software implementation outperforms the commonly used Smith-Waterman approach by a factor of 33 on average and up to 58.5 in the best case on a wide range of network protocols.

Source: C. Kreibich and J. Crowcroft: Efficient Sequence Alignment of Network Traffic. Internet Measurement Conference (IMC), 2006, Rio de Janeiro, Brazil.

Spamscatter: Characterizing Internet Scam Hosting Infrastructure

from usenix security last year.

Unsolicited bulk e-mail, or SPAM, is a means to an end. For virtually all such messages, the intent is to attract the recipient into entering a commercial transaction — typically via a linked Web site. While the prodigious infrastructure used to pump out billions of such solicitations is essential, the engine driving this process is ultimately the “point-of-sale” — the various money-making “scams” that extract value from Internet users. In the hopes of better understanding the business pressures exerted on spammers, this paper focuses squarely on the Internet infrastructure used to host and support such scams. We describe an opportunistic measurement technique called spamscatter that mines emails in real-time, follows the embedded link structure, and automatically clusters the destination Web sites using image shingling to capture graphical similarity between rendered sites. We have implemented this approach on a large real-time spam feed (over 1M messages per week) and have identified and analyzed over 2,000 distinct scams on 7,000 distinct servers.

Source: Spamscatter: Characterizing Internet Scam Hosting Infrastructure, David S. Anderson, Chris Fleizach, Stefan Savage and Geoffrey M. Voelker.

Spatial-Temporal Characteristics of Internet Malicious Sources

trying to get back to reading a paper a day. so far, 3 days, 3 papers. and my brain is thanking me for it. first up, long term traffic analysis. folks have been doing this kind of thing a lot lately, trying to identify "bad neighborhoods" for blocking those "guilty by association".

This paper presents a large scale longitudinal study of the spatial and temporal features of malicious source addresses. The basis of our study is a 402-day trace of over 7 billion Internet intrusion attempts provided by DShield.org, which includes 160 million unique source addresses. Specifically, we focus on spatial distributions and temporal characteristics of malicious sources. First, we find that one out of 27 hosts is potentially a scanning source among 232 IPv4 addresses. We then show that malicious sources have a persistent, non-uniform spatial distribution. That is, more than 80% of the sources send packets from the same 20% of the IPv4 address space over time. We also find that 7.3% of malicious source addresses are unroutable, and that some source addresses are correlated. Next, we show that most sources have a short lifetime. 57.9% of the source addresses appear only once in the trace, and 90% of source addresses appear less than 5 times. These results have implications for both attacks and defenses.

Source: Chen, Zesheng; Ji, Chuanyi; Barford, Paul. Spatial-Temporal Characteristics of Internet Malicious Sources, To appear in IEEE INFOCOM (Mini-Conference), April, 2008.

Learning to Extract Signature and Reply Lines from Email

automated email analysis for information mining - for yourself, not in any investigatory capacity - has long been an interest of mine. so much gold lying in my inbox if only it could be processed. i know that IBM has spent some time on this, working towards a new product called ReMail, and MSFT and others routinely do email research. however, email clients still suck.

i've been playing around with the stuff described here to see if i can't auto-organize my contacts based solely on their email messages.

We describe methods for automatically identifying signature blocks and reply lines in plain-text email messages. This analysis has many potential applications, such as preprocessing email for text-to-speech systems; anonymization of email corpora; improving automatic content-based mail classifiers; and email threading. Our method is based on applying machine learning methods to a sequential representation of an email message, in which each email is represented as a sequence of lines, and each line is represented as a set of features. We compare several state-of-the-art sequential and non-sequential machine learning algorithms on different feature sets, and present experimental results showing that the presence of a signature block in a message can be detected with accuracy higher than 97%; that signature block lines can be identified with accuracy higher than 99%; and that signature block and reply lines can be simultaneously identified with accuracy of higher than 98%.

Source: Learning to Extract Signature and Reply Lines from Email, Vitor R. Carvalho and William W. Cohen.

Cuckoo Hashing

I found out about this data structure last night while reading programming.reddit.com, which is often a valuable resource. Cuckoo Hashing was described in the comments as a simpler implementation of a solution to the same problem Judy sparse arrays are trying to solve (O(1) lookups and inserts, minimal space usage for even large collections). I found an implementation of the Cuckoo Hash (http://www.tcllab.org/canasai/software/ckhash/ckhash-0.4.1/) and have been writing Python bindings for it, too.

You can read more about the data structure here:

We present a simple dictionary with worst case constant lookup time, equaling the theoretical performance of the classic dynamic perfect hashing scheme of Dietzfel- binger et al. (Dynamic perfect hashing: Upper and lower bounds. SIAM J. Comput., 23(4):738–761, 1994). The space usage is similar to that of binary search trees. Besides being conceptually much simpler than previous dynamic dictionaries with worst case constant lookup time, our data structure is interesting in that it does not use perfect hashing, but rather a variant of open addressing where keys can be moved back in their probe sequences. An implementation inspired by our algorithm, but using weaker hash functions, is found to be quite practical. It is competitive with the best known dictionaries having an average case (but no nontrivial worst case) guarantee on lookup time.

Source: Cuckoo Hashing, Rasmus Pagh and Flemming Friche Rodler, 2003.