dress dispersion table- with high probability, the content has appeared frequently enough to be a candidate worm signature. Pseudocode for the main loop of the Early Bird system is shown in Figure 5 baM wal s 444. 12345 ketTle) Gensnr-1 Parks nd (cnomalousADEntry(currentADEntry. packet) t DispTh)) tHash)> PravalenceTh) srclP, dstlP, packet Time) 14 endwhile Figure 7: A screenshot of the main screen of the Early Bire interface. Each zone is labeled by a prefix and shows th n every packet. When rent anoma ammeters he prevalence which can be changed by the user. More detailed screens show contains the source and detailed counts for each anomaly, as shown for Sasser in Figure required for the scaled 12. coordinates real-time updates from the sensors, coalesces The content prevalence table sees the most activity related signatures, activates any network-level or host- in the system and serves as a high-pass filter for fre- level blocking services and is responsible for administra quent content. The multi-stage filter data structure is tive reporting and control. Our implementation is written cleared on a regular interval (60 seconds in our imple- in C and the aggregator also uses the MySql database to mentation).By contrast, the address prevalence table has log all events, the popular rrd-tools library for graphical typically fewer values-only those strings exceeding the reporting, and PHP scripting for administrative control prevalence threshold-and can be garbage collected over A screenshot of the main screen of the early Bird user longer time scales(even hours) interface showing zones and a summary of the current Each of these mechanisms can be implemented at high system activity is shown in Figure 7 speeds in either software or hardware, with relatively Finally, in order to automatically block outbreaks, the modest memory requirements as we quantify in the next Early Bird system automatically generates and deploys section. Moreover, our approach makes no assumptions precise content-based signatures formatted for the Snort about the point of deployment, whether at the endpoint, inline intrusion prevention system [1]. A sample such edge, or core. However the optimal parameters settings signature for Kibvu. B is shown below may depend on the point of deployments. In Section 6 we empirically explore the parameter settings used by (msp: 2112067784 Fra May14 03:51200 2004 Early Bird pr content:"1909090904d3fe37790 909090ff63649090909090";) 6 Experience Based on the content sifting algorithm just described, we 6.2 Implementation and environment have built a prototype system which has been in use on The current prototype Earlybird sensor executes on a the UCSD campus for over eight months. In this sec- 1.6Ghz AMD Opteron 242 1U server configured with tion, we describe our overall system design, the imple- a standard Linux 2.6 kernel. The server is equipped mentation and experimental environment, our initial ex- with two Broadcom Gigabit copper network interfaces periments exploring the parameter space of the content for data capture. The Early Bird sensor itself is a single- sifting algorithm, our evaluation of false positives and threaded application which executes at user-level and false negatives, and our preliminary results in finding live captures packets using the popular libpcap library. The worms at our site ystem is roughly 5000 lines of code(not including ex ternal libraries) with the bulk of the code dedicated to 6.1 System design elf-monitoring for the purpose of this paper. The scal- The Early Bird system consists of two major components: able implementation itself is a much smaller fraction of Sensors and an Aggregator. Each sensor sifts through this code base. In its present untuned form, Early Bird traffic on configurable address space "zones"of responsi- sifts though over ITB of traffic per day and is able to bility and reports anomalous signatures. The aggregator keep up with over 200Mbps of continuous traffic whedress dispersion table – with high probability, the content has appeared frequently enough to be a candidate worm signature. Pseudocode for the main loop of the EarlyBird system is shown in Figure 5. ProcessPacket() 1 InitializeIncrementalHash(payload,payloadLength,dstPort) 2 while (currentHash=GetNextHash()) 3 if (currentADEntry=ADEntryMap.Find(currentHash)) 4 UpdateADEntry(currentADEntry,srcIP,dstIP,packetTime) 5 if ( (currentADEntry.srcCount > SrcDispTh) and (currentADEntry.dstCount > DstDispTh) ) 6 ReportAnomalousADEntry(currentADEntry,packet) 7 endif 8 else 9 if ( MsfIncrement(currentHash) > PravalenceTh) 10 newADEntry=InitializeADEntry(srcIP,dstIP,packetTime) 11 ADEntryMap.Insert(currentHash,newADEntry) 12 endif 13 endif 14 endwhile Figure 6: The EarlyBird loop performed on every packet.When the prevalence threshold is exceeded, dispersion counting is done by creating an ADentry. ADentry contains the source and destination bitmaps and the scale factors required for the scaled bitmap implementation. The content prevalence table sees the most activity in the system and serves as a high-pass filter for fre￾quent content. The multi-stage filter data structure is cleared on a regular interval (60 seconds in our imple￾mentation). By contrast, the address prevalence table has typically fewer values – only those strings exceeding the prevalence threshold – and can be garbage collected over longer time scales (even hours). Each of these mechanisms can be implemented at high speeds in either software or hardware, with relatively modest memory requirements as we quantify in the next section. Moreover, our approach makes no assumptions about the point of deployment, whether at the endpoint, edge, or core. However the optimal parameters settings may depend on the point of deployments. In Section 6 we empirically explore the parameter settings used by our EarlyBird prototype. 6 Experience Based on the content sifting algorithm just described, we have built a prototype system which has been in use on the UCSD campus for over eight months. In this sec￾tion, we describe our overall system design, the imple￾mentation and experimental environment, our initial ex￾periments exploring the parameter space of the content sifting algorithm, our evaluation of false positives and false negatives, and our preliminary results in finding live worms at our site. 6.1 System design The EarlyBird system consists of two major components: Sensors and an Aggregator. Each sensor sifts through traffic on configurable addressspace “zones” of responsi￾bility and reports anomalous signatures. The aggregator Figure 7: A screenshot of the main screen of the EarlyBird user interface. Each zone is labeled by a prefix and shows the cur￾rent anomalies (worms), and prevalence/dispersion parameters which can be changed by the user. More detailed screens show detailed counts for each anomaly, as shown for Sasser in Figure 12. coordinatesreal-time updatesfrom the sensors, coalesces related signatures, activates any network-level or host￾level blocking services and is responsible for administra￾tive reporting and control. Our implementation is written in C and the aggregator also uses the MySql database to log all events, the popular rrd-tools library for graphical reporting, and PHP scripting for administrative control. A screenshot of the main screen of the EarlyBird user interface showing zones and a summary of the current system activity is shown in Figure 7. Finally, in order to automatically block outbreaks, the EarlyBird system automatically generates and deploys precise content-based signatures formatted for the Snort￾inline intrusion prevention system [1]. A sample such signature for Kibvu.B is shown below. drop tcp $HOME_NET any -> $EXTERNAL_NET 5000 (msg:"2712067784 Fri May 14 03:51:00 2004"; rev:1; content:"|90 90 90 90 4d 3f e3 77 90 90 90 90 ff 63 64 90 90 90 90 90|";) 6.2 Implementation and environment The current prototype Earlybird sensor executes on a 1.6Ghz AMD Opteron 242 1U server configured with a standard Linux 2.6 kernel. The server is equipped with two Broadcom Gigabit copper network interfaces for data capture. The EarlyBird sensor itself is a single￾threaded application which executes at user-level and captures packets using the popular libpcap library. The system is roughly 5000 lines of code (not including ex￾ternal libraries) with the bulk of the code dedicated to self-monitoring for the purpose of this paper. The scal￾able implementation itself is a much smaller fraction of this code base. In its present untuned form, EarlyBird sifts though over 1TB of traffic per day and is able to keep up with over 200Mbps of continuous traffic when