From conversations with operators over the past two weeks, the cumulative impact of Code Red, Code Red II, and SirCam on internet infrastructure is becoming measurable. The aggregate effects are larger than any single incident's effects.
A short writeup of what the operators are seeing.
Bandwidth
The baseline scan traffic across the internet has roughly doubled since June. The two Code Red variants together produce a sustained scanning load that consumes a noticeable fraction of carrier capacity. Operators with peering relationships are reporting:
- Increased volume on their links, with the increase concentrated in specific TCP/80 patterns matching Code Red.
- Larger router CPU utilisation due to the increased small-packet rate (Code Red's scan probes are short connections).
- Higher costs for operators with usage-based pricing.
The carriers are absorbing the cost; smaller customers are not seeing direct charges yet. Whether this changes is a question for the next quarter.
DNS
DNS infrastructure has been hit harder than expected. Code Red's scanning includes IP-by-IP probing, which means each scan attempt does not produce a DNS lookup. SirCam, by contrast, sends mail to recipients in the user's address book, producing DNS lookups for each recipient's domain. The DNS load from SirCam-mediated mass-mailing has been substantial.
Several major DNS resolvers have reported sustained 50%+ increased query rates over the past month. The query load is largely lookup-of-recipient-domains rather than the typical browsing traffic.
Mail relays
Mail volume globally is up substantially. The combination of SirCam, ongoing other mass-mailing worms, and the surrounding spam volume produces a baseline that is higher than previous norms.
Several relay operators have reported:
- Inbound queue depths consistently higher than 2000 levels.
- Anti-virus signature update cycles compressed to 4-hour intervals.
- Outbound delivery delays on hosts not equipped to handle the volume.
The operators with mature filtering (as I described) are coping. The operators without are being overwhelmed.
Anti-virus and IDS infrastructure
Anti-virus vendor signature delivery has been under stress. Multiple new signatures per day, expedited delivery to customers, increased network and server load on the AV providers. The cycle is sustainable; it is more strained than it was.
IDS vendors are similarly busy. New signatures for Code Red, Code Red II, and the various variants. The community-maintained Snort ruleset has been adding several rules per week.
What the operators are doing
Four patterns I have seen.
Tightening egress filtering. Several operators have moved from "egress filtering for source-address validation" to "egress filtering plus rate limiting plus pattern matching". The cost is modest; the operational benefit is real.
Pre-emptively dropping Code Red traffic. Carriers are filtering the .ida exploit pattern at the network edge, regardless of whether the destination is a vulnerable IIS server. This reduces the propagation rate; some legitimate IIS traffic that happens to match the pattern (rare but possible) is dropped.
Forming inter-operator coordination groups. The SANS Internet Storm Center is becoming the de facto coordination point for this kind of incident. Operators are reporting attack patterns to ISC and reading the consolidated data daily.
Investing in capacity. The carriers are having to add capacity faster than business growth would otherwise require. The cost of worm traffic is producing capacity-purchase decisions.
What this teaches
Three things.
Worms have economic externalities. The cost of Code Red is not just to the compromised IIS operators. It is to every operator on the internet who handles the resulting scan traffic. The cost-allocation does not match the harm-allocation.
The infrastructure is more fragile than it looks. A small set of operators making operationally questionable decisions (running unpatched IIS) produces a sustained burden on the entire internet. The systemic risk from individual choices is real.
The defensive coordination is forming, slowly. ISC, the Honeynet Project, informal carrier coordination — each is incomplete. The cumulative trend is toward better coordination.
A small reflection
For my own modest infrastructure, the past two months have been operationally noisy but not damaging. My honeypot has been busy; my patched servers are unaffected; the increased scan traffic is bounded.
For the internet as a whole, this period has been the most operationally significant since Mafiaboy. The cumulative effects will play out over months.
More as the year develops.