December is the right month for taking stock. I have spent the last week working through the CERT advisory archive and my own notes from a year of Bugtraq reading. The aggregate is more revealing than any individual advisory.
This post is the patterns I see, with examples and my best guesses about where each is going.
The numbers
A rough count of advisories published in the public space this year, by category:
- Memory corruption (buffer overflows, format strings, integer overflows): about 60 distinct advisories I have logged. This is the largest category by a substantial margin.
- Input validation failures (SQL injection avant la lettre, command injection, path traversal): about 25.
- Cryptographic failures (weak random number generation, predictable session IDs, broken protocol implementations): about 15.
- Race conditions (TOCTOU, unsafe signal handling, concurrency bugs): about 10.
- Default-credential and configuration failures: about 10.
- Denial-of-service-only issues (no privilege escalation, just service disruption): about 20.
- Worms and worm-like malware (Happy99, Melissa, ExploreZip, several smaller incidents): three major plus a long tail.
The dominant category, by quite a margin, is memory corruption. The dominant single product is — perhaps unsurprisingly — the C language ecosystem in general: BIND, sendmail, wu-ftpd, various SUID binaries, and the family of small utilities that have been around since BSD 4.3. The bugs are old in style, fixed in pattern, and depressingly recurrent.
What changed compared to 1998
A few visible shifts.
The mass-mailing worm became routine. Happy99 in January, Melissa in March, ExploreZip in June, several smaller variants throughout the year. The category did not exist as a regular concern in 1998. It is a permanent feature of the threat landscape now.
Distributed denial-of-service emerged. Trinoo in summer, TFN evolution through autumn, the Minnesota incident as the public demonstration that this was real. Twelve months ago this was a theoretical concern in academic papers. It is a deployed reality now.
The disclosure conversation matured. Some of the harder voices on full disclosure have softened slightly. Vendors are responding faster. The community of paid security researchers has grown enough that there is a meaningful middle ground between "publish immediately" and "never publish".
Open-source security tools have visibly matured. Snort is now a serious project. OpenSSH is the dominant SSH implementation. Nessus is genuinely competitive with commercial scanners. The open-source security-tool ecosystem has gone from "interesting" to "core" in a single year.
What did not change
For balance, the things I expected to change and which essentially have not.
The memory-safety problem. The bug class is the same as it was last year. The same products keep producing the same bugs. The auditing efforts at OpenBSD and elsewhere are useful but slow; they are not changing the trend line. As long as we write internet-facing daemons in C, this category will dominate the advisory list.
Default configurations. Most products still ship with too much enabled. The IIS lockdown wizard is a workaround for an install that is, by default, unwise. Apache's default module set is more than most servers need. Linux distributions still install services that should not be on by default. The structural problem of generous defaults has not been solved.
Patching speed. The aggregate population of internet-facing hosts is patched, on the available evidence, about as slowly as it was last year. The diligent operators are diligent; the long tail is long; the average is somewhere in the middle. Nothing has structurally changed how quickly patches reach hosts.
Encryption deployment. Most internet traffic is still in cleartext. SSH has displaced Telnet for shell access. SSL is on most major commercial sites. Mail is still cleartext for most users. PGP usage is rounding error. The deployment of useful crypto has progressed slowly, and 1999 did not accelerate it.
A taxonomy of 1999's most important bugs
Four particular advisories that I think are structurally significant.
The BIND NXT vulnerability (written up here). Not because the bug itself is unusual — it is a textbook buffer overflow — but because it was in BIND, which is on essentially every DNS server in the world, and because the patching curve was clearly visible. This was the year's reminder that critical infrastructure is held together by a small number of critical software projects with all the bug shapes their codebases imply.
The IIS rolling advisories (covered here). The structural pattern of one product producing a steady drumbeat of bugs throughout the year. Each individual bug is patched. The product's overall security posture remains poor because of the structural issues — too many default extensions, large attack surface, opaque internals.
The Trinoo and TFN DDoS tools. A category change. Distributed attack tools went from a research idea to operational reality. The defensive response is not yet adequate at any scale.
The Melissa virus. The proof that mass-mailing macro malware is operationally viable. The follow-up by ExploreZip established that the category is permanent.
Where I think 2000 goes
A few predictions, written down so I can score them at year-end as I resolved to do:
Y2K will be uneventful in the moment and consequential afterwards. The midnight transition will pass with few visible failures. The first quarter of 2000 will see a wave of security advisories from rushed remediation. Some of those will be substantial.
A major commercial site will be DDoSed publicly. The trajectory is clear. By midyear I expect a household name to be offline for hours, with consequent press coverage and probably a spike in regulatory attention.
The first widely-deployed AI-style intrusion detection will appear. Probably not in 2000, but the prior work — pattern matching, statistical anomaly detection, the Snort family — is converging on something more sophisticated. By 2001 I expect the first serious production tool in this space.
Linux will continue its slow march toward operational maturity. POSIX capabilities becoming usable, kernel hardening features becoming standard, distributions tightening their defaults. The transition from "interesting hobby OS" to "production-ready alternative to commercial Unix" will be visibly complete by year-end.
The disclosure conversation will continue maturing. Some kind of widely-accepted norm — possibly an RFC — for responsible disclosure timing will emerge. It will not satisfy everyone but it will be enough to stabilise the community's behaviour.
At least one widely-deployed product will be retired publicly because of cumulative vulnerabilities. Not from neglect, but because the operator decided the cost of running it was not justified by the benefit. This will set a precedent that other operators will reference.
A small reflection
The single biggest thing 1999 has taught me is that the structure of the threat landscape changes faster than I had expected. Twelve months ago, distributed DoS was theoretical and email worms were a concern but not a regular phenomenon. Now they are both routine. The pace of evolution on the offensive side is high.
The defensive side, by contrast, is structurally slow. The tools improve; the practices improve; the pace of improvement is at the timescale of years, not months. The asymmetry is uncomfortable.
What I think this means in practice: defenders cannot win on technique alone. The structural improvements — encryption everywhere, capability-based security, platform diversity, BCP 38 deployment — are the things that move the line. Individual operators contribute to those structural improvements, but they do not happen at any individual operator's pace.
The operational discipline of 1999, then, is not to defend perfectly. It is to contribute to the slow structural improvement while doing the unglamorous patching, monitoring, and segmentation that holds the line in the meantime.
More in the new year. The new year is, I think, going to be interesting.