The annual scoring exercise. The 2006 predictions made in January get explicit review; new predictions for 2007 are made.
This post is a longer treatment because the calibration discipline is structurally important and the cumulative archive of predictions across years is now genuinely useful.
2006 predictions, scored
From various posts through 2005 and 2006, the explicit predictions for 2006:
1. Continue weekly cadence. 95%. Resolved AFFIRMATIVE.
2. Settle into new role productively. 80%. Resolved AFFIRMATIVE.
3. At least four conferences. 80%. Resolved AFFIRMATIVE.
4. Speak at at least one conference. 70%. Resolved PARTIAL.
5. Substantial piece on internal segmentation. 55%. Resolved PARTIAL.
6. Major worm event of comparable scale to Sasser or larger. 65%. Resolved PARTIAL — Mocbot was substantial but smaller than Sasser-class.
7. Mass-mailing continues at sustained volume. 95%. Resolved AFFIRMATIVE.
8. Specific UK retail-banking phishing incident. 75%. Resolved AFFIRMATIVE.
9. Continued Microsoft Trustworthy Computing progress. 90%. Resolved AFFIRMATIVE — Vista, IE 7.
10. Two-factor authentication ships at major UK retail bank. 75%. Resolved AFFIRMATIVE.
11. Sony BMG aftermath produces specific regulatory or legal precedent. 70%. Resolved PARTIAL — specific settlements; broader regulatory precedent slower.
The cumulative score: 7 affirmatives, 4 partials, 0 clear misses. Calibration is reasonable.
Calibration assessment
Specific patterns from 2006.
Direction calls were uniformly right. All predicted directions resolved as predicted.
Magnitude calls were mostly right. The Mocbot prediction over-stated the expected scale; otherwise magnitudes were reasonable.
Timing was approximately right. Specific events occurred at roughly the predicted cadence.
The cumulative calibration archive across multiple years suggests I am reasonably calibrated. Specific subsequent predictions are at appropriate confidence levels.
2007 predictions
For the year ahead, with explicit probabilities and deadlines.
Threat-side
1. Continued mass-mailing at sustained volume. 95%, 31 December 2007. No structural change visible.
2. At least one major Vista-targeting malware family. 60%, 31 December 2007. Vista deployment makes the platform attractive; specific targeting will follow.
3. A meaningful mobile-platform malware incident. 55%, 31 December 2007. The category has been forming for years; an operational incident is overdue.
4. Continued growth in DDoS-for-hire commercial activity. 90%, 31 December 2007. The economic infrastructure favours continued growth.
5. A specific high-profile data breach involving substantial UK consumer impact. 80%, 31 December 2007. The cumulative pressure builds; specific incidents will become public.
6. Specific commercial-cybercrime infrastructure attack against a major UK gambling operator. 85%, 31 December 2007. The threat is operationally constant; specific incidents are predictable.
7. Continued web-application worm activity. 80%, 31 December 2007. The category trajectory continues.
Defensive-side
8. Vista SP1 ships. 80%, 31 December 2007. The Microsoft cadence is consistent.
9. Continued Microsoft Trustworthy Computing progress. 90%, 31 December 2007. The trajectory is established.
10. Specific PCI compliance enforcement actions. 80%, 31 December 2007. The trajectory continues; specific operators will face enforcement.
11. Cumulative two-factor authentication deployment at major UK retail banks. 85%, 31 December 2007. The trajectory points toward this becoming standard.
12. Specific industry-coordination structures around DDoS-mitigation. 60%, 31 December 2007. Specific UK operators are discussing structural coordination; specific outcomes are uncertain.
Structural
13. Continued data-breach disclosure expansion. 85%, 31 December 2007. The regulatory trajectory continues.
14. Specific UK regulatory tightening on data protection. 60%, 31 December 2007. The political trajectory points toward this; specific timing is uncertain.
15. Specific industry-level conversations about software-vendor liability. 50%, 31 December 2007. The Sony BMG aftermath continues; specific conversations continue.
Personal
16. Continue the CISO role at Gala Coral productively. 90%, 31 December 2007. The role-fit is good; the cumulative engagement is productive.
17. Continue weekly cadence on the notebook. 95%, 31 December 2007.
18. Attend at least four conferences. 80%, 31 December 2007.
19. Speak at at least one conference. 70%, 31 December 2007.
20. Write a book. 60%, 31 December 2007. I have been thinking about a book on DDoS for some time; the specific writing is bounded; the publication timing is the open question.
A meta-prediction
21. By end of 2007 I will have nine full years of prediction-scoring data. 95% (resolves trivially if I keep the discipline).
The cumulative meta-analysis from nine years will be substantively interesting. Specific systematic biases visible across the cumulative archive will inform subsequent prediction discipline.
A specific note on the book prediction
For some time I have been thinking about writing a substantive book on DDoS — the DDoS-for-hire trajectory, the DDoS extortion against gambling operators, the structural patterns of attack and defence, the cumulative operational discipline.
The specific intent: a practitioner-targeted book that addresses the operational realities of DDoS in 2007, drawing on the cumulative observation across multiple years and multiple operational contexts.
The 60% probability reflects realistic uncertainty about timing. Writing a book takes substantial sustained effort; specific role demands at Gala Coral compete for time; the publication timeline is bounded but not predictable.
For practitioners considering whether to write similar long-form material: the cumulative archive of weekly notebook writing is substantial substrate. The specific work of organising that substrate into a book-length treatment is bounded; the cumulative impact of book-length work is structurally larger than weekly posts.
More on the book prediction across 2007.
A closing reflection
The calibration discipline produces cumulative learning across years. Specific predictions inform; specific reviews inform; specific cumulative patterns inform. The cumulative archive supports both individual practice and broader community contribution.
For my continued practice: the discipline continues. Specific 2007 predictions will be reviewed at year-end 2007. The cumulative archive continues growing.
More in 2007. The cumulative discipline continues.