The annual scoring exercise. The 2008 predictions get explicit review; new predictions for 2009 are made.
This is a longer post because the calibration discipline matters and 2009 has substantial uncertainty embedded in it.
2008 predictions, scored
The explicit predictions made for 2008:
1. Continue weekly cadence. Resolved AFFIRMATIVE.
2. Continue the CISO role at Gala Coral productively. Resolved AFFIRMATIVE.
3. Speak at Infosec Europe. Resolved AFFIRMATIVE.
4. Attend at least four conferences. Resolved AFFIRMATIVE.
5. Specific follow-up book or substantial extended writing. Resolved PARTIAL.
6. Continued mass-mailing at sustained volume. Resolved AFFIRMATIVE.
7. Major Vista-targeting malware. Resolved PARTIAL.
8. Mobile-platform malware incident. Resolved PARTIAL.
9. Continued DDoS-for-hire growth. Resolved AFFIRMATIVE.
10. UK consumer-impact data breach. Resolved AFFIRMATIVE.
11. Major DNS-protocol vulnerability disclosed. Resolved AFFIRMATIVE — Kaminsky.
12. Politically-motivated DDoS following Estonia pattern. Resolved AFFIRMATIVE — Russia-Georgia.
13. Vista SP1 ships. Resolved AFFIRMATIVE.
The cumulative score: 10 affirmatives, 3 partials, 0 misses.
2009 predictions
For the year ahead, with explicit probabilities and deadlines.
Threat-side
1. Conficker variants continue at sustained operational impact. 95%, 31 December 2009. The architecture is established; specific operators will iterate.
2. At least one major data-breach disclosure with substantial UK consumer impact. 85%, 31 December 2009. The trajectory continues.
3. Continued mass-mailing at sustained volume. 95%, 31 December 2009.
4. Continued politically-motivated DDoS following Estonia / Russia-Georgia pattern. 80%, 31 December 2009.
5. A meaningful mobile-platform malware incident. 70%, 31 December 2009. iPhone exposure grows; specific research will operationalise.
6. Continued web-application worm activity. 80%, 31 December 2009.
7. Specific cumulative subsequent internet-infrastructure-protocol vulnerability disclosure. 60%, 31 December 2009. DNS, BGP, others have demonstrated fragility; specific subsequent work likely.
8. A specific incident involving major UK financial institution security failure. 75%, 31 December 2009. The crisis-period stress combined with specific cumulative threat-actor activity.
Defensive-side
9. Continued Microsoft Trustworthy Computing progress. 90%, 31 December 2009.
10. Specific cumulative cumulative DNSSEC deployment progress. 55%, 31 December 2009. The Kaminsky disclosure may motivate cumulative action; specific deployment is bounded.
11. Continued cumulative two-factor authentication deployment. 85%, 31 December 2009.
12. Specific cumulative industry-coordination response to Conficker. 80%, 31 December 2009.
Structural
13. Continued data-breach disclosure expansion. 85%, 31 December 2009.
14. Specific cumulative recession-driven operational impact across security operations. 85%, 31 December 2009. Cost pressures will affect the field.
15. Specific UK regulatory tightening on data protection. 60%, 31 December 2009.
Personal
16. Continue the CISO role at Gala Coral productively. 80%, 31 December 2009. Bounded uncertainty about specific subsequent role developments.
17. Continue weekly cadence on the notebook. 95%, 31 December 2009.
18. Attend at least four conferences. 80%, 31 December 2009.
19. Speak at at least one conference. 70%, 31 December 2009.
20. Specific subsequent substantial writing project. 55%, 31 December 2009.
A meta-prediction
21. By end of 2009 I will have eleven full years of prediction-scoring data. 95%.
The cumulative meta-analysis from eleven years will be substantively interesting.
A specific note on the Gala Coral prediction
For 2009, the prediction about continuing in the Gala Coral role is at 80% rather than higher. Several reasons.
The cumulative role has been operationally productive across nearly three years. Specific structural projects are completing; specific cumulative organisational maturity has developed.
Specific subsequent professional opportunities are emerging. The cumulative practitioner network from the DDoS book, the conference engagement, and the broader cumulative trajectory have produced specific opportunities for what comes next.
Specific cumulative thinking about an independent practice. I have been thinking about whether to start something independent for some time. The cumulative trajectory may make 2009 the year that decision becomes operational.
The 80% probability reflects realistic uncertainty. Specific subsequent role decisions will depend on specific cumulative cumulative considerations through Q1 and Q2 2009.
For my own continued discipline: the notebook will document the trajectory as decisions are made.
A closing reflection
The calibration discipline produces cumulative learning across years. Specific predictions inform; specific reviews inform; specific cumulative patterns inform.
For my continued practice: the discipline continues. The cumulative archive continues growing.
More in 2009.