Snort 1.7 is now released as stable. After three months of running the beta on my own sensor, I have deployed 1.7 to production. The upgrade is operationally smooth and the new capabilities are worth a short post.
This is mostly a status note. The architectural details I wrote about in June hold; the release adds polish and a few notable refinements.
What is in 1.7 final
The substantive changes from 1.6 to 1.7:
stream4 preprocessor. Replaces the older stream preprocessor. Maintains state for active TCP connections, handles reassembly with sensible behaviour for overlapping segments, and detects evasion techniques that exploit reassembly ambiguity. The most significant single improvement.
Multi-pattern matching. The rules-engine now uses an Aho-Corasick-style search structure for content matches. Performance with large rulesets is dramatically improved.
Improved logging. New output plugins for syslog, database insertion, and a unified binary format. The unified format is particularly useful — it stores all alert metadata in a compact binary form that downstream tools can post-process.
Stricter rule parsing. Rules with subtle errors that earlier versions silently accepted are now flagged at startup. This caught two bad rules in my own ruleset that had been silently producing unhelpful behaviour.
Better preprocessor framework. The interface I built against in July is now formally documented. Custom preprocessors are easier to write and the framework is more stable across versions.
Performance improvements throughout. My sensor's CPU utilisation under typical load has dropped from about 35% to about 20%. The improvement comes mostly from the multi-pattern matching but also from various micro-optimisations.
The upgrade procedure
For my own sensor, the upgrade was a 30-minute exercise:
- Build 1.7 from source on a test machine.
- Copy my custom preprocessor source into the new tree, rebuild, verify it works.
- Test the new binary against my existing ruleset on the test machine.
- Note any rules that produce errors with the stricter parser; fix them.
- Stop the production Snort, replace the binary, restart with the new ruleset.
- Verify operation by reviewing alerts for an hour.
The two rules that needed fixes were both my own — old rules with non-standard syntax that 1.6 had accepted but 1.7 rejects. Fixing them took five minutes; the strict parsing was, on balance, a useful nudge to clean up technical debt in my ruleset.
What is now operationally feasible
Three things that I could not do before but can now:
Rulesets with thousands of rules. The community ruleset that ships with 1.7 has about 1,400 rules. Earlier Snort versions struggled with this on commodity hardware. 1.7 handles it without issue. My own ruleset is now closer to 350 rules; I have room to grow.
Deeper protocol normalisation. The improved preprocessors normalise more thoroughly before matching, which means my rules can be written against the normalised form of the traffic rather than against wire-form variants. Several of my rules have been simplified as a result.
Better integration with downstream tooling. The unified output format is read directly by barnyard, a separate tool that handles alert post-processing — sorting, deduplication, output to multiple destinations. The Snort sensor's job becomes capture and detection; everything after is a separate process that can run with different operational characteristics.
For a small deployment like mine, the immediate benefit is mostly the performance improvement. For a larger deployment, the architectural changes are substantial.
The new community-ruleset relationship
A notable shift with 1.7: the community ruleset is now formally distributed by Sourcefire (the company Marty Roesch founded earlier this year) rather than maintained as a loose collection of community-contributed rules.
The shift produces some discomfort in the community. The ruleset is being maintained more rigorously — fewer obviously-broken rules, better testing — but is also moving toward a commercial model where the most current rules are part of a paid subscription. The free version lags by some interval.
My view is that this is broadly fine. Sourcefire's resources are clearly improving the quality of the rules; the lag for the free version is acceptable for most uses; the commercial subscription is reasonably priced for organisations that need the freshest rules. The model is similar to commercial antivirus: a free baseline and a paid premium tier.
For my own use, I will continue with the free baseline. My honeypot data and personal monitoring are not worth the subscription cost. For the small commercial work I help friends with, the subscription would be worth considering depending on the specific deployment.
What I am doing with the new capabilities
A short list of changes I have made on my sensor in the past fortnight:
Added the unified output plugin alongside my existing logfile output. The unified-format files are processed by a small Perl script I wrote that does the same kind of structured-log analysis as my other monitoring. The aggregate alert metrics are now uniform across all my data sources.
Enabled the stream4 preprocessor with conservative settings. Earlier versions of stream reassembly produced occasional false positives on my traffic; the new version has been clean for three months of beta use.
Upgraded my custom preprocessor to match the formal interface from 1.7. The changes were minor; the result is a preprocessor that should work cleanly with future Snort versions.
Reviewed and pruned my ruleset as part of the strict-parser fixes. About 20 old rules were either redundant or incorrect; removing them reduced the ruleset size and made the remaining rules easier to audit.
A small look forward
The Snort project is still moving fast. The 2.0 development series is starting; the design discussions on the mailing list suggest substantial architectural changes ahead — possibly inline mode (where Snort actively drops malicious traffic rather than just alerting), possibly multi-threading, possibly a more flexible rule language.
For my own use, 1.7 is good enough that I do not need to chase the development tree for at least the next year. I will revisit when 2.0 releases.
The broader open-source security ecosystem continues to mature. Snort, OpenSSH, the Honeynet tools, netfilter — all developing at a sustainable pace, all producing operational tools that compete with or exceed commercial alternatives. The dot-com funding climate has not noticeably slowed the open-source work; if anything, it has accelerated it by funnelling skilled people through projects that need volunteers.
For anyone running Snort: 1.7 is a worthwhile upgrade. Schedule the downtime; do the migration; benefit from the improvements for the next year of operations.
More as the year develops.