Earlier this year I committed to reading OpenSSH source carefully. A week of evenings has produced enough notes to write a useful post. The implementation is more disciplined than I had assumed; the architectural choices are educational.
This is a writeup of what stood out, with specific references where relevant. The version I have been reading is OpenSSH 2.1.0.
The high-level structure
OpenSSH ships several distinct programs that share a substantial code base:
sshd— the server daemon.ssh— the client.scp— file copy, built onssh.ssh-keygen— key generation and management.ssh-agent— the credential-caching agent.ssh-add— adds keys to the agent.
The shared code is, by my rough count, about 60% of the total source. The protocol handling, the cryptographic operations, the channel management — all are in the shared layer. Each program is a thin layer of policy on top of the shared infrastructure.
This is the right architecture. The cryptographic and protocol code is exercised by every binary, which means bugs are discovered against breadth of usage. The policy layer for each binary is small and reviewable.
The protocol layering
SSH-2 is, in OpenSSH's implementation, three distinct layers:
The transport layer handles the underlying connection — packet framing, encryption, integrity, key exchange, server authentication. The code lives in kex.c, packet.c, and a handful of cipher-specific files. This is the layer that sets up the encrypted channel between client and server.
The authentication layer handles user authentication once the transport is established. The code in auth.c, auth-rsa.c, auth-pam.c, and several others. This is where the server decides whether to accept the user.
The connection layer handles multiplexed channels over the authenticated connection — the actual shell session, port forwards, X11 forwards, agent forwards. The code in channels.c. This is where the user-visible features live.
Reading the source layer-by-layer is the right approach. Each layer is largely self-contained, with well-defined interfaces between them. A reader who tries to read the whole codebase in source-file order will be confused; a reader who reads one layer at a time gets a coherent picture.
The cryptographic discipline
This is where I was most impressed. The OpenSSH project has clearly thought hard about how to make cryptographic code reviewable.
Algorithm negotiation is structured. The set of supported ciphers, MACs, and key-exchange methods is a list, traversed in a defined order. Each algorithm has an entry with its name, key length, IV length, and implementation pointers. Adding a new algorithm is a clean addition; removing a vulnerable one is a single-line change.
Key material is handled carefully. Keys are wiped from memory with explicit memset calls when no longer needed, in the function that allocated them. Reading the source, I can trace every key from creation to disposal. There are no places where a key is created and the disposal is implicit.
Constant-time operations where they matter. The HMAC verification uses a constant-time comparison so timing attacks cannot extract information from differential response times. The same care is visible in the password-hash comparison.
Random number generation is centralised. The arc4random function (or, where the OS provides it natively, the OS's RNG) is the single source of cryptographic randomness. Every other piece of code that needs random data goes through this central function. This means the RNG quality is a single audit point rather than scattered across the code.
Cryptographic dependencies are minimal. OpenSSH depends on OpenSSL for the underlying primitives. The interface is narrow: specific functions for AES, HMAC, Diffie-Hellman, RSA. The wrapper layer is small. This means an OpenSSL bug affects OpenSSH only through a small surface.
The overall impression is of code written by people who took cryptographic engineering seriously and treated each piece of cryptographic code as if it would be reviewed. This is exactly the discipline I would want from any security-critical project.
What I found surprising
The privilege separation is recent and incomplete. OpenSSH 2.1.0, which I am reading, runs the connection-handling code as root. Compromise of the connection-handling produces immediate root. The OpenSSH team has been clear that this is wrong and is working on a privilege-separation architecture (sshd would fork into a privileged parent and an unprivileged child, with the child handling the network protocol and only the parent doing the actually-privileged operations like opening pty devices).
The privilege separation work is in development; the version I am reading does not have it. By the next stable release I expect it to be in.
The compatibility shims are extensive. OpenSSH supports both SSH-1 and SSH-2 — necessary for transition. The dual-protocol support produces noticeable code complexity. There are several places where the same logical operation is implemented twice, once for each protocol version. The eventual deprecation of SSH-1 will simplify the code substantially.
The portability layer is significant. OpenSSH runs on many Unix variants. The portability layer, which abstracts over OS differences, is several thousand lines of code. Reading it gave me appreciation for how many subtle differences exist between supposedly-similar Unix implementations.
Some choices are pragmatic rather than ideal. The configuration-parsing code, for instance, is straightforward but limited — it handles the documented keywords correctly but is not particularly elegant about syntax errors or unknown options. Several of the comments in the parser amount to "this is good enough". Reading them is a useful reminder that perfect is the enemy of shipping.
What this changes about my deployment
A few specific things.
I am more confident in OpenSSH as critical infrastructure. Reading the source has confirmed what I had hoped — the code is written by people who care, with the disciplines of careful cryptographic engineering. I am happy to continue depending on OpenSSH for shell access to my hosts.
I am going to upgrade promptly when the privilege-separation version ships. Running the connection-handling as root is the largest single weakness I could identify in the current version. The fix is well-understood and is coming. Until it ships, I am paying more attention to the structural defences — chroot, dedicated user, network-level filtering — that limit the consequences of any compromise of sshd itself.
I am going to read more security-critical source. The OpenSSH exercise has been valuable enough that I am going to make this a regular discipline. Next on the list: OpenSSL (much harder; the codebase is large and the cryptography is more abstract), and the TCP/IP implementation in some BSD variant for comparison with the Linux stack I read in 1999.
A small reflection on cryptographic engineering
Reading OpenSSH source has reinforced something I have been slowly internalising over the past two years. Cryptographic engineering is a different discipline from cryptographic theory. The theory is about whether the algorithm is sound. The engineering is about whether the implementation is correctly using the algorithm, in all the edge cases, with appropriate handling of every input.
The number of ways an implementation can go wrong despite using a correct algorithm is large. Timing attacks, side-channel leaks, subtle padding errors, key-management mistakes, RNG failures. Each of these has produced real-world vulnerabilities in real-world systems despite the underlying cryptography being sound.
The OpenSSH source shows a project that has taken this seriously. Other implementations of the same protocols have been less careful and have produced advisories. The variance between implementations is large enough to matter for what you choose to deploy.
For my own infrastructure, the lesson is to be more careful about which implementation of any cryptographic protocol I use. The protocol being secure is necessary but not sufficient. The implementation being engineered with discipline is also necessary. I am going to apply this filter more deliberately for the next few years.
More as the year develops.