Log4Shell

The Log4Shell disclosure (CVE-2021-44228) emerged into public knowledge through Friday afternoon UK time, with the proof-of-concept-driven mass exploitation beginning Friday evening and continuing through Saturday. The vulnerability is in Apache log4j 2 — the most widely-used Java logging library, present in essentially every Java-based application written in the past decade — and triggers from a single specially-crafted log entry containing a JNDI lookup expression. An attacker who can cause the application to log attacker-controlled content gains the ability to execute arbitrary code as the application user. The conditions for exploitation are met by a substantial fraction of internet-facing Java applications because user-controlled content (HTTP headers, form fields, URL parameters) is routinely logged and the JNDI lookup is processed without sanitisation.

The deployment population. log4j 2 is present in the dependency tree of essentially every Java enterprise application of the past decade — applications built directly on log4j, applications built on frameworks that include log4j, applications built on platforms that include those frameworks. The transitive-dependency depth of log4j in the Java ecosystem is substantial; the customer-organisation software-bill-of-materials inventories that we have been pushing through 2018-2021 are about to become operationally critical in a way they have not been before. The customer-organisation populations who have invested in SBOM discipline have a clear path to identifying their exposure; those who have not are working with grep and intuition this weekend, which is operationally inadequate to the scale of the cleanup.

The technical content. The vulnerability is in log4j's JndiLookup class, which is enabled by default in affected versions and which processes ${jndi:ldap://...} expressions encountered in log content. The processing performs a JNDI lookup against the specified URL, which can return a remote-code-loading reference that the JNDI resolver will instantiate. The exploitation chain is therefore: attacker sends content that the application will log, application logs the content, log4j processes the JNDI expression, JNDI fetches and instantiates the attacker-controlled remote code. The mitigation paths are: upgrade to log4j 2.16.0 (which removes JNDI lookup functionality), or upgrade to 2.15.0 (which restricts JNDI to localhost only), or set the system property log4j2.formatMsgNoLookups to true, or remove the JndiLookup class from the deployed JAR. The upgrade path is correct but operationally substantial; the system-property and JAR-modification mitigations are tractable interim measures while the upgrade work proceeds.

The customer-portfolio response is the most operationally demanding work I have ever managed in a single vulnerability disclosure. Every customer-organisation Java application is in scope. The audit cycle is running across the customer estates this weekend with full-team engagement. The pen-testing engagement queue has been temporarily suspended to free analyst capacity for the customer-organisation post-incident hunt and remediation work. The SOC analyst rotation has been accelerated for the weekend to maintain coverage. The team has been working approximately twenty-hour shifts in alternating coverage.

The exploitation-rate question. The mass-exploitation against internet-exposed Java applications began Friday evening and is continuing through this writing on Saturday night. The exploit-payload diversity is substantial — initial-access reconnaissance, cryptocurrency-mining payload deployment, ransomware-targeting reconnaissance, state-actor-attributed espionage activity, and several other payload categories observable across the various honeypot infrastructures the security-research community is running. The operator-side attention to the disclosure is unprecedented in scale; multiple operator clusters have been observed actively scanning and exploiting in parallel.

For the customer-portfolio: the vCISO clients are all engaged on the audit work; the SOC customers are receiving heightened detection content and elevated alerting; the EmilyAI deployments have absorbed the Log4Shell-related detection-content updates and are processing the substantial alert volume that the active exploitation is producing across customer environments. The aggregate operational picture across the portfolio at the time of writing is that customer-organisation patching cadence is fast (most customers will have the immediate-mitigation steps deployed by Sunday afternoon), the customer-organisation hunt activity is in progress (looking for evidence of exploitation that occurred before the mitigations landed), and the customer-organisation post-incident work will continue through next week and beyond.

The wider strategic point is that the Java ecosystem's collective attack surface against this single library has been demonstrated in a way that the security community has, for years, been articulating in the abstract. The structural question — whether the Java ecosystem's dependency-tree management practices are operationally adequate to the security risks that the dependency tree creates — is now, post-Log4Shell, a substantive customer-organisation strategic conversation rather than an academic concern. The supply-chain-and-software-composition-analysis investments that customer organisations make in the next several years will be shaped by Log4Shell as the worked example.

I will write more through next week as the operational picture firms up. The next several weeks are going to be the most demanding period the team has worked through, and the customer-portfolio work will continue beyond that. The Log4Shell cleanup is the operational story of the close of 2021 and will be a significant part of the 2022 conversations as well.


Back to all writing