Tech Solvency - The Story So Far

Contents

• Key updates
• Context - who (and what) is affected
• Summaries
• Technical analysis
• Remediation
• Affected (and unaffected) products
• Other product and tool lists - see especially new CISA list on GitHub (but only has public info - see these lists if your product is not listed here)
• Detection and testers
• Exploitation
• News and posts

Key updates

• Blackberry researchers discover log4j use by Initial Access Brokers (IABs) against VMware Horizon (2022-01-26)
• CVE-2021-44832 (CVSS 6.6) - do not be alarmed (yet) - it appears to require ability to write a local config file to be exploited ("where an attacker with permission to modify the logging configuration file can construct a malicious configuration")
• Big new joint CISA / Five Eyes mitigation advisory (2021-12-22)
• Worm? - Kevin Beaumont and Marcus Hutchins say not really, because it has a hard-coded LDAP server - but better versions may be feasible soon
• NOTE: All previous mitigations - based on anything other than upgrading to log4j 2.16 (or higher) or entirely removing JndiLookup classes - are no longer effective mitigation
• Newer NIST CVE 2021-45046 - changed to RCE 9.0 (but requires non-default config)
• Version 2.17 is out - fixes the DoS, but IMO if your vendor only has a 2.16-based fix, apply that now instead of waiting (CVSS 10 is more urgent)
• Apache security summary - regularly updated - summary of valid workarounds below
• CISA has issued Emergency Directive 22-02 - required patching timeline changed from Dec 24 to immediately
• VMware latest workarounds (script to remove class) urgent - Conti ransomware seen leveraging log4shell against VMWare (Cimpanu)
• cisagov/log4j-scanner - CISA has a scanner!
• Apache is now publishing known post-EOL log4j 1.2 vulnerabilities (even though they will not be fixed) (2022-01-19

Context - who (and what) is affected

• Impact: arbitrary code execution as the user the parent process is running as (code fetched from the public Internet, or lolbins already present on system, or just fetching shared secrets or environment variables and returning them to the attacker)
• Targets: Servers and clients that run Java and also log anything using the log4j framework - primarily a server-side concern, but any vulnerable endpoint could be a target or a pivot point
• Downstream projects: until proven otherwise, assume anything that includes log4j, or depends on something that does, is affected in a way that requires mitigation; see below
• Affected versions: log4j 2.x confirmed - log4j 1.x only indirectly (previous information disclosure vulns, harder to exploit) (in some configurations). Also, presence of 1.x is not good - 1.x went EOL in August 2015!
• Appliances: Don't forget appliances and other opaque or third-party systems that may be using Java server components, but won't be detected by un-credentialed vulnerability scanning or simple exploitation tests
• Log forwarding: logging infrastructure often has many "northbound" (send my logs to someone) and "southbound" (receiving logs from someone) forwarding/relaying topologies. Chaining them together for exploitation must also be considered. (For those not familiar, these are terms of art in the NMS (Network Monitoring/Management Systems)/logging space - ref, ref, ref)
• Cloud: Multiple large providers also affected (but this guide focuses mostly on customer-managed side)
• Deadlines: CISA orders federal agencies to patch Log4Shell by December 24th

Scope / seriousness

• "hearing folks compare #log4shell is "as bad as heartbleed" - imo it's much, much worse. aside from having RCE as the impact, the number of interdependencies around log4j (and particularly the age of them) is orders of magnitude higher" -@caseyjohnellis
• "What people seem to miss: The #Log4Shell vulnerability isn't just a RCE 0day. It's a vulnerability that causes hundreds and thousands of 0days in all kinds of software products. It's a 0day cluster bomb." -@cyb3rops (Florian Roth)
• "A project with a footprint like Log4j is not possible to avoid as a transient dependency even if you don’t directly import it. Log4j is a canonical logging utility for a huge ecosystem. Its current radius is beyond doing due diligence." - @rakyll (AWS)
• "The Log4j vulnerability is the most serious vulnerability that I've seen in my decades-long career." - CIA Director Jen Easterly, in interview
• The Wikipedia article on log4j is informative to understand usage and scope
• Log4j vulnerability: what should boards be asking? (NCSC)
• Earliest detection known: 2021-12-01 04:36:50 UTC
• Misnomers: No, it is not also called LogJam. That name is already taken. (Initial LunaSec post used that name, then picked a new one once they found out.)
• Pronunciation: its main author pronounces it "log 4 jay", not "logforge"

back to top

Summaries

• CVEs: CVE-2021-44228, CVE-2021-45046 (not quite as bad). Note also unrelated (but also bad) CVE-2021-4104, announced 2021-12-13 and affecting 1.2 JMSAppender behavior (not the default)
  "Apache Log4j2 2.0-beta9 through 2.12.1 and 2.13.0 through 2.15.0 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. From version 2.16.0, this functionality has been completely removed. Note that this vulnerability is specific to log4j-core and does not affect log4net, log4cxx, or other Apache Logging Services projects."
• NIST CVE-2021-44228
• NIST CVE 2021-45046 - changed to RCE 9.0
• Apache security summary - regularly updated
• Original Vendor vuln announcement post (Apache)
• YfryTchsGD/Log4jAttackSurface - screenshots of exploit evidence, including major cloud providers
• Microsoft guidance - excellent
• Reddit /r/netsec rollup thread
• magoo.github.io - good short links overview
• noperator gist with software list (BishopFox Cosmos team)
• NCSC-NL repo - operational information, including big software list
• slf4j commentary
• Microsoft's response doc
• Sophos overview
• Sophos explainer
• Trend Micro overview
• Useful explainer diagrams (mubix)
• CERT-CC - polished, concise coverage - includes filterable list of vendor statements
• Brief SE post on differences between log4j (1.x and 2.x), slf24j, and logback
• Wikipedia article
• CrowdStrike Quick Reference Guide

back to top

Technical analysis

• CISA guidance - likely to become a primary resource and coordination point
• Cloudflare - good tech analysis, and response breakdown. See also general blog post, including note that sanitizing of Cloudflare Logs is now opt-in
• Cisco Talos
• Apache project vulnerability announcement
• LunaSec has a very good follow-up post on mitigation, second vuln
• Fastly
• Kevin Beaumont / GossiTheDog Twitter thread - good rolling thread
• Kudelski Security - good analysis
• GitHub advisory
• LunaSec
• mbechler blog post
• Google overview and guidance (Google Cloud, but generally useful)
• Randori
• Rapid7
• SANS diary initial, and follow-up, long haul
• Veracode
• Cado Security has good attack-surface and attack analysis
• Commit that disabled JNDI by default in log4j2
• Docker - context and updated software
• Musana quick guide
• Blumira post
• Fastly analysis of observed attacks
• Reddit /r/blueteamsec post, high quality
• Mandiant
• Why recursive substitution in log4j2 is dangerous 2019 - (lots of ways to evade detection/WAF) -
• Understanding the Impact of Apache Log4j Vulnerability(Google Security Blog)
• Blumira analysis of local trigger - drive-by alternate attack method - (Javascript Websocket, just by surfing)
• Center for Internet Security (CIS)

back to top

Remediation

Big new joint CISA / Five Eyes mitigation advisory (2021-12-22)

Direct remediation:

• Upgrade any older Log4j 2.x 2.15.x insufficient! 2.16.x (Note: requires Java 8!). 2.15.0-rc1 also insufficient.
• Originally patched in this PR - lots of good comments
• For older log4j, you can rename/remove the JndiLookup.class
• Note that any project still using log4x 1.x is running a deprecated and unsupported version with other known vulnerabilities - CVE-2019-17571 is CVSS 9.8(!)
• Latest Java by itself, even with trustURLCodebase = false, there are bypasses
• It's been stated that Your application is only affected if using logj4-core (Reddit /r/programming thread). I'm not a programmer, so validate for your situation. confirmed by Apache security summary

Mitigations - easy but partial

• Putting Cloudflare in front of your site (and terminating your SSL there) could be an easy but only partial solution
• (@MalwareTechBlog): If you can't upgrade log4j, you can mitigate the RCE vulnerability by setting log4j2.formatMsgNoLookups to True (-Dlog4j2.formatMsgNoLookups=true in JVM command line) (but only for >= 2.10.0).
  no longer sufficient
• reload4j A drop-in replacement for log4j 1.x

Mitigations - official project itself (but always check latest Apache guidance)

• Still valid:
• Java 8+: upgrade to 2.16.0
• Java 7: upgrade to 2.12.2
• Otherwise, in any release not 2.16.0, may remove JndiLookup from classpath
• Users advised not to enable JNDI in Log4j 2.16.0. If JMS Appender required, use Log4j 2.12.2.
• Note: Remove the JndiLookup and JndiManager classes from the log4j-core jar. Removal of the JndiManager will cause the JndiContextSelector and JMSAppender to no longer function
• Users of Log4j 2.10 or greater may add -Dlog4j2.formatMsgNoLookups=true as a command line option or add log4j2.formatMsgNoLookups=true to a log4j2.component.properties file on the classpath to prevent lookups in log event message.
  no longer sufficient
• Users since Log4j 2.7 may specify %m{nolookups} in the PatternLayout configuration to prevent lookups in log event message
  no longer sufficient
• sargent/nojndi - PoC removal of JNDI lookups across the JVM

Mitigations - harder

• WAF to limit exploit queries - but this only eliminates part of the attack surface (and also bypasses are actively advancing)
• egress filtering to block unexpected outbound traffic and to monitor for denied outbound connections
• Cybereason "vaccine" "This project is called 'Logout4Shell' and walks you through setting up a Java-based LDAP server and includes a Java payload that will disable the 'trustURLCodebase' setting in a remote Log4j server to mitigate the vulnerability."
• Hotpatch from AWS - "injects a Java agent into a running JVM process" - GitHub repo
• nccgroup/log4j-jndi-be-gone - Byte Buddy Java agent-based fix
• Infiniroot post using nginx and Lua to block
• list of domains (and all subdomains) to block - not perfect, but could help
• Even TCP egress control isn't enough - if you are not controlling outbound DNS (UDP), you should be
• Barracuda has WAF coverage
• log4jail - "A fast firewall reverse proxy with TLS (HTTPS) and swarm support for preventing Log4J attacks"
• Edgecast has WAF coverage

Mitigations - ecosystem

• Cloudflare rolling out mitigations for all customers, even free customers w/o WAF
• Google Cloud Armor WAF also now protecting
• Imperva blocking many
• David Crawshaw on broader ecosystem challenges
• Filippo Valsorda on professionalizing maintainership
• Patrick Howell O'Neill in MIT Technology Review on criticality and economics of linchpin FOSS
• The Long-Term Impact of Log4j (Coalfire)

back to top

Affected (and unaffected) products

See other product and tool lists if your product is not listed here)

Note: this list focuses primarily on customer-controlled components. For fully cloud-based components, top section of the YfryTchsGD repo is pretty good as a starting point.

Disclaimer: caching/summaries is best effort and may be out of date or incorrect - always validate for yourself

Claimed patched (previously vulnerable, now remediated/mitigated or updates available)

• Apache Druid - advisory, 0.22.1 released 12/10, see this thread
• Apache Flink
• Apache Iceberg (minor patch 1.x)
• Apache Solr
• Apple iCloud circumstantially patched, PoC no longer works
• AppNeta (official)
• AppSheet - mostly unaffected, minor patching completed
• Aptible - uses Elasticsearch, mitigated or class removed
• Backblaze
• Badlion (Minecraft client)
• Barracuda (official tweet)
• Blancco (management console patch)
• Chef (Infra Server - includes Elasticsearch, was patched) - blog post
• Cloudian (support wall)
• Connect2id (patched - upgrade, other details)
• Code42 / Crashplan (mostly unaffected, with notes)
• Constrast Security (SaaS, mitigated)
• ControlUp (multi, unaffected or mitigated)
• Coralogix (multi, unaffected or mitigated, note potential northbound integrations)
• Couchbase
• cPanel (its Solr component - mitigations) - blog post
• Darktrace (EIS reported as patched, behind support wall)
• DeepInstinct
• Dropwizard patched to 2.16 - previously stated as unaffected
• FileCap - 5.1.1 released, patches log4j (later patches reported as unrelated)
• Genesys - all products ("already implemented remediations where possible and are working to communicate any potential exploit paths with specific customers that may be impacted."
• Ghidra (PoC)
• Graylog (tweet)
• GuardedBox (patched - official Tweet)
• HackerOne (patched)
• HostiFi (patched)
• Hubitat (patched, but impact was limited)
• IBM Cúram aka Curam
• IBM WebSphere " affects the WebSphere Application Server Admin Console and the UDDI Registry Application" - patched
• IronNet (patched)
• JGAAP (evllabs) (patched)
• Jitsi (patched)
• Kafka Connect for Azure CosmosDB (patched)
• LastPass (LogMeIn) - Universal Proxy affected - patch
• LeanIX - fixed
• LogicMonitor - patches applied
• LogRhythm (reports of patching, but support wall)
• logstash - upgrade to fix
• Lucent Sky - AVM On-Demand or managed instances
• Mailcow (use of Solr, Github thread)
• Metabase (patched to user newer log4j)
• Mimecast (patched)
• Minecraft (server and client) - update to 1.18.1, guidance
• Minecraft - many server variants documented by a third party (Bisect Hosting)
• Nulab (patched)
• New Relic (JavaAgent patched)
• Okta - some products affected, patching needed urgently
• OneLogin reported as not affected, but no official statement yet (contact support)
• OpenNMS (various components)
• OpenSearch
• Oracle (patched - support wall)
• OWASP ZAP (zaproxy)
• Oxygen XML Editor
• PhenixIDr - vulnerable, apply mitigations, next releases will be patched
• Ping Identity (support wall) and PingFederate ref - PingAccess? - private support comms say flag mitigations insufficient, jar modifications strongly encouraged
• Plesk
• PortEx - patched in release 3.0.2
• Puppet (most not affected except Continuous Delivery for Puppet Enterprise, claimed patched)
• Qmatic (products unaffected, infra patched)
• RedHat (OpenShift, OpenStack - mitigations required)
• RedHat (larger rollup - pending)
• Respondus (not impacted, but patched proactively)
• SAS (JMP not impacted; SAS Profile remediated 12/10)
• Security Onion - Elasticsearch, Logstash, TheHive/Cortex all updated
• SiteGround - official tweet
• Sophos (most components except Cloud Optix SaaS, which was patched)
• SpigotMC (Minecraft server)
• SumoLogic Collector (patched)
• SysAid (patched versions available, on-prem and cloud)
• Threemax (stated as unaffected per support reply - link needed)
• TeamViewer (only server side needed patching)
• Ubiquiti UniFi (patched) (newer patch released 12/17
• Ubuntu - upstream patched, pkgs available
• VoiceThread (official tweet)
• Wallarm
• WitFoo
• Wowza - Streaming Engine, affected versions 4.8.8.01 and higher
• Xilinx - patch availabler
• ZoomInfo
• Zscaler

Confirmed affected - version differences, workarounds suggested, status pending, or not yet analyzed

• Apache Struts (LunaSec ref only, need better link)
• Apache James SMTP Server - Twitter PoC (@dlitchfield)
• Apereo CAS and community workarounds
• Atlassian products (forum thread), also FAQ (Bitbucket, Confluence, Jira, etc.) and community thread with official reply - Bitbucket Server uses affected Elasticsearch
• Forescout - unconfirmed - KB 12049 (support wall); all versions now have patches available - minimum base version patches may be required first; see also blog post
• IGEL - only UMS affected - patch
• LogiAnalytics - mixed, work in progress - good transparency/li>
• ManageEngine (ADManager Plus, ADAudit Plus, etc) - commmunity post, official reply, patch as caution
• MediaWiki - has Elasticsearch; workarounds
• neo4j (community official post, 4.2+ affected, workaround, patched release pending)
• OpenMRS (suggests -D workaround, patched release pending)
• PEGA (remediation via SQL cmd for on prem and self-managed)
• Rosette - only RNI WebServices (deprecated product) - "vulnerability will be removed"
• Powerschool (Reddit only, appears exploitable, official pending)
• SAFE FME Server - not vulnerable, updating 2.x to 2.15 anyway, but also not providing patches for unsupported versions
• SDL WorldServer affected, workarounds listed
• Seafile - Pro only, Elastic search dependency, workarounds listed, risk "low"
• Silver Peak GMS Orchestrator (Aruba/HPE) - no patch yet, workarounds listed
• Tosca - licensing server uses Elasticsearch, claimed OK but patch also pending; Java Engine patch pending

Claimed unaffected / not vulnerable (no action taken or required)

• 3CX (community thread only)
• ActiveState
• Adaptive (unaffected)
• ADP
• AlienVault
• Alteryx (unaffected - oddly, statement currently only available as a banner at the top of community pages)
• Anaconda (unaffected, but there is component involvement)
• Apache Guacamole
• Apache Spark (unaffected, though use of 1.x is also a security risk)
• ArcticWolf
• Asana (unaffected)
• AutoDesk / AutoCAD (official, but updates will be in Trust Center)
• Automox - no related components
• AVM Fritzbox (claimed unaffected per support reply - link needed
• Amazon AWS (packages in repo using default config) - more affected in second bulletin; see also list of AWS environment variables (h/t Greg Linares); see also hotpatch post and hotpatch GitHub; AWS OpenSearch domains need to be patched by customer
• Axio - no related components
• BigBlueButton (official reply to GitHub issue)
• bisq (mitigations as a precaution only)
• Bitnami (packages in repo using default config)
• BlackKite (official)
• Box (official)
• Broadcom Layer7 API Gateway
• Caddy
• Calyptix - no products affected
• Cato Networks - unaffected; cloud IPS now also has rules
• Cerberus FTP - uses log4j rewrite
• Chaser discrimiNAT - claims no related components
• Check Point - community thread discusses log4j present in some products; see SK176865
• Cloudron - claimed no impact
• Commvault - claimed "no affected versions" - could be 1.x though?
• curl / libcurl - unaffected (official tweet)
• Cybereason
• Dashlane (unaffected, no Java, official Tweet)
• Databricks (unaffected)
• Datto
• DirectAdmin (community forum, but official response)
• Discourse (no log4j)
• draw.io (official Tweet)
• Drupal
• Edgecast (official)
• ESET (community thread, official reply)
• F5 - all products listed as not vulnerable except Traffix SDC (pending); see also explainer
• Fast LTA - no use of log4j (Silent Cubes and Silent Bricks instead)
• FileWave (claimed no log4j, used unaffected log4qt C++ port until version 12.4)
• FireDaemon (no Java involvement at all)
• Flywheel - claimed unaffected per one support report - verify with vendor
• Forcepoint (claimed vulnerable, support wall - NGFW, SMC, Web Security, DLP, Security Manager)
• FreeIPA (community, official answer)
• FusionAuth - unaffected (uses logback, a different framework)
• Gearset (official)
• GIMP (official)
• Grafana (official)
• Grandstream (official)
• H3 (Scale Computing, official) - includes 1.x but claimed mitigated for 1.x issues (support wall)
• Hashicorp (claimed unaffected)
• HomeAssistant (unaffected)
• Hubspot (claimed unaffected)
• i2p (unaffected - no direct use, and no indirect use in jetty, etc.)
• Infoblox (reported unaffected except NetMRI still being investigated, but support wall) - see also updated guidance
• Informative (unaffected, monitoring)
• Inflectra (unaffected)
• InfluxDB (unaffected, log4j absent)
• IronCore Labs (unaffected, official tweet, uses logback)
• JFrog
• Jotform
• Keycloak
• Kong - multiple private emails saying support says no Java at all - link needed
• LeoStream
• LiquidFiles (no use of log4j)
• Linksys (no local use of log4j - cloud use not impacted - but only as of 2021-12-13, seems stale)
• Mikrotic (forum)
• MISP (tweet, official)
• Moodle (official)
• Nagios (tweet, official)
• NASA Ingenuity helicopter (search page for 'log4j')/li>
• OpenVPN (official)
• Palo Alto (Panorama and all other products) - component list shows log4j
• Peplink (official)
• Percona (official)
• Perforce (official)
• Pexip (official)
• PiHole (official Reddit reply)
• Pitney Bowes (official)
• Portainer (official)
• Postman (official)
• Progress MOVEit (formerly ipSwitch) - 1.x only, claims no workaround needed
• ProtonMail unaffected
• Proxmox (community thread with official response - also, using Xenforo but not Elasticsearch
• Pulseway (official) - nice coverage of analyzed components
• QF-Test - unaffected
• Quest (general statement, no direct links to resources)
• Quest KACE (support wall) - see reddit thread
• Redis - not directly, but can relay/forward/append
• ReportURI - unaffected, good writeup on validation approach
• Remediant (official tweet)
• Red5
• RingCentral (multiple products, all claimed unaffected)
• SentinelOne (official)
• Shibboleth
• Smile CDR
• Netwrix / Stealthbits - multi, all unaffected at this writing
• Synology (official) - see also community post
• Tealium
• Team Password Manager
• TechSmith / Camtasia - no components
• ThreatLocker - no components
• Tridium - Reddit post with pasted claimed-official answer, confirmation shared privately - but verify
• Tripwire
• Veeam (official)
• Vespa Engine
• Watchguard
• Whatsup Gold (unaffected, use of Elasticsearch claimed unaffected)
• WildFly (official tweet)
• Wireshark (official)
• Worksphere (official)
• XCP-ng (community thread w/official reply)
• XiltriX (official)
• Zabbix (official)
• Zerto (official)
• Zesty.io
• Ports of log4j to other non-Java languages (log4perl, log4php, log4net, and log4r, etc.) - likely unaffected due to this vulnerability being Java-dependent

Claimed unaffected by default (but configurable to be affected if user opted for log4j or added extensions)

• Eclipse (Passage only, 1.x patched as a precaution - all others claimed unaffected)
• Jenkins (but plugins may use log4j)
• Spring framework

Multi-product - vulnerable, mixed, or not yet fully determined

• Alcatel-Lucent
• AppDynamics - multiple products with known vulnerable components, upgrades and workarounds listed
• ArcGIS (Esri) - version 10.8 claimed OK; 10.7.1 and earlier unsupported and "assumed vulnerable" - see also community thread, claims presence of log4j, and official blog post
• Arista - multiple products
• Avaya - placeholder only as of 2021-12-13 00:44 UTC
• Avigilon - most unaffected except Compass as of 2021-12-16 00:42 UTC
• Axis (official, mixed, PDF)
• BigFix (HCM) - some affected
• Bitdefender
• BMC mitigation released (support wall), according to private email, as of 2021-12-14 16:38 UTC. Public links available for Control-M and Helix
• Boomi - community post, addressed / pending per indirect tech-support reply
• Broadcom including CA Automic and related. SiteMinder has a fix
• Carbon Black - separate refs for Cloud Workload Appliance and Carbon Black EDR Server and App Control
• Cisco (placeholder / rollup)
• Citrix (multiple affected products)
• ConcreteCMS (products, cloud, WAF detection)
• ConnectWise - on-prem vs cloud, various mitigations, FortiSIEM included in StratoZen, etc.
• Dell (official response rollup, previous rollup (multiple products, including Wyse Management Studio (support wall); iDrac claimed unaffected); see also /r/sysadmin post
• Extreme Networks (multiple products)
• Eaton - no known public or supportwall references, but customer emails reported as mix of products, with IPM2, VCOM, Network Manager, and Yukon in the affected list, EATS30 under investigation, all others unaffected (as of 2022-01-10 08:04 UTC)
• ForgeRock (multiple products)
• FortiNet (multiple products); see also community discussion
• F-Secure (multiple affected products)
• GitHub (multi-pronged activity)
• GoAnywhere (patched releases and mitigation guidance)
• Google Cloud (some unaffected, regularly updating - Looker added 12/18)
• Gradle (blog post)
• HMS Networks (multi / varies - titles tagged "log4shell")
• HPE (placeholder / pending, points to general alerts page)
• Huawei (placeholder / rollup)
• IBM (multi / rollup) - you can get an ID and then sign up for notifications to get product alerts
• Jetbrains (TeamCity, general) - see also blog post, some affected, update
• Juniper
• Kaseya (pending/ rollup)
• Kofax (multi)
• community post roll-up
• LanSweeper - brief "nothing yet, in progress" mention
• Lepide - most unaffected, others "patch available 12/17"
• Maltego (multiple products)
• McAfee (multiple products, status being updated - KB95091)
• Microsoft's response doc - includes products breakdown, direct and indirect
• Mitel - investigation in progress
• MongoDB - none affected except Atlas Search (patched)
• N-Able (multiple components)
• Nagios (multiple components)
• NetApp - large list, mostly unaffected other than Cloud Manager and SnapCenter Plug-in for VMware vSphere; SANnav pending; see also overall response doc
• Nextcloud (unaffected)
• Nexus (pending)
• Nutanix - PDF, quite a few product listed
• NVIDIA - some components
• OpenHAB - long/mixed thread, still in motion
• PagerDuty - largely unaffected, patched quickly, some mitigations needed - nice writeup, good example of how to do it
• PaperCut - some products affected
• Polycom - multiple products
• Pulse Secure - all products listed are noted as "Not Affected" as of 2021-12-14 16:44 UTC
• PureStorage (mixed / placeholder / mitigations)
• Qlik - products in various states
• QNAP (official) - mostly unaffected, investigating third-party
• Quantum (official)
• Rancher (official forum) - most unaffected, but "The Helm chart for Istio v1.5, provided by Rancher and which is currently deprecated since Rancher v2.5.0, includes Zipkin and is vulnerable to Log4j"
• Rapid7 (rollup)
• RedHat (rollup / in progress)
• Ruckus/Brocade (multiple products) - see also community post
• Sailpoint (multiple products, various statuses, no mention of IIQ)
• Salesforce (rollup / in progress, including Tableau); unofficial workaround
• Software AG (rollup, points to articles for ARIS, Cumulocity, webMethods)
• Solarwinds (multiple products)
• SonicWall (rollup / in progress)
• SUSE (no direct impacts except OpenStack Cloud, updated pending)
• Splunk (official, in progress, some components affected)
• StormShield (official, multi, mostly unaffected, near bottom)
• Symantec (rollup - lots of products declared non-affected - SYMSA19793)
• Tibco
• US Signal (MSP)
• Venafi - all unaffected except UniCERT CA
• Veritas (NetBackup, references mitigation steps), full app list
• VMware (placeholder KB), advisory - couple updated, including NXS-T DC, vROps, vROps Cloud, VMware Horizon, VMware HCX - see also blog and updated guidance on removing classes and FAQ; updated 2021-12-16 14:15 PST: Added new remove_log4j_class.py; also consolidated blog
• Zebra (multi, lots of patches, some pending)
• Zyxel (multi, unaffected as of 2021-12-14 16:54)

Potentially affected (circumstantial use of log4j or behind support wall)

• ColdFusion (official), and community thread and third-party blog post
• Adobe Experience Manager (community thread only)
• Algosec (support wall)
• Apache Cassandra - log4j not included, but opt-in appender config is possible
• Apache Dubbo
• Apache Geode
• Apache Hadoop
• Apache Tapestry
• Apache Wicket
• Apple Xcode (community thread, official response "aware of this issue")
• Avantra (support wall)
• Autopsy - claims use of Apache Solr, link needed, tweet mention
• Adobe Connect (community thread with mitigations)
• Blackbaud (claimed unaffected, but behind support wall)
• Blackberry (integration doc)
• Bomgar (support wall) - claimed only impact is Privilege Management Reporting, patch TBA, as of 2021-12-14 16:23
• Cellebrite Blacklight - claimed use of Elasticsearch (so info leak only?) - link needed, tweet mention
• Cohesity - support wall
• CyberArk - support wall
• DigiCert (integration doc only)
• Docker - context and updated software
• Dynatrace (official) - see also blog post and community post from before the official; Synthetics only?
• EJBCA (certificate authority server) - Hacker news thread only, claimed uses 1.2.17, so 1.x caveats may apply
• Entrust IdentityGuard (PDF) - indirect RSA ref, claims log4j use
• Gigamon (support wall)
• Guidewire (support wall)
• Grails
• Honeywell non-answer answer "If needed, we will provide our customers with any necessary corrective actions."
• IBM QRadar - Reddit thread - need link to any articles behind support wall
• iManage (support wall) - multiple products may be affected, including Work Indexer, Records Manager, Security Policy Manager, Threat Manager (variou versions)
• Ivanti MobileIron, etc (support wall) - see also ZDnet article
• Jamf and community thread (has mitigation specifics)
• Keysight / Ixia (rollup, individual products behind support wall))
• Meinberg
• Micro Focus ZENworks Configuration Management (support wall)
• Mulesoft (support wall)
• OpenFire (workaround + notice of impending release)
• ProofPoint (support wall), but also public statement (PDF), and official news
• Radware (only abstract rollup is publicly available - support wall)
• Ricoh (placeholder only - 2021-12-16 06:37 UTC)
• Rockwell Automation (support wall)
• Roon (community discussion)
• Rubrik (unknown - support wall)
• Rundeck (placeholder, 2021-12-11 23:33 UTC - includes mitigations)
• SAP Business Objects (integration doc)
• ServiceNow (? - support wall), community thread
• Sailpoint Identity IQ (integration doc only)
• SonarQube (community thread with workarounds)
• Synopsys (support wall, but it's free signup)
• Thales (support wall)
• TP-LINK Omada SDN (community thread)
• Trend Micro (in progress, bottom section)
• TrueNAS (brief community discussion)
• vArmour (support wall)
• Varonis (support wall)
• Zimbra (integration doc)
• Zoom (investigating)

Not yet determined, non-commital, or mixed/controversial

• Apache Kafka claimed unaffected, but this pull request seems to show otherwise; CVE list indicates patch activity
• Blender (original claimed PoC was apparently a joke)
• DocuSign ("patching or mitigating as vulnerable configurations are identified")
• Elastic - claimed no RCE in Elasticsearch, but an info disclosure vuln was patched, but this community thread is mixed (2021-12-11 23:49 UTC). Elastic Cloud on Kubernetes requires mitigations. Other components/offerings listed as unaffected.
• Flexera / Revenera (placeholder / in progress - 2021-12-13 02:56 UTC)
• Mathematica (community forum only, not definitive)
• Nextcloud (no Java per GitHub issue, but replies indicate potential for otherwise)
• SmartBear (official) - working on mitigations, no statement of direct log4j involvement
• Untangle (community forum only, no authoritative answer 2021-12-13 04:21)

Indirect / integration known (can relay/forward/integrate, but no default dependency)

• Acunetix - "logging features of the Java IAST AcuSensor might be using Log4j indirectly, as the library will be used by Java if it is available in Tomcat"
• Apache Tomcat
• Camunda (forum post, can integrate but not directly affected)
• Elastic Kibana (integration doc)
• IBM Enterprise Records (integration doc)
• Lightbend (uses logback and/or slf4j by default)
• ODL OpenflowPlugin / OpenDaylight (integration doc)
• Oracle Fusion Middleware / WebLogic (integration doc)
• QNAP (evidence of integration - link to support page, no info so far)
• Rapid7 InsightOps (integration doc)
• SAP Cloud Foundry
• SAP Commerce (integration doc)
• Selenium (integration doc only, depends on implementation)
• Sentry.io (log4j docs only - need statement of posture)
• Sun Java Instant Messaging (integration doc)
• Splunk (integration doc)

Other rollup lists

• NCSC-NL has a GitHub-based list, should scale better
• SwitHak gist - big list of many software packages, watch comments for near-realtime interim adds
• Kevin Beaumont is working on a centralized repo - announcement tweet, WIP
• See also YfryTchsGD/Log4jAttackSurface - PoC screenshots for cloud and non-cloud software
• Follow Arnim Rupp's GitHub stars for a steady stream of useful tools and repos
• authomize list of affected software (CSV)
• Snyk Labs list - compact but solid list of related references
• CERT-CC - includes filterable list of vendor statements, ~1500 and growing
• list of storage/backup systems affected

back to top

Detection

Finding potentially vulnerable software

• Important to know where log4j is in your environment - authenticated vuln scanners ideal, but some appliances won't be visible and may have unknown Java components
• Work the inventory angle hard and early. Like Shellshock, this is the only long-term winning strategy
• But also Assume compromise - instrument for detection of lateral exploitation and movement
• Best Unix-like: yahoo/check-log4j (Yahoo! Paranoids) -- pure portable shell - find pre-2.16-ish JndiLookup.class, optionally remove class
• EmergingThreats/log4shell-detection - not a tool, more an overview of detection considerations and background
• hashes of vulnerable log4j jar files (mubix) - add these to your EDR / AV rules to hunt for vulnerable jars
• reddit /r/sysadmin thread on detection
• Hunting with Crowdstrike
• local-log4j-vuln-scanner "JAR and WAR archives are inspected and class files that are known to be vulnerable are flagged."
• log4j-detector (mergebase)
• InfoWorld overview article
• logpresso/CVE-2021-44228-Scanner - log4j2-scan, with optional --fix parameter that backs up the .jar and strips JndiLookup.class
• Rapid7 - InsightVM and Nexpose now have detection coverage, recently fixed
• Carbon Black - official community post
• silentsignal.eu enumeration tool
• Qualys - post includes detection coverage; see also detection with Qualys Multi-Vector EDR
• scan4log4shell IBM X-Force Red, Python
• Axonius detection guidance
• Picus detection coverage
• Fenrir (Florian Roth) 0.9.0 release supports both IOC detection and vulnerable version detection
• docker scan now detects vulnerabler versions
• Tenable/Nessus coverage and blog
• CodeShield-Security/Log4JShell-Bytecode-Detector - detect vulnerable versions
• LanSweeper - detection guidance at bottom
• OpenVAS (Greenbone) detection, include in-depth info about coverage
• GitHub - Dependabot and automation to detect
• CERCTT CVE-2021-4228_scanner - jar detector - PowerShell and Python versions
• dtact/divd-2021-00038--log4j-scanner - jar detector - Go, with Windows, macOS, and Linux executables
• fox-it/log4j-finder - Python - scans on disk and inside jars
• scitotec/log4j-recognizer - Java - a class to detect when log4j is loaded, including by downstream plugins, etc.
• santosomar/log4j-ioc-detector - Bash - recursive grep to look for JNDI in expected test files
• Trail of Bits broad dependency analyzer - good for log4j and other purposes, CVEdb and OSV integration
• palantir/log4j-sniffer - Go - search filesystem for files with vulnerable classes
• SentineLabs/log4j_response - Python / PowerShell - vulnerable class search
• Qualys/log4jscanwin and log4jscanlinux- C++ / shell - vulnerable class search
• Rubrik APIs can be used to detect vulnerable jars/wars/etc
• ossie-git/log4shell_sentinel - detector made for rolling updates, CSV output, etc.

Detecting exploitation attempts

• To detect others trying to exploit, look for jndi:ldap, jndi:dns, etc in logs - but obfuscation also likely
• Some products only use log4j for local logging, so SIEM/syslog/Splunk searches are good, but may not provide full coverage
• Even if you don't have egress filtering, if you have netflow, you might be able to detect exploitation
• log4shell-detector (Florian Roth) - looking for many obfuscated combinations of potential exploitation attempts in logs - see also Ansible role to run it
• big IOC collection
• Detecting log4j 2 RCE Using Splunk, continued
• Finding applications that use Log4J - Rumble
• Check Point coverage
• McAfee NSP signature (KB95088)
• list of known source IPs attempting detection or exploit (gnremy)
• YARA and other rules (Florian Roth)
• base64 payloads (@nathanqthai)
• Detection using Elastic
• Fox-it post on detection with Suricata
• Detection with Thor (Nextron)
• Fidelis Endpoint has detections
• Trend Micro has detections in multiple products
• thomaspatzke/Log4Pot small Log4Shell honeypot - tweet
• back2root/log4shell-rex - detect a variety of obfuscation techniques
• Zeek detection coverage

Vulnerability scanning and testing

• To test your own vulnerability, pass ${jndi:ldap://[some-IP-you-listen-on]/uniquestring]} as data that might be logged (form fields, User-Agent strings, usernames, etc.) to your target, and then check your HTTP listener logs for uniquestring
• When testing, be sure to hit other vectors and protocols - IMAP, SMTP, SNMP, HTTP on alternate ports, HTTP headers, syslog, username fields ...
• Note that since a variety of processes in the data/logging chain can have latency, positive test results may be delayed
• cisagov/log4j-scanner - CISA has a scanner!
• google/log4jscanner
• log4jscan (BurpSuite)
• byt3bl33d3r HTTP server tester
• Centralize detection for your own tests with canary tokens, including a new dedicated token type
• codeql query to find vulnerable code paths (Paulino Calderon)
• log4jpwn - test environment
• ssstonebraker/log4j-scan-turbo (shell wrapper, fast (48 parallel threads), very broad spectrum - requires CanaryToken)
• PortSwigger ActiveScan++ detection support
• semgrep code scanning support
• Nuclei template
• Huntress log4shell vulnerability tester
• Aqua mitigation walkthrough
• X-Force Red analysis, includes IOCs
• fullhunt/log4j-scan - Python
• Fenrir (Florian Roth) 0.9.0 release supports both IOC detection and vulnerable version detection
• Lightrun detection Twitter thread
• christophetd/log4shell-vulnerable-app - vulnerable Spring Boot web app, using Log4j 2.14.1 (spring-boot-starter-log4j2 2.6.1) and JDK 1.8.0_181
• Cyb3rWard0g/log4jshell-lab (Microsoft Sentinel team mention) "A basic lab environment to test some of the public proof of concepts to trigger and learn more about CVE-2021-44228 and expedite the time it takes to deploy multiple scenarios"
• jerrinot/log4shell-ldap/ - Docker LDAP server for processing test responses
• proferosec/log4jScanner/ - Go
• Qualys offering free scanning for 30 days

Other defense stacks and guides

• TrustedSec log4j Playbook
• Log4jTools (MalwareTech) - just one tool at this writing but likely to be more
• Datto RMM components and community script
• Trustwave mitigation component updates (near bottom of post)
• jfrog/log4j-tools - Python - look for both log4j2 inclusion and log4j2 calls
• craiu log4j_blocklist.txt - list of domains to sinkhole internally to counter exploitation
• StormShield countermeasures
• mandiant/heyserial - broad deserialization hunting
• For-ACGN/Log4Shell - "Check, exploit, generate class, obfuscate, TLS, ACME about log4j2 vulnerability in one Go program."

back to top

Exploitation

• Visible activity tracking (Greynoise)
• PoC (tangxiaofeng7/apache-log4j-poc)
• PoC (HyCraftHD/Log4J-RCE-Proof-Of-Concept)
• BlackHat 2016 presentation
• nesting = leakage of server environment variables
• @snovvcrack impacket approach
• Openwall thread
• rogue-jndi - malicious LDAP server (Veracode)
• JNDIExploit
• PoC predates announcement by 9 months different!
• jas502n/Log4j2-CVE-2021-44228 - exploit walkthrough (Chinese)
• WAF and log-analysis bypass techniques
• pimps/JNDI-Exploit-Kit has support

back to top

News and posts

• ADN (Anchorage Daily News, Alaska) ;)
• AP
• Ars Technica - initial and coverage of cloud impact and 4-day update, and second vulnerability
• Akamai
• Bank Infosecurity
• BishopFox
• Bleeping Computer (Minecraft patch), malware push, major products
• Blumira
• CERT.at (German)
• CIRCL.LU
• Center for Internet Security (CIS), and response summary (mentions this page)
• Cisco developer response guidance
• CNN
• Crowdstrike
• CSO Online
• Cyber Kendra
• Dark Reading
• DigitalOcean
• Duo
• Engadget
• ExtraHop
• F-Secure
• Forrester (mentions this page)
• Fortinet Fortiguard
• Graham Cluley
• Hacker News and mention of this page
• HHS.gov - compact 2-page overview PDF
• The Hill
• Huntress
• JPCERT CC
• Juniper
• Kaspersky, additional post
• Lawfare
• LA Times
• Malwarebytes Labs
• Mashable
• McAfee
• Mogwai Labs
• National Law Review
• Netlab 360 - botnet involvement
• New Scientist
• NPR
• Portswigger
• PCMag, state-sponsored activity
• Praetorian
• Qualys
• The Record (Catalin Cimpanu), and CISA Dec 24 deadline (mentions this page)
• The Register
• Reuters (major companies struggle to patch)
• RiskIQ
• Security Boulevard, API concerns
• SecurityWeek
• Slashdot
• Snyk
• Sonatype
• Symantec threat intelligence
• Sysdig
• SystemTek
• TechCrunch
• Tenable (PoC), scrutiny of third-party software
• Truesec
• Trustwave
• US-CERT (brief summary)
• VentureBeat (on less-obvious uses of log4j)
• The Verge (SMS shortcode provider vulnerable)
• Vice
• Vigilance.fr (Orange Business Services)
• VMware explainer
• Wall Street Journal
• Wired, will haunt for years
• ZDNet, second vuln, MobileIron urging

back to top

Return to The Story So Far (list of notable security events)

Follow @techsolvency for security-only updates, or @TychoTithonus (me) for general/personal (and password cracking / hashcat stuff)