Preface

There is a category of irony in this field that stops being funny once you've tracked it across three consecutive fiscal years, the most systematically targeted software in the software supply chain has been security software itself. Not fintech payment APIs. Not identity federation providers. Vulnerability scanners. CI/CD integrity checking utilities. A compression library stitched into the SSH authentication plumbing of roughly half the planet's Linux fleet.

This is not bad luck. It is target selection. And the selection criteria are worth examining carefully, because they reveal something about attacker sophistication that most vendor incident reports for obvious commercial reasons, tend to understate.

This blog covers four confirmed, distinct supply chain compromises across March 2024, March 2025, and March 2026, the last two both detonating within 24 hours of each other in the same month. The individual incidents have been documented elsewhere. The correlations drawn here have not been, to my knowledge, published in this form. I'll flag specifically where I think the industry got the framing wrong.

Case Study I — XZ Utils (CVE-2024-3094): The Two-Year Patience Operation

Background

On March 28, 2024, Andres Freund, a Microsoft engineer benchmarking something unrelated on his personal Debian sid system,  noticed SSH authentication lagging roughly 500ms. That observation is the reason we're not writing a completely different kind of retrospective right now. The backdoor embedded in XZ Utils 5.6.0 and 5.6.1 scored a CVSS 10.0 and was caught three weeks before stable landing in Debian and Fedora.

The Jia Tan Long Game

The threat actor operated under GitHub handle JiaT75, presenting as "Jia Tan." Account created in October 2021. What followed was 30+ months of being a genuinely useful contributor. bug fixes, performance improvements, technically competent engagement in PR threads. The account passed code review repeatedly because the code was, in fact, correct.

This is the part most writeups spend one sentence on: the attacker did real work. That's not incidental to the operation,  it is the operation. Key milestones:

• November 2021: First commit: a modest buffering fix, passes review cleanly
• Mid-2022: Coordinated second persona Jigar Kumar begins pressuring maintainer Lasse Collin to merge Jia Tan's patches faster; a third account "Hans Jansen" applies similar pressure
• January 2023: Jia Tan granted commit access after sustained quality contributions
• February 2024: Malicious payload introduced via two binary test files: tests/files/bad-3-corrupt_lzma2.xz and tests/files/good-large_compressed.lzma

The pressure campaign is a textbook HUMINT technique, fabricating social consensus around a target, applied to open source project governance. Lasse Collin has since spoken publicly about the stress of solo maintainership under active social pressure.

Technical Architecture of the Backdoor

The build-time injection mechanism deserves more attention than it got in 2024 retrospectives. During make, the bad-3-corrupt_lzma2.xz test file was de-obfuscated using the Unix tr utility,  a string translation tool with zero association to cryptographic operations, meaning it triggers no build-time scanning heuristics from any scanner in common use. The resulting bash script injected a precompiled object file into the liblzma build stage.​

The final implant hooked RSA_public_decrypt resolution in SSHD. On systemd-linked builds where libsystemd links liblzma, the backdoor weakened public key authentication — an authentication oracle in the SSH stack. Before arming itself, it ran environment checks: x86_64 architecture, systemd presence, libsystemd in the process memory map. That specificity is why it passed undetected on macOS builds and non-systemd CI environments during pre-release testing.​

MITRE Mapping

‍

Case Study II — reviewdog → tj-actions (CVE-2025-30066 / CVE-2025-30154): The Automated Trust Exploit

The Pivot Architecture

On March 14, 2025, tj-actions/changed-files, embedded in over 23,000 repositories, was found executing a malicious Python script on CI runners. Initial reporting framed this as a single-action compromise.

The reviewdog (GitHub) org was infiltrated through GitHub's automated team invitation system, under certain repository configurations, contributors hitting activity thresholds receive automatic team invitations with write access. The attacker cleared that threshold, received an invite to @reviewdog/actions-maintainer, pushed a malicious commit, and redirected the v1 tag. Everything downstream followed from a single tag pointer change.

The full dependency kill chain:

Step 1 — reviewdog/action-setup is a utility action — it installs the reviewdog linting tool onto the CI runner. It's a building block that other reviewdog actions depend on.

Step 2 — reviewdog/action-typos (and other reviewdog actions) call action-setup first to install the tool before doing their own work. So when action-setup was poisoned, every action that used it inherited the malicious code silently.

Step 3 — tj-actions/eslint-changed-files uses reviewdog actions internally to lint changed files, so it pulled in the poisoned action-setup transitively.

Step 4 — tj-actions/changed-files is one of the most widely used GitHub Actions in existence — it detects which files changed in a PR. It internally depends on eslint-changed-files, so it too pulled in the poison.

Step 5 — 23,000+ repositories had tj-actions/changed-files in their workflows. None of them changed a single line of their own code. They just ran their normal CI pipelines and the malicious memory-scraping payload executed silently on their runners.

The Core Point: The attacker never touched the 23,000 affected repositories directly. They compromised one foundational action that everything else depended on, like poisoning a water treatment plant instead of knocking on 23,000 individual doors.

The malicious script performed direct process memory access via /proc/{PID}/mem on the GitHub Actions Runner process:

• Enumerated PIDs matching Runner.Worker, Runner.Listener, runsvc, run.sh
• Parsed /proc/{PID}/maps to locate active memory regions

Regex-scanned raw memory for GitHub's internal secret JSON structure: "name":{"value":"...","isSecret":true}

MITRE Mapping

‍

Case Study III — Aqua Security / Trivy (March 2026): The Blockchain-Backed, Multi-Stage Infostealer

Three Waves, Three Different Failure Modes

Wave 1:  pull_request_target Misconfiguration (Late February 2026)

An autonomous bot, hackerbot-claw, exploited the pull_request_target trigger in Trivy's GitHub Actions workflow. Unlike the standard pull_request event, this trigger executes the base branch workflow with full write-scope access to repository secrets when an external PR is opened. A privileged Personal Access Token was extracted.​

Wave 2: Incomplete Remediation (March 1–19, 2026)

Aqua publicly disclosed Wave 1 and rotated credentials on March 1. The rotation missed the aqua-bot service account token, which had been provisioned with identical release-signing permissions to the PAT that was rotated. The attacker retained residual access for 18 days through the unrevoked bot token.​

Wave 3: Tag Poisoning + Binary Backdoor (March 19, 17:43 UTC)

Using aqua-bot's retained credentials, the attacker force-pushed 76 of 77 version tags in trivy-action and all 7 in setup-trivy to a malicious commit, while simultaneously publishing trivy v0.69.4 through the compromised release automation. The only surviving clean tag — v0.35.0 — was anchored by GitHub's immutable releases opt-in feature.

Payload Forensics: The entrypoint.sh Delta

A 6.16× file size increase. The GitHub release UI surfaced no indicator of this change. No mainstream SCA tool performs artifact file integrity monitoring on action entrypoints across pipeline runs.

The 105-line injected block executed a full credential sweep before invoking the legitimate scanner, so pipelines appeared to succeed normally while being drained. The sweep branched on runner type: GitHub-hosted runners got process memory scraping (identical technique to tj-actions); self-hosted runners got a broad filesystem trawl targeting SSH keys, AWS/GCP/Azure credentials, Kubernetes tokens,

Docker configs, Terraform state, Helm secrets, and cryptocurrency wallet files. Harvested data was encrypted with AES-256-CBC under a session key wrapped with RSA-4096, rendering the payload.enc irrecoverable without the attacker's private key even if the binary is captured and analyzed.​

The Blockchain C2: The Non-Takedownable Infrastructure Problem

The trivy v0.69.4 binary dropped ~/.config/sysmon.py, named to blend into environments running legitimate sysmon tooling, slept 5 minutes (sandbox evasion), then polled C2 at:​

hxxps://tdtqy-oyaaa-aaaae-af2dq-cai.raw.icp0.io

This endpoint is served by the Internet Computer (ICP) blockchain, operated by the DFINITY Foundation. No registrar. No DNS operator. No hosting provider. No abuse@ address that produces a takedown. The standard incident response playbook, registrar complaint, BGP null-route, CISA notice,  does not function against ICP-hosted infrastructure by design. Blocking *.icp0.io at the network perimeter works, but will generate increasing collateral damage as legitimate ICP applications grow in deployment.

The fallback exfil path was equally considered: if primary C2 was unreachable, the malware created a public GitHub repository named tpcp-docs in the victim org, published a timestamped release, and uploaded the RSA-encrypted credential bundle as a release asset, routing exfiltration over objects.githubusercontent.com, a domain organizations cannot practically block without breaking their CI/CD pipelines.

MITRE Mapping

Case Study IV — litellm v1.82.8 (PyPI, March 24, 2026): The .pth Auto-Execution Escalation

Background

LiteLLM, one of the most widely used Python libraries for routing calls across OpenAI, Anthropic, Cohere, and other LLM APIs, had versions v1.82.7 and v1.82.8 published to PyPI with a malicious payload. The packages were uploaded without a corresponding GitHub tag or release, meaning the standard release pipeline was bypassed entirely and the attacker pushed directly to the package registry using compromised maintainer credentials.

The GitHub issue raised by the community was closed as "not planned" within hours and immediately flooded with hundreds of bot accounts to dilute the discussion thread — a suppression technique that suggests pre-planned operational security, not an opportunistic actor.

The .pth Auto-Execution Mechanism

This is the technical escalation that distinguishes the litellm attack from everything before it in this report. Previous attacks required a CI/CD pipeline execution or an active SSH session to trigger the payload. The litellm malware used a Python .pth file named litellm_init.pth.

Python .pth files are path configuration files processed automatically by the Python interpreter on every startup, no import statement, no explicit call, no user action required. Any Python process launched on a machine with the package installed runs the payload silently and automatically from the moment of installation. That includes local developer laptops, not just CI runners.

The mechanism creates an unintended fork bomb side effect: the .pth file spawns a child Python process via subprocess.Popen, which itself triggers the same .pth file on startup, exponentially forking processes until the machine crashes. This was a bug in the malware, a lock file or PID check would have made it run silently indefinitely. Ironically, the crash is what got it noticed.

Three-Stage Payload

MITRE Mapping

The Gap: Where ATT&CK Falls Short for CI/CD

Three specific attack vectors from these incidents have no clean ATT&CK mapping:

1. pull_request_target privilege escalation
The exploitation of GitHub's pull_request_target trigger to access base branch secrets from an external, untrusted PR is a CI/CD-specific initial access technique. T1195.001 is the closest mapping but it describes dependency compromise, not workflow trigger abuse. There is no ATT&CK technique for "abuse of CI/CD workflow trigger privilege scope."

2. Transitive dependency propagation as lateral movement
The reviewdog → tj-actions kill chain involved no credential theft, no network pivot, and no exploited vulnerability in the traditional sense. It was pure trust inheritance through a dependency graph. T1199 (Trusted Relationship) approximates it, but that technique was designed for third-party IT provider relationships, not automated package dependency graphs. The distinction matters for detection engineering.

3. Blockchain-hosted C2
T1102 (Web Service) is the closest mapping for ICP blockchain C2, but T1102 was designed for attacker use of legitimate web services like Twitter or Pastebin. ICP blockchain infrastructure is categorically different, it is decentralized, immutable, and has no abuse reporting path. The takedown resistance profile of blockchain C2 is not captured anywhere in the current ATT&CK framework. This is a genuine gap that MITRE will need to address as this technique proliferates.

Cross-Incident Correlations: What The Vendors Aren't Publishing

The March Effect

XZ Utils: March 2024. tj-actions/reviewdog: March 2025. Trivy/Aqua: March 2026.

Some of this may be disclosure timing rather than detonation timing,  investigations take weeks. But there's a structural argument: March is when Q1 release cycle pressure peaks, when core maintainers are pulled to conference travel (RSA, KubeCon EU, S4 all fall Q1–Q2), and when enterprise Q1 deployment cycles mean downstream adoption of new releases is at annual highs. A threat actor instrumenting GitHub commit frequency and PR review latency as attention proxies would empirically find March to be the optimal exploitation window. Three consecutive Marches suggests this hypothesis is worth taking seriously.

The Security Tool Targeting Thesis

The three compromised projects, xz-utils (embedded in sshd via liblzma), trivy-action (security scanner), tj-actions/changed-files (CI change detection), share a property that application libraries do not: they run with elevated implicit trust at the privileged intersection of an organization's entire deployment surface.

A compromised security scanner doesn't harvest the credentials of one pipeline. It harvests credentials for every target it's scanning on behalf of the organization,  container registries, cloud accounts, IaC configurations, Kubernetes clusters. The blast radius multiplier of a compromised security tool is categorically higher than a compromised application library. Threat actors have demonstrably figured this out. The defence community has been slower to articulate it.

C2 Infrastructure Evolution: A Clear Progression

The progression from passive implant → ephemeral log exfil → typosquatted HTTPS → GitHub CDN → blockchain C2 describes a systematic search for takedown-resistant infrastructure. ICP is the current endpoint of that search.

The Incomplete Remediation Trap Is Structural

In both tj-actions and Trivy, post-disclosure remediation left residual access intact. Aqua Security confirmed the aqua-bot token survived the March 1 rotation sweep. This is not an Aqua-specific failure, it reflects how bot tokens are universally managed: provisioned once, embedded in repository secrets, and excluded from the rotation discipline applied to human credentials.​

Human PATs live in secret managers with audit trails. Automated service account tokens have no owner, no expiry policy, and are consistently the first thing missed in a rotation sweep. The structural fix is obvious and not widely implemented: bot tokens scoped to release and tag write access should rotate on the same schedule as privileged human credentials and should be the first audited in any post-compromise review.

Commit Metadata Spoofing: GitHub UI Timestamps Are Unreliable

The malicious Trivy commit carried author timestamps backdated to 2024-07-09, set via GIT_AUTHOR_DATE and GIT_COMMITTER_DATE, with the author name copied from a legitimate prior committer and a commit message reading "Upgrade trivy to v0.53.0 (#369)". The GitHub release UI displayed this metadata identically before and after tag repointing. No visual indicator reflected the change.​

Any incident response timeline reconstruction that anchors to GitHub web UI metadata is working with evidence that an attacker can forge with a single shell command. The only forensically reliable anchor is the commit SHA, verified against SLSA provenance attestations, Sigstore transparency logs, or signed release records.

The sysmon.py Persistence Signature: Shared Toolkit or Shared Actor?

The Trivy binary (v0.69.4) dropped a backdoor at ~/.config/sysmon.py. The litellm malware installs persistence at ~/.config/sysmon/sysmon.py with a matching systemd user service. The naming convention, the directory structure, the AES-256-CBC + RSA-4096 encryption scheme, and the IMDS metadata endpoint harvesting are identical across both incidents — and both incidents detonated in March 2026, within five days of each other.

This is either the same threat actor reusing the same tooling framework across two separate targets, or two different actors using a shared commercial/underground malware kit. Either interpretation is more alarming than coincidence. The sysmon.py naming pattern should now be treated as a threat cluster indicator, any file matching ~/.config/sysmon*.py on a developer machine or CI runner warrants immediate triage.

Detection Engineering: What Standard Tooling Misses

Signal 1 — Entrypoint File Size Monitoring

Record the byte-size of every downloaded GitHub Action entrypoint.sh per {action}@{sha} at pipeline execution time. Alert on deviations exceeding 50% from established baseline. The Trivy injection, a 6.16× size increase, would have been caught before a single credential was stolen. This is a 10-line wrapper around your runner bootstrap. Almost nobody has built it.​

Signal 2 — Dangling Commit Detection

The malicious Trivy commit e0198fd2b6e1679e36d32933941182d9afa82f6f is unreachable from the master branch, it exists only because the tag references it. GitHub's API exposes commit reachability. 

Signal 3 — Blockchain Domain Egress from CI Runners

GitHub-hosted runners have no legitimate reason to contact *.icp0.io or *.ic0.app. Network-level blocking on CI runner subnets plus DNS query logging for these domains would have provided pre-exploitation detection of the Trivy binary's C2 polling cycle.

Signal 4 — Novel Token/Subnet/Repo Triples in GitHub Audit Logs

Flag any git.push event where the tuple (hashed_token, /16 subnet prefix, target_repo) has not appeared in the prior 7-day baseline. Legitimate developer activity rarely introduces all three dimensions simultaneously. Stolen tokens used from attacker-controlled infrastructure to push to unfamiliar repositories will.

Signal 5 — PyPI Package Without Corresponding GitHub Release

The litellm attack bypassed GitHub entirely by uploading directly to PyPI using compromised maintainer credentials. Monitor your Python dependency lockfiles for any package version that has no corresponding GitHub release tag or signed provenance attestation on PyPI. Tools like pip-audit combined with PyPI's provenance API can flag version-to-release mismatches. Any package version on PyPI with no matching GitHub release should be treated as unverified until proven otherwise.

Top 10 Takeaways for Security Teams

1. Stop using version tags in GitHub Actions, use commit SHAs

Tags are mutable pointers. An attacker with repo write access can silently repoint a tag to malicious code with a forged timestamp and zero UI indication. A full commit SHA is cryptographically fixed and eliminates the entire tag-poisoning attack class.

2. After any CI/CD compromise, audit bot tokens before anything else.

Human PATs get rotated; bot tokens get forgotten. In both the tj-actions and Trivy incidents, unrevoked service account tokens with identical permissions gave attackers continued access after public disclosure. Maintain a dedicated inventory and rotate bot tokens on the same 90-day cycle as privileged human credentials.

3. The security scanner is the highest-value target in your pipeline.

It holds simultaneous credentials for cloud accounts, container registries, Kubernetes clusters, and IaC configs, everything it scans on your behalf. A compromised scanner doesn't expose one pipeline; it exposes every target that pipeline is authorized to reach.

4. Monitor GitHub Action entrypoint file sizes between pipeline runs.

The malicious Trivy entrypoint ballooned from 2,855 to 17,592 bytes, a 6.16× increase that no tool flagged. A simple size baseline per {action}@{sha}, alerting on deviations above 50%, would have caught it before a single credential was stolen.

5. You are accountable for dependencies you never explicitly chose.

The tj-actions attack hit 23,000 repositories through a utility action three dependency levels upstream that none of those teams ever referenced. Map your full transitive Action dependency tree and flag anything with write access or arbitrary shell execution that you cannot directly account for.

6. Block blockchain egress domains on CI runner subnets proactively.

The Trivy C2 was hosted on the ICP blockchain at *.icp0.io. No registrar, no hosting provider, no takedown path exists. Network-layer blocking is the only available control, and it must be in place before an incident, not in response to one.

7. GitHub commit timestamps are forensically unreliable.

GIT_AUTHOR_DATE and GIT_COMMITTER_DATE are freely spoofable. The malicious Trivy commit was backdated to 2024 with a copied author name and a plausible message. For any post-incident investigation, only the commit SHA validated against SLSA provenance or Sigstore transparency logs can be trusted.

8. pull_request_target gives external PRs access to your secrets.

Unlike pull_request, this trigger executes base branch workflows with full secret scope even on forks from untrusted contributors. Audit every workflow using this trigger and apply least-privilege scoping. If there is no documented justification for using it, switch to pull_request.

9. Treat March as your highest-risk month and plan accordingly.

Three consecutive Marches, four confirmed supply chain detonations, two in March 2026 alone, the second hitting the day after this report was published. Q1 release pressure, maintainer conference travel, and peak enterprise deployment cycles converge into a reliable exploitation window. Run your pre-quarter bot token audit, increase dependency monitoring, and add extra review scrutiny to security tooling repositories before March hits.

10. Precision targeting and mass exposure are not mutually exclusive.

The reviewdog compromise was engineered specifically to reach Coinbase's CI pipeline. The 23,000 other repositories harvested along the way were collateral cover. If your pipeline ran a compromised action during the exposure window, your credentials were taken, regardless of whether you were the intended target. Scope breach assessments accordingly.

None of These Incidents Were Caught by the Vendor's Own Security Tooling

This is the observation that should be in every executive summary and isn't: Aqua Security, a company whose core product is container and software supply chain security, had its own supply chain compromised in a way that its own tooling did not detect. CrowdStrike detected the Trivy compromise through behavioral anomalies in customer pipelines, not through Aqua's own scanning infrastructure. tj-actions was caught by a third-party researcher, not by GitHub's security team or any of the affected organizations' security tools. XZ Utils was caught by an engineer noticing a performance anomaly during unrelated benchmarking. In all three cases, the detection was accidental, external, or behavioral, never the result of the security controls that were nominally in place. The litellm compromise was caught because the fork bomb bug crashed machines, not because any security tool flagged the malicious package. A cleaner implementation would have run silently indefinitely. Five incidents, zero proactive detections by nominated security controls.

The End

Surprised it happened in March again? The threat actors clearly have better quarterly planning than most security teams. Three years, three Marches, three compromises, but sure, let's schedule the pen test for Q4. The tool designed to protect your secrets was the one stealing them, and nobody blinked. The attacker committed clean code patiently for two and a half years; your patch sat in approval for two and a half sprints. And somewhere right now, a CISA advisory is being drafted that will recommend pinning to commit SHAs, while a fifth incident is already being staged for next March, using a persistence path nobody has added to their detection rules yet.

References:

• https://www.aquasec.com/blog/trivy-supply-chain-attack-what-you-need-to-know/ 
• https://github.com/aquasecurity/trivy/discussions/10425
• https://thehackernews.com/2026/03/trivy-security-scanner-github-actions.html
• https://github.com/aquasecurity/trivy/security/advisories/GHSA-cxm3-wv7p-598c
• https://opensourcemalware.com/blog/teampcp-aquasec-com-github-org-compromise 
• https://www.cyberdaily.au/security/13366-code-security-platform-trivy-compromised-by-malicious-packages
• https://www.upwind.io/feed/trivy-supply-chain-incident-github-actions-compromise-breakdown
• https://www.crowdstrike.com/en-us/blog/from-scanner-to-stealer-inside-the-trivy-action-supply-chain-compromise/
• https://openssf.org/blog/2025/06/11/maintainers-guide-securing-ci-cd-pipelines-after-the-tj-actions-and-reviewdog-supply-chain-attack/
• https://github.com/advisories/ghsa-mrrh-fwg8-r2c3
• https://unit42.paloaltonetworks.com/github-actions-supply-chain-attack/ 
• https://.seqrite.com/blog/cve-2024-3094-unveiled-xz-utils-compromise-sparks-security-alarm/
• https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/