Poison in the Pipeline: How Threat Actors Hijacked PyTorch Lightning to Target the Global AI Developer Ecosystem

On April 30, 2026, a sophisticated supply chain attack quietly infiltrated one of the most trusted frameworks in the artificial intelligence development ecosystem. PyTorch Lightning, a widely used open-source Python library with more than 31,100 stars on GitHub and millions of monthly downloads, was compromised when threat actors published two malicious versions to the Python Package Index (PyPI). The attack targeted developers, CI/CD pipelines, cloud environments, and downstream repositories in what security researchers are calling one of the most technically intricate open-source supply chain incidents of 2026.

What Is PyTorch Lightning and Why It Matters

PyTorch Lightning is an open-source Python framework that sits on top of PyTorch, the foundational deep learning library developed by Meta. It provides a clean, high-level interface for training, deploying, and shipping machine learning models, abstracting away much of the engineering boilerplate that researchers and engineers would otherwise write from scratch. Its appeal spans the breadth of modern AI work: image classification, large language model fine-tuning, diffusion model training, time-series forecasting, and reinforcement learning pipelines all commonly rely on it.

Because of its position in the dependency trees of countless AI projects, the library carries enormous implicit trust from its users. A developer who runs pip install lightning does not expect to be installing malware. That trust, precisely, is what the attackers exploited.

According to PyPI download statistics, the lightning package receives hundreds of thousands of downloads per day. When malicious code is embedded in a release of such a package, the blast radius extends far beyond individual workstations. It reaches continuous integration runners, automated training jobs, cloud-hosted notebooks, containerized build environments, and any downstream package that depends on lightning as a transitive dependency. The scale of potential exposure made this attack a critical-severity incident from the moment it was discovered.

Timeline of the Attack

The incident unfolded with alarming speed. On April 30, 2026, two consecutive releases of the lightning package were published to PyPI in rapid succession: version 2.6.2, which introduced the malicious payload, and version 2.6.3, which retained the full malicious functionality while slightly modifying metadata and loader behavior to evade detection. Version 2.6.3 was not a patch or a rollback. It was a second wave of the same attack, dressed up to look like a bug fix.

Both versions were uploaded by the same publisher account, and security researchers believe they were part of a single coordinated operation rather than two separate events. The speed of the double release suggests the attacker anticipated that the first version might be quickly flagged and sought to insert a second, slightly modified version before defenses could respond.

Socket's AI-powered security scanner flagged both versions as potentially malicious just eighteen minutes after publication. A community member also identified the compromise independently and filed a disclosure issue in Lightning AI's GitHub repository. That issue was closed without a public explanation. When Socket filed a follow-up disclosure issue, it was closed within one minute by an account identified as pl-ghost, which then posted a dismissive meme in the thread, a move that confirmed, rather than allayed, suspicion that the project's GitHub account had itself been compromised.

Despite the rapid detection, the malicious versions remained live on PyPI for approximately 42 minutes before administrators quarantined the project. For a package that receives hundreds of thousands of daily downloads, 42 minutes represents a significant exposure window. PyPI administrators subsequently deleted both versions and lifted the quarantine. The last known safe version is 2.6.1, published on January 30, 2026.

How the Attack Was Carried Out

The attacker's method of entry was not a vulnerability in PyPI's infrastructure or a weakness in the lightning codebase itself. According to the project's own post-incident statement, the threat actor compromised the PyPI publishing credentials associated with the lightning package. With those credentials in hand, they cloned the legitimate open-source code, injected a malicious payload, and pushed the tampered builds directly to PyPI as versions 2.6.2 and 2.6.3. The entire attack bypassed the project's source control entirely. The GitHub repository was never touched during the publication phase, which is why there was no evidence of compromise in the source code itself.

The payload was embedded inside a hidden directory named _runtime, tucked within the package structure where casual inspection would be unlikely to notice it. A modified __init__.py file was the trigger: it launched a background thread the moment the lightning module was imported. Users received no visible output, no warning message, and no indication that anything unusual was occurring. The background thread silently executed a helper script called start.py.

The start.py script then downloaded the Bun JavaScript runtime, version 1.3.13, directly from the internet, and used it to execute a heavily obfuscated JavaScript file called router_runtime.js. The obfuscated file weighed approximately 11 megabytes and used heavy hex-encoded variable obfuscation to defeat static analysis tools that rely on pattern matching rather than behavioral inspection. Once executed, router_runtime.js became the central engine of the attack.

What the Malware Stole

The credential harvesting scope of the malware was broad and methodical. The payload was designed to target the full range of secrets that a machine learning developer or AI researcher might have present in their working environment. That included SSH private keys, shell history files, cloud provider credentials for all three major providers, GitHub personal access tokens, npm authentication tokens, API keys for third-party services, and cryptocurrency wallet files.

In modern AI development workflows, these categories of credential represent access to infrastructure of enormous value. A compromised GitHub token can grant write access to repositories that hold model weights, training scripts, deployment pipelines, and infrastructure-as-code. A compromised cloud credential can provide access to GPU-backed training clusters, data storage buckets holding proprietary datasets, and production inference endpoints. The attacker was not randomly fishing for credentials. They were targeting the specific secrets that live in AI development environments.

Once harvested, the stolen credentials and files were exfiltrated to attacker-controlled GitHub repositories. Using validated GitHub tokens, the malware then attempted to inject worm-like payloads into up to 50 branches per compromised repository, silently overwriting files with commits designed to impersonate legitimate sources, including Anthropic's Claude Code developer tool.

Persistence Mechanisms and Propagation Vectors

What distinguished this attack from simpler credential-theft campaigns was its layered approach to persistence and self-propagation. Once a developer's environment was infected, the malware planted hooks in two of the most commonly used developer tools to ensure it would execute again on future sessions, without requiring the lightning package to be imported again.

The first persistence hook targeted Claude Code, Anthropic's terminal-based AI coding assistant. The malware wrote a SessionStart hook with a wildcard matcher into the repository's .claude/settings.json file, pointing to a malicious JavaScript file at .vscode/setup.mjs. This hook fires every time a developer opens Claude Code within the infected repository, requiring no tool use or explicit user action beyond launching the session. Security researchers noted that this may represent one of the first documented instances of malware abusing Claude Code's hook system in a real-world attack.

The second persistence hook targeted Visual Studio Code. A parallel entry was written to .vscode/tasks.json using a runOn: folderOpen task configuration that executes a corresponding malicious script every time the project folder is opened in VS Code. Together, these two hooks ensured that any developer who continued working in an infected repository would re-execute the payload on every session, regardless of whether the malicious lightning package was ever installed again.

The npm propagation vector added another layer of reach. The malware modified the developer's locally installed npm packages by injecting a postinstall hook into the package.json file of those packages. It then incremented the patch version number and repacked the modified packages into .tgz tarballs. Should an unsuspecting developer publish any of those tampered packages from their local environment, the malware would propagate to npm and become available to downstream users from there. This self-replicating behavior transformed infected developer machines into unwitting distribution nodes.

If the malware held a GitHub token with write access to a repository, it also pushed a malicious GitHub Actions workflow named "Formatter." On every subsequent push to the repository, this workflow dumped all repository secrets using GitHub's own expression syntax and uploaded them as a downloadable Actions artifact. The workflow files were pinned to specific commit SHAs to give them the appearance of legitimate, versioned dependencies, a technique used to reduce suspicion during code review.

The Mini Shai-Hulud Campaign and Threat Actor Attribution

This attack did not emerge in isolation. Security researchers have assessed it as an extension of a broader, ongoing supply chain campaign that has been tracked under the name Mini Shai-Hulud. The campaign, named after the sandworm creature from Frank Herbert's science fiction series, is characterized by its self-replicating, worm-like propagation behavior and its consistent focus on harvesting developer credentials from open-source ecosystems.

Earlier iterations of the campaign targeted SAP-related npm packages, including version 7.0.4 of the intercom-client npm package, which was compromised using a preinstall hook to execute the same family of credential-stealing malware. The technical overlaps between the npm attacks and the PyTorch Lightning incident, including the reuse of JavaScript payload infrastructure, the Bun runtime delivery mechanism, and the worm propagation logic, point to a consistent threat actor operating across multiple package ecosystems.

Attribution analysis by security researchers linked the operation to a threat group identified as TeamPCP, which has previously been suspended from social media platforms for policy violations and has since established a presence on dark web infrastructure. The group has claimed ties to the LAPSUS$ cybercrime collective and has been connected to prior supply chain attacks targeting organizations including Checkmarx, Bitwarden, Telnyx, LiteLLM, and Aqua Security's Trivy project. Indicators of compromise across those incidents show consistent patterns of repository structure manipulation, credential exfiltration to attacker-controlled GitHub repositories, and the use of branch creation and deletion behavior to probe write-access permissions before attempting payload delivery.

Semgrep researchers noted that commit messages prefixed with the string EveryBoiWeBuildIsAWormyBoi and repositories carrying the description "A Mini Shai-Hulud has Appeared" serve as indicators of compromise linked to this campaign. These thematic markers, while useful as search anchors, are not exhaustive and should not be treated as the only signals worth monitoring.

The Compromised GitHub Account and Attempted Lateral Movement

The behavior of the pl-ghost GitHub account during the incident window provided important forensic evidence about the attacker's objectives beyond the initial PyPI publication. In the approximately 70 minutes following the malicious releases, the account performed six create-and-delete branch operations across multiple Lightning AI repositories. Four of those branches used random ten-character lowercase names, a pattern that Socket's researchers identified as consistent with the Shai-Hulud worm's write-access probing behavior. A fifth branch attempted to impersonate Dependabot, GitHub's automated dependency management bot, but used a misspelled identifier and the wrong namespace separator, exposing the impersonation attempt.

The attacker appears to have been testing whether the compromised pl-ghost credentials could be used to reach other Lightning AI repositories, with the goal of potentially accessing additional PyPI publishing tokens through GitHub Actions workflow execution. This would have allowed the malicious infrastructure to spread to other packages within the Lightning AI ecosystem, including litAI, utilities, and torchmetrics.

This attempted lateral expansion appears to have failed. Lightning AI's repository controls, including branch protection rules, required status checks, tag-gated PyPI publish workflows, and workflow approval requirements, blocked the path from GitHub access to malicious package publication for those additional repositories. The successful lightning upload had likely come through a separate, project-scoped PyPI token, while the GitHub credential was being used in parallel to test whether other packages could be reached. Those parallel attempts stalled before payload delivery or CI execution, limiting the attack's reach to the single lightning package.

Industry and Ecosystem Impact

The downstream implications of this attack extend well beyond any individual developer who may have installed the compromised versions during the 42-minute exposure window. The lightning package sits in the dependency trees of numerous AI and machine learning projects across the research and production landscape. Organizations building commercial AI products, research institutions running large-scale training infrastructure, and independent developers building specialized tools all commonly include lightning as a direct or transitive dependency.

Any machine that imported the malicious package during the affected window should be treated as fully compromised. This is not a precautionary recommendation based on theoretical risk. The malware executed automatically on import, without requiring any secondary user action. The execution was silent. There was no user-facing output, no process that would have appeared suspicious in a task manager, and no log entry generated by the malware itself that would alert a developer to its presence. By the time a user became aware of the compromise, credential exfiltration had already occurred.

The incident is tracked under Sonatype identifier sonatype-2026-002817 and carries a critical severity rating. At the time of initial disclosure, no CVE had been assigned, a reminder that the absence of a CVE number does not indicate low risk in supply chain attacks of this nature.

Response and Remediation

The project's maintainers confirmed that the attack compromised their PyPI publishing channel through stolen credentials, not the source code repository itself. The GitHub source code was not modified during the attack. They have since begun an investigation into how the publishing credentials were obtained, rotated all relevant tokens, and published a security advisory recommending that all users of the affected versions take immediate remediation steps.

Organizations and developers who installed or updated to versions 2.6.2 or 2.6.3 are advised to treat all affected environments as compromised and take the following steps. First, immediately pin the lightning dependency to version 2.6.1, the last verified clean baseline, or remove the package entirely until a fully audited clean release is available. Second, rotate all credentials, tokens, API keys, SSH keys, and cloud secrets that were present in the affected environment. Third, audit all repositories for the injected persistence files, specifically unexpected contents in .claude/settings.json and .vscode/tasks.json, as well as any newly added GitHub Actions workflows such as the "Formatter" workflow described in the attack analysis. Fourth, review CI runner environments for signs of credential exposure, paying particular attention to any Actions artifacts named format-results that may have captured and uploaded repository secrets.

Developers should also audit their local npm package installations for modified package.json files with unexpected postinstall hooks, and review any packages published from affected machines for signs of tampering, including unexpected version bumps or payload injection.

What This Attack Reveals About Open-Source Security

The PyTorch Lightning incident is a textbook example of how the trust model underpinning open-source package distribution can be turned against users. PyPI, npm, and similar registries operate on a foundation of trust: when a package is published under a recognized name by a recognized publisher, users assume the contents are legitimate. Attackers who understand this trust model do not need to break into PyPI's servers. They simply need to obtain the publishing credentials of an existing, trusted package maintainer, after which the registry's own infrastructure becomes the delivery mechanism for the attack.

The 42-minute window between publication and quarantine illustrates both the speed of modern threat detection and the speed of modern attacks. Socket's AI scanner flagged the malicious versions within 18 minutes of publication. PyPI administrators acted quickly to quarantine the package. Yet even within that short window, the scale of lightning's daily download volume means a meaningful number of developers, automated build systems, and cloud-hosted workflows may have pulled and imported the malicious versions before protections were in place.

The use of version 2.6.3 as a decoy, presenting itself as a potential fix for 2.6.2 while retaining the identical malicious payload with modified evasion metadata, demonstrates a level of operational sophistication that goes beyond opportunistic attacks. The attacker anticipated the detection lifecycle and attempted to extend the window of successful infection.

The abuse of developer tools as persistence vectors, specifically Claude Code's hook system and VS Code's task runner, signals an evolution in how supply chain attackers think about longevity. Rather than relying solely on the compromised package remaining installed, the malware embedded itself into the developer's daily workflow through tools that are opened and used repeatedly throughout the working day. This approach ensures re-execution without requiring the original attack vector to remain active.

The Broader Pattern of AI Ecosystem Targeting

The choice of PyTorch Lightning as a target was not incidental. AI and machine learning development environments are particularly attractive targets for credential-stealing campaigns because of the concentration of high-value secrets they typically contain. A machine learning engineer's workstation or CI runner routinely holds cloud provider keys with access to GPU compute clusters, object storage buckets containing proprietary training datasets, model registries, experiment tracking systems, and deployment pipelines. In many organizations, the same credentials used for training are also used for production inference, meaning a single compromised token can provide access across the entire AI product lifecycle.

The TeamPCP campaign's progression from SAP-related npm packages to the PyTorch Lightning ecosystem reflects a deliberate targeting strategy. By moving from enterprise development tooling to AI-specific frameworks, the attackers expanded their potential harvest to include the rapidly growing community of developers building on top of large language models, diffusion models, and other modern AI architectures. These developers increasingly work with API keys for model providers, fine-tuning infrastructure credentials, and cloud accounts with substantial compute budgets attached.

The broader Mini Shai-Hulud campaign's pattern of hitting npm, PyPI, and container registries in coordinated waves within short timeframes suggests a well-resourced operation with clear intelligence about which packages carry the most downstream impact and which maintainer accounts are most likely to have access to multiple high-value repositories through shared credentials or GitHub organization membership.

Key Takeaways for Security Teams

The PyTorch Lightning supply chain attack reinforces several critical principles for organizations operating in environments where open-source dependencies are a routine part of the software delivery pipeline. Dependency pinning is not merely a reproducibility practice but a security control that limits the exposure window when a trusted package is compromised. Automated dependency scanning tools that operate at behavior level, rather than relying solely on known-bad signatures or version lists, provide detection capabilities that static analysis cannot match. The detection of this attack within 18 minutes by Socket's AI scanner demonstrates the practical value of that approach.

CI and CD environments require the same credential hygiene disciplines as production systems. Secrets present in build runners carry real-world value to attackers, and the assumption that build infrastructure is a lower-risk environment than production has repeatedly proven to be wrong. Short-lived credentials, least-privilege token scoping, and regular rotation reduce the harvest value of any single compromise. Repository controls, specifically branch protection, required status checks, and workflow approval gates, can prevent a compromised GitHub token from becoming a pathway to publishing malicious package versions to a downstream registry.

Finally, this incident is a reminder that the absence of visible indicators is not evidence of safety. The malware executed silently, left no user-facing traces, and in many cases would have completed its work without the affected developer ever realizing that anything had happened. Organizations that rely solely on manual inspection or reactive incident response will consistently lag behind attackers who have designed their payloads specifically to evade those defenses.