FORGE-05
Supply Chain Compromise in AI Frameworks
Unverified hardware, firmware, images, drivers, packages, models, or infrastructure artifacts can introduce compromise through trusted supply paths.
FORGE-05 Supply Chain Compromise in AI Frameworks
Definition
AI Infrastructure Supply Chain Compromise refers to the introduction of compromised, tampered, or untrusted components into AI environments through upstream supply-chain relationships. This includes hardware, firmware, software artifacts, images, and infrastructure templates that cross from vendors, integrators, registries, public sources, or internal mirrors into the operator’s trusted environment.
Description
The AI infrastructure supply chain spans two lanes. The physical lane includes servers, accelerators, NICs, storage devices, BMCs, and other components that pass through manufacturers, integrators, logistics providers, and refurbishment paths before reaching production. A system may arrive with firmware, management configuration, or node images prepared by a third party, and the operator may not have independently verified that state before deployment. The digital lane includes container images, GPU drivers and runtimes, Python packages, Kubernetes operators, model-serving components, infrastructure-as-code templates, and model artifacts pulled from vendor catalogs, public registries, open-source repositories, or internal mirrors. These components may be legitimate, maliciously tampered with, outdated, misconfigured, or pulled from an unexpected source.
Both lanes share the same core problem: components are often trusted because they arrive through an expected path, not because their integrity, provenance, configuration, and update history were independently verified. This dependency can continue throughout the product lifecycle, as operators rely on vendors to update cryptographic mechanisms when standards and threats evolve. Much of this risk is addressable with widely available controls, including signed images and artifacts, build provenance, dependency pinning, SBOMs, registry scanning, controlled internal mirrors, and safer model formats such as safetensors where applicable. The exposure concentrates where these controls are not adopted, where artifacts are pulled directly from public sources into production, or where internal mirrors are trusted without verifying integrity, provenance, and update history.
See also: FORGE-10 Vendor Embargo Gaps & Patch Velocity Failures for container runtime and driver patching risks.
Impact and Failure Modes
Unverified third-party staging or integration
A system integrator, reseller, or logistics partner preloads firmware, management credentials, BIOS settings, or node images before delivery. If the provider accepts the systems without independent verification, it may inherit whatever configuration, vulnerability, or implant was introduced upstream.
Unsafe model checkpoint loading
Some model-loading paths, especially pickle-based formats, can execute code during deserialization. A malicious or tampered model artifact can therefore turn the act of loading a model into code execution on training, evaluation, or inference nodes.
Poisoned AI infrastructure dependency or container image
A compromised CUDA or NCCL dependency, GPU operator, device plugin, model-serving image, or privileged base image can introduce credential theft, backdoors, or cluster-level access into GPU nodes and Kubernetes environments that automatically pull or update trusted infrastructure components.
Tampered internal mirror or registry
An attacker who compromises an internal package index, container registry, model registry, or artifact repository can replace trusted components with malicious versions. Because production systems often trust internal mirrors more than public sources, the compromise can spread quickly across workloads, nodes, or clusters.
Compromised node image or provisioning template
A malicious or accidental change to a golden image, bootstrap script, Kubernetes manifest, Terraform module, or build pipeline can propagate to every GPU worker provisioned from that artifact.
Lack of cryptographic agility in long-lived systems
An AI or HPC platform may remain in service for many years while depending on vendors to update firmware signing, key exchange, digital signatures, and authentication mechanisms. If the hardware or firmware cannot support newer algorithms, including future post-quantum requirements, the operator may be forced to continue using outdated cryptography or replace the platform.
Prevention and Mitigation Strategies
For providers and datacenter operators
- Verify before trust at every handoff: Independently verify firmware versions, BIOS and management settings, installed images, and component inventory when receiving systems from integrators, resellers, logistics partners, manufacturers, or refurbishment paths. Re-establish trust after any third-party staging, repair, or return rather than accepting the delivered state.
- Control and restrict artifact sources: Mirror approved packages, container images, GPU drivers, model artifacts, Helm charts, and infrastructure-as-code templates into controlled internal registries. Do not allow production systems to pull directly from unreviewed public registries or repositories. Pin images by immutable digest, lock dependency versions, and require signatures, checksums, or build provenance for artifacts in privileged infrastructure paths.
- Treat privileged AI infrastructure components as high-risk: Apply extra scrutiny to components that run with cluster or node privilege, such as GPU operators, device plugins, CUDA/NCCL dependencies, drivers, and privileged base or model-serving images. A compromise in these components can reach GPU nodes or the Kubernetes control plane directly, so they should come only from verified sources and be pinned by digest where possible.
- Restrict build and provisioning systems: Limit cloud credentials, registry write access, Kubernetes privileges, and outbound connectivity for image builders, CI/CD pipelines, and fleet automation. A compromised build step should not be able to exfiltrate secrets, overwrite trusted artifacts, or fetch follow-on payloads.
- Maintain inventory and prepare for recall: Track SBOMs, hardware bills of materials, image digests, firmware versions, dependency versions, and deployment locations so a compromised artifact or component can be identified, blocked, and removed quickly across the fleet.
For customers and teams running their own pipelines
A meaningful share of supply-chain risk can be reduced in your own pipeline, while the physical hardware and provider-managed base-image lane require asking the provider for evidence.
- Pin and verify what you pull: Pin container images by digest, lock dependency versions, and verify signatures where available so upstream changes cannot silently enter your environment.
- Prefer safer model-loading paths: Load models from safer formats such as safetensors rather than pickle-based checkpoints where possible, and scan model artifacts before loading them into privileged training, evaluation, or inference environments.
- Mirror and scan instead of pulling directly from public sources: Pull packages, images, and models through a controlled internal mirror that scans artifacts before promotion to production.
- Limit what build and training jobs can reach: Scope credentials and outbound network access for CI/CD and training jobs so a compromised dependency cannot exfiltrate secrets or fetch a follow-on payload.
Attack Scenarios
An attacker publishes a malicious package with a name similar to a popular ML dependency, or compromises a legitimate upstream package. A data scientist installs or updates it during environment setup. The package executes during installation or import and steals credentials, model artifacts, or training data, or establishes a foothold for later access.
A team downloads a popular pretrained model from a public hub and loads it directly into a privileged training or inference environment. Incidents involving malicious pickle-based checkpoints on Hugging Face showed that model loading can become code execution when unsafe serialization formats are used. Even when the format itself is non-executable, a poisoned model may still pass basic validation and later produce attacker-controlled behavior on selected inputs.
A provider receives GPU servers from a third-party integrator that assembled, racked, and imaged the systems before delivery. The servers arrive with an operating system already installed, along with firmware updates and management configuration applied during staging. The provider verifies that the systems boot and pass health checks, but does not independently verify what was included in the pre-installed OS image. The image contains a persistent backdoor that survives the provider's standard onboarding process.
An attacker obtains a server vendor's firmware signing key. They produce a malicious BMC firmware image that passes the provider's signature verification because it is signed with the legitimate key. The attacker uploads it to the vendor's firmware support portal. The provider downloads and deploys it fleet-wide as a routine update. Once installed, it provides persistent remote access to the management plane on every node that accepted the update.
References
NIST SP 800-218, Secure Software Development Framework. NIST. Secure SDLC practices for software producers.
OpenSSF Scorecard. Open Source Security Foundation. Automated security health checks for open-source projects.
PyTorch torch.load() Documentation. PyTorch. Documents unsafe deserialization risks in pickle-based model loading.
https://docs.pytorch.org/docs/stable/generated/torch.load.html
Pickle Scanning and Pickle Security. Hugging Face. Model artifact malware scanning on the Hub.
Hugging Face Hub Security. Hugging Face. Platform-level model and artifact security controls.
Safetensors, Secure Model Weight Format. PyTorch / Hugging Face. Safe alternative to pickle-based model serialization.
NVIDIA NGC, Signed Container Images. NVIDIA. Verified GPU container image provenance.
https://docs.nvidia.com/ngc/gpu-cloud/ngc-catalog-user-guide/index.html
Chainguard Images Documentation. Chainguard. Minimal, hardened container base images.
Exporting a Software Bill of Materials. GitHub. SBOM generation for dependency tracking.
Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor. JFrog, 2024. Backdoored pickle checkpoints discovered on Hugging Face Hub.