Welcome to Week 21: How to Run Turing-Complete Malware in PyTorch's Backward Pass
We'll read this recent (and important) blog post from Elon Litman (Stanford) discussing a new way to run malware in PyTorch, and what threats it allows. Then, we might also chat a bit about how we might respond to this vulnerability.
Blog Post Link Here
This paper demonstrates a novel and dangerous supply-chain exploit in modern deep learning frameworks: running arbitrary, Turing-complete programs (such as Conway's Game of Life or hidden malware payloads) entirely inside PyTorch's .backward() pass using a mathematically zero-valued loss function. The vulnerability arises from a fundamental disagreement between hardware-level floating-point arithmetic (specifically IEEE 754 truncation under large offsets like $2^{30}$) and the symbolic differentiation rules used by autodiff engines. By constructing a regularization term that evaluates to a strict 0.0 on the GPU and in all logged training metrics, an attacker can trick autograd into producing a targeted gradient vector.
As a result, a routine call to total_loss.backward() and optimizer.step() forces the optimizer to silently overwrite model weights with arbitrary state updates—all without ever modifying the base model checkpoint, architecture, dataset, or optimizer. The paper highlights a critical blind spot in current AI security: verifying a model's file hashes, scanning code for malicious hooks, or monitoring training loss curves and auxiliary metrics is insufficient, because malicious execution can successfully hide behind completely flat, zero-valued regularization losses.
Join us at CASI for discussion at 8 pm, and (optional) quiet reading from 7 pm.
📖 Reading Recommendations, Questions, or Comments? Contact us here!
🔎 View past meeting notes here.
What's this?
A super warm group of folks discussing their favorite topics!
In the first half, we host an optional quiet reading space
In the second half, we have a discussion where peo