Anthropic Open-Sources FlashProve, a Cryptographic Proof-of-Inference Prototype for MoE Models
Anthropic's Frontier Red Team, working with EPFL, MIT and StarkWare, released FlashProve, an open-source prototype that generates a cryptographic proof confirming a specific language model produced a specific reply to a prompt. The project addresses a trust gap in hosted inference APIs, where callers cannot verify whether a cheaper model was substituted or the token stream was tampered with, and where outside evaluators cannot confirm the model they tested matches the one actually served in production. The repository, published under the Apache 2.0 license, includes a prover, a fast C++ verifier, a Lean-based reference verifier with formal proofs of completeness, soundness and zero-knowledge properties, a protocol specification, and an interactive visualizer. This is a research release, not a production security control, and Anthropic explicitly notes gaps between the paper, the formal Lean protocol, and the running code that still require manual review.
Detalhes técnicos
FlashProve was validated against Llama 4 Maverick, a 400-billion-parameter Mixture-of-Experts model (model weights are not included and must be pulled separately from Hugging Face under Meta's license). The prover operates on only the ~17 billion parameters active per token, streaming weights through in layer-sized segments, and runs on a single 80GB GPU with roughly 290GB of host memory. On one H100 with 24 CPU cores it achieves about 110 tokens/second; a proof for a 1,024-token request is ~18.78MB and verifies in about 17 seconds on 24 threads (21 seconds single-threaded). A one-time offline commitment step takes about 39 minutes per model and covers roughly 16 million subsequent requests, after which clients download a 6.5MB commitment plus one proof per request. Compared to vLLM running the same model in FP8 across eight H100s, overhead measured about 72x per GPU on average and up to 94x under heaviest load. The prover itself lacks formal verification tooling and must be audited manually, unlike the Lean-verified components.
Linha do tempo
-
Anthropic Open-Sources FlashProve, a Cryptographic Proof-of-Inference Prototype for MoE Models
ler a análise desta fonte ocultar
Anthropic's Frontier Red Team, working with EPFL, MIT and StarkWare, released FlashProve, an open-source prototype that generates a cryptographic proof confirming a specific language model produced a specific reply to a prompt. The project addresses a trust gap in hosted inference APIs, where callers cannot verify whether a cheaper model was substituted or the token stream was tampered with, and where outside evaluators cannot confirm the model they tested matches the one actually served in production. The repository, published under the Apache 2.0 license, includes a prover, a fast C++ verifier, a Lean-based reference verifier with formal proofs of completeness, soundness and zero-knowledge properties, a protocol specification, and an interactive visualizer. This is a research release, not a production security control, and Anthropic explicitly notes gaps between the paper, the formal Lean protocol, and the running code that still require manual review.
Technical Details
FlashProve was validated against Llama 4 Maverick, a 400-billion-parameter Mixture-of-Experts model (model weights are not included and must be pulled separately from Hugging Face under Meta's license). The prover operates on only the ~17 billion parameters active per token, streaming weights through in layer-sized segments, and runs on a single 80GB GPU with roughly 290GB of host memory. On one H100 with 24 CPU cores it achieves about 110 tokens/second; a proof for a 1,024-token request is ~18.78MB and verifies in about 17 seconds on 24 threads (21 seconds single-threaded). A one-time offline commitment step takes about 39 minutes per model and covers roughly 16 million subsequent requests, after which clients download a 6.5MB commitment plus one proof per request. Compared to vLLM running the same model in FP8 across eight H100s, overhead measured about 72x per GPU on average and up to 94x under heaviest load. The prover itself lacks formal verification tooling and must be audited manually, unlike the Lean-verified components.