Vals benchmark finds open-weight Chinese model DeepSeek nearly matching OpenAI's gated cyber model Daybreak on simulated network attacks
AI evaluation company Vals tested frontier and open-weight models on multi-step offensive tasks against two simulated corporate network scenarios, one reachable through a public-facing website and one requiring lateral movement from a first compromised company network into a second. OpenAI's Daybreak, restricted to trusted partners, led with 81 percent of tasks completed on average over two runs, while the Chinese open-weight model DeepSeek completed 73 percent, matching Daybreak on some individual tasks. Cost strongly favours the open model: Vals measured $17 to $46 in tokens for GPT-6-Sol to run an attack versus $2 to $3 for DeepSeek, which can also be hosted on the operator's own hardware. The result undercuts the premise of gated-release programmes such as Anthropic's Project Glasswing, which trusted-access lead Logan Graham described to WIRED in April as a way to give defenders a head start before such capabilities spread broadly. No vulnerability, exploitation campaign or threat actor is reported; the article covers a capability benchmark and its implications for both attackers and defenders.
Detalhes técnicos
The evaluation scored completion of specific objectives in mock environments rather than real-world intrusions. Observed attacker behaviour, per the article, has not followed the capability curve: criminal and state-backed operators continue to conserve resources and rely on older, less capable proprietary and open models for most activity. Alex Zenla of Edera notes that abliterated open-weight models — whose safety training is disabled by modifying weights — already support advanced attack and vulnerability discovery work, while closed labs likely hold unreleased models beyond these capabilities, making the dynamic a cat-and-mouse race.
Remediação
No patch or indicator applies. Defenders are advised by Vals CEO Rayan Krishnan to consider self-hosted open-weight models for defensive automation given the order-of-magnitude cost advantage, and to assume well-resourced adversaries have had some access to closed frontier models regardless of gating.
Linha do tempo
-
Vals benchmark finds open-weight Chinese model DeepSeek nearly matching OpenAI's gated cyber model Daybreak on simulated network attacks
ler a análise desta fonte ocultar
AI evaluation company Vals tested frontier and open-weight models on multi-step offensive tasks against two simulated corporate network scenarios, one reachable through a public-facing website and one requiring lateral movement from a first compromised company network into a second. OpenAI's Daybreak, restricted to trusted partners, led with 81 percent of tasks completed on average over two runs, while the Chinese open-weight model DeepSeek completed 73 percent, matching Daybreak on some individual tasks. Cost strongly favours the open model: Vals measured $17 to $46 in tokens for GPT-6-Sol to run an attack versus $2 to $3 for DeepSeek, which can also be hosted on the operator's own hardware. The result undercuts the premise of gated-release programmes such as Anthropic's Project Glasswing, which trusted-access lead Logan Graham described to WIRED in April as a way to give defenders a head start before such capabilities spread broadly. No vulnerability, exploitation campaign or threat actor is reported; the article covers a capability benchmark and its implications for both attackers and defenders.
Technical Details
The evaluation scored completion of specific objectives in mock environments rather than real-world intrusions. Observed attacker behaviour, per the article, has not followed the capability curve: criminal and state-backed operators continue to conserve resources and rely on older, less capable proprietary and open models for most activity. Alex Zenla of Edera notes that abliterated open-weight models — whose safety training is disabled by modifying weights — already support advanced attack and vulnerability discovery work, while closed labs likely hold unreleased models beyond these capabilities, making the dynamic a cat-and-mouse race.
Remediation
No patch or indicator applies. Defenders are advised by Vals CEO Rayan Krishnan to consider self-hosted open-weight models for defensive automation given the order-of-magnitude cost advantage, and to assume well-resourced adversaries have had some access to closed frontier models regardless of gating.