/
← Voltar ao feed
medium

Hacktron benchmark: small LLMs run repeatedly beat frontier models on cost-to-recall for oauth2-proxy auth bypass 0-days

1 fonte(s) · 1 atualização(ões) · publicado 28/04/2026 21:00 · coletado há 1 dia

Hacktron published a vendor blog post arguing that frontier models such as Anthropic's Mythos are unnecessary for most vulnerability discovery work. To support this, it benchmarked its scanning pipeline across several LLMs against two zero-day authentication bypasses it found in oauth2-proxy v7.15.0: CVE-2026-34457 and CVE-2026-40575. Its main finding is that small models run multiple times reach high recall at a fraction of the cost of a single large-model run. Hacktron puts its optimized workflow at about $12 to find both bugs, versus over $200 and 1-2 days of researcher time with Claude Code. The post is partly marketing, and the benchmark results are self-reported.

Technical Details

  • CVE-2026-34457: a health check User-Agent match bypasses authentication in auth_request mode. Exploitation depends on a second system, such as an upstream Nginx, treating a 200 OK health check response as an authentication success signal. In standalone mode the behavior is harmless.
  • CVE-2026-40575: authentication bypass through X-Forwarded-Uri header spoofing.
  • The bugs were initially found by researcher Rahul ("iamnoooob") prompting Claude Code with a Trail of Bits audit-context skill and a "weird proxies" pass. The session cost about $200 in tokens.
  • The Hacktron pipeline does call graph construction, context gathering, code enrichment, LLM analysis, post-processing and minimal human triage. The benchmark measured recall only, not precision.
  • Gemini 3.1 Flash Lite performed exceptionally well. Claude Opus 4.6 and Gemini 3.1 Pro were more reliable than small models but still not consistent. GPT-5.4 Nano has about a 10% chance per run of surfacing Finding A, and about 90% cumulative after roughly 20 runs. GPT-5.4 Mini has about 70% per run and reaches 90% in 2 runs.
  • Context mattered more than model size. Runs succeeded when the context engine surfaced oauth2-proxy deployment-mode information.
  • The post cites Mozilla's report of 271 vulnerabilities found with Mythos and estimates its token cost at $100k or more. That figure is speculation.

Remediation

The article does not give patch versions. Operators of oauth2-proxy should consult advisories GHSA-5hvv-m4w4-gf6v and GHSA-7x63-xv5r-3p2x and upgrade to a fixed release. Deployments using auth_request mode behind a reverse proxy should be reviewed first.

Linha do tempo

  1. hacktron.ai ↗ · pub. 28/04/2026 21:00 · coletado 19/10 18:03 primeiro registro

    Hacktron benchmark: small LLMs run repeatedly beat frontier models on cost-to-recall for oauth2-proxy auth bypass 0-days

    ler a análise desta fonte

    Hacktron published a vendor blog post arguing that frontier models such as Anthropic's Mythos are unnecessary for most vulnerability discovery work. To support this, it benchmarked its scanning pipeline across several LLMs against two zero-day authentication bypasses it found in oauth2-proxy v7.15.0: CVE-2026-34457 and CVE-2026-40575. Its main finding is that small models run multiple times reach high recall at a fraction of the cost of a single large-model run. Hacktron puts its optimized workflow at about $12 to find both bugs, versus over $200 and 1-2 days of researcher time with Claude Code. The post is partly marketing, and the benchmark results are self-reported.

    Technical Details

    • CVE-2026-34457: a health check User-Agent match bypasses authentication in auth_request mode. Exploitation depends on a second system, such as an upstream Nginx, treating a 200 OK health check response as an authentication success signal. In standalone mode the behavior is harmless.
    • CVE-2026-40575: authentication bypass through X-Forwarded-Uri header spoofing.
    • The bugs were initially found by researcher Rahul ("iamnoooob") prompting Claude Code with a Trail of Bits audit-context skill and a "weird proxies" pass. The session cost about $200 in tokens.
    • The Hacktron pipeline does call graph construction, context gathering, code enrichment, LLM analysis, post-processing and minimal human triage. The benchmark measured recall only, not precision.
    • Gemini 3.1 Flash Lite performed exceptionally well. Claude Opus 4.6 and Gemini 3.1 Pro were more reliable than small models but still not consistent. GPT-5.4 Nano has about a 10% chance per run of surfacing Finding A, and about 90% cumulative after roughly 20 runs. GPT-5.4 Mini has about 70% per run and reaches 90% in 2 runs.
    • Context mattered more than model size. Runs succeeded when the context engine surfaced oauth2-proxy deployment-mode information.
    • The post cites Mozilla's report of 271 vulnerabilities found with Mythos and estimates its token cost at $100k or more. That figure is speculation.

    Remediation

    The article does not give patch versions. Operators of oauth2-proxy should consult advisories GHSA-5hvv-m4w4-gf6v and GHSA-7x63-xv5r-3p2x and upgrade to a fixed release. Deployments using auth_request mode behind a reverse proxy should be reviewed first.