SentinelLABS: Building Security AI Agents Backwards from Evaluation with End-to-End and Competency Tests
This SentinelOne / SentinelLABS blog post (Part 2 of the "LLMs in the SOC" series) argues that security agents should be designed so their performance is measurable from the start, rather than treating evaluation as an afterthought following a proof-of-concept. It is a methodology piece, not a threat report: no incident, vulnerability, or threat actor is described. The authors propose pairing end-to-end evaluations (alert burndown, disposition accuracy, time-to-verdict, cost per alert) with per-competency evaluations (e.g., SIEM query construction, investigation efficiency, policy adherence, evidence-sufficiency judgment). They illustrate the risk with an investigation agent that reached a correct verdict while querying the wrong device identifier, and describe how a competency test case would catch and then guard against regression of that failure. The post also covers prioritizing evaluation effort, using domain experts for ground truth, and validating agents in parallel with internal SOC and Wayfinder Managed Services analysts before release, using the Purple AI Agentic Investigation agent as the example.
Detalhes técnicos
- Every evaluation decomposes into a task, a scenario, and rubrics.
- End-to-end metrics can hide the cause of regressions; competency evaluations isolate one skill with its own inputs, success definition, and varied cases.
- Verdict-only scoring is risky: a 95% match with experts may reflect pattern-matching on alert features rather than real evidence retrieval.
- Prioritize competencies by impact of failure × probability of model failure × decidability of ground truth; prefer binary pass/fail grading.
- SIEM query construction cases specify investigative intent, target data source, and expected results, graded automatically by running queries against representative data.
- Observed production failures become permanent test cases; a few dozen well-chosen cases per capability give a strong initial signal.
Remediação
- Define the agent's work and success criteria before choosing models, prompts, or tools.
- Build agents from individually testable competencies and run both end-to-end and competency evaluations continuously.
- Re-run the suite after every material change to a model, prompt, tool, or workflow.
- Co-author cases with domain experts, and have experts establish ground truth independently of the agent's verdict.
- Validate agents alongside human analysts in production-like settings before external release.
Linha do tempo
-
SentinelLABS: Building Security AI Agents Backwards from Evaluation with End-to-End and Competency Tests
ler a análise desta fonte ocultar
This SentinelOne / SentinelLABS blog post (Part 2 of the "LLMs in the SOC" series) argues that security agents should be designed so their performance is measurable from the start, rather than treating evaluation as an afterthought following a proof-of-concept. It is a methodology piece, not a threat report: no incident, vulnerability, or threat actor is described. The authors propose pairing end-to-end evaluations (alert burndown, disposition accuracy, time-to-verdict, cost per alert) with per-competency evaluations (e.g., SIEM query construction, investigation efficiency, policy adherence, evidence-sufficiency judgment). They illustrate the risk with an investigation agent that reached a correct verdict while querying the wrong device identifier, and describe how a competency test case would catch and then guard against regression of that failure. The post also covers prioritizing evaluation effort, using domain experts for ground truth, and validating agents in parallel with internal SOC and Wayfinder Managed Services analysts before release, using the Purple AI Agentic Investigation agent as the example.
Technical Details
- Every evaluation decomposes into a task, a scenario, and rubrics.
- End-to-end metrics can hide the cause of regressions; competency evaluations isolate one skill with its own inputs, success definition, and varied cases.
- Verdict-only scoring is risky: a 95% match with experts may reflect pattern-matching on alert features rather than real evidence retrieval.
- Prioritize competencies by impact of failure × probability of model failure × decidability of ground truth; prefer binary pass/fail grading.
- SIEM query construction cases specify investigative intent, target data source, and expected results, graded automatically by running queries against representative data.
- Observed production failures become permanent test cases; a few dozen well-chosen cases per capability give a strong initial signal.
Remediation
- Define the agent's work and success criteria before choosing models, prompts, or tools.
- Build agents from individually testable competencies and run both end-to-end and competency evaluations continuously.
- Re-run the suite after every material change to a model, prompt, tool, or workflow.
- Co-author cases with domain experts, and have experts establish ground truth independently of the agent's verdict.
- Validate agents alongside human analysts in production-like settings before external release.