Elastic Research Shows Embedding Models Can Surface Risky AI Agent Sessions Missed by Rule-Based Detection
Elastic Security Labs researcher Marcus Jeffes published a methodology study testing whether embedding models (vector search) can detect risky or malicious behavior inside AI agent sessions (e.g., Claude Code, Cursor), an area where traditional keyword/regex detection and even LLM-based review (like Uber's Agentic AI Detection and Response (ADR)) fall short. Using 399 of his own sessions (7,454 turns) stored as vectors in Elastic, the author found that roughly one in seven sessions required human review, and that the agent's reasoning channel carried more detection signal than the user prompt channel. This is a research/methodology article, not a vulnerability disclosure or active incident, but it is relevant to defenders building AI agent monitoring and SOC detection capabilities.
Detalhes técnicos
Sessions were chunked per turn into four channels (user, reasoning, agent response, tool calls), each embedded separately with jina-embeddings-v5-text-small, then merged via reciprocal rank fusion (RRF). Labeling of the 7,454 turns was done using Opus 4.6 as an LLM judge, with planted positive/negative control sessions to validate labeling accuracy (18 of 23 planted bad sessions escalated automatically, 7 of 8 hard negatives cleared). A query set of 264 questions was built against MITRE ATLAS (v2026.07) covering 86 techniques across 16 tactics to benchmark detection coverage. Vector search outperformed keyword (BM25) search overall, though BM25 remained competitive for literal command matching. Exemplar retrieval (using a known-bad session as the query) was the strongest method for finding similar malicious sessions but was narrower in scope than descriptive hunting, mainly useful for detecting repeats of a known-bad workflow rather than novel but similarly risky behavior. Credential access techniques scored perfectly (rank 1) while discovery techniques showed detection gaps, attributed to immature query sets, lack of multi-user data, or inherent difficulty of the technique.
Remediação
The author recommends organizations exploring AI agent session monitoring to (1) chunk and embed sessions by channel rather than as a single vector per session to preserve fine-grained signal, (2) prioritize monitoring the agent reasoning channel, which showed the highest fidelity for incident detection, (3) combine descriptive hunting (to catch novel behaviors) with exemplar retrieval (to catch repeats of known-bad workflows), and (4) use a MITRE ATLAS-aligned query baseline to measure and improve detection coverage over time rather than relying solely on generative-model-based review, which is costly and non-deterministic.
Linha do tempo
-
Elastic Research Shows Embedding Models Can Surface Risky AI Agent Sessions Missed by Rule-Based Detection
ler a análise desta fonte ocultar
Elastic Security Labs researcher Marcus Jeffes published a methodology study testing whether embedding models (vector search) can detect risky or malicious behavior inside AI agent sessions (e.g., Claude Code, Cursor), an area where traditional keyword/regex detection and even LLM-based review (like Uber's Agentic AI Detection and Response (ADR)) fall short. Using 399 of his own sessions (7,454 turns) stored as vectors in Elastic, the author found that roughly one in seven sessions required human review, and that the agent's reasoning channel carried more detection signal than the user prompt channel. This is a research/methodology article, not a vulnerability disclosure or active incident, but it is relevant to defenders building AI agent monitoring and SOC detection capabilities.
Technical Details
Sessions were chunked per turn into four channels (user, reasoning, agent response, tool calls), each embedded separately with jina-embeddings-v5-text-small, then merged via reciprocal rank fusion (RRF). Labeling of the 7,454 turns was done using Opus 4.6 as an LLM judge, with planted positive/negative control sessions to validate labeling accuracy (18 of 23 planted bad sessions escalated automatically, 7 of 8 hard negatives cleared). A query set of 264 questions was built against MITRE ATLAS (v2026.07) covering 86 techniques across 16 tactics to benchmark detection coverage. Vector search outperformed keyword (BM25) search overall, though BM25 remained competitive for literal command matching. Exemplar retrieval (using a known-bad session as the query) was the strongest method for finding similar malicious sessions but was narrower in scope than descriptive hunting, mainly useful for detecting repeats of a known-bad workflow rather than novel but similarly risky behavior. Credential access techniques scored perfectly (rank 1) while discovery techniques showed detection gaps, attributed to immature query sets, lack of multi-user data, or inherent difficulty of the technique.
Remediation
The author recommends organizations exploring AI agent session monitoring to (1) chunk and embed sessions by channel rather than as a single vector per session to preserve fine-grained signal, (2) prioritize monitoring the agent reasoning channel, which showed the highest fidelity for incident detection, (3) combine descriptive hunting (to catch novel behaviors) with exemplar retrieval (to catch repeats of known-bad workflows), and (4) use a MITRE ATLAS-aligned query baseline to measure and improve detection coverage over time rather than relying solely on generative-model-based review, which is costly and non-deterministic.