Qevlar AI
Organizational context shouldn't decay, it should compound. Qevlar turns every investigation into sharper context for the next one.
In the Spring of 2025, there was an incident involving a critical infrastructure provider. The attacker had pivoted through a compromised vendor credential into the operational network. The analysts identified the intrusion within 90 minutes. Figuring out the total blast radius took 11 more days. It initially seemed counter-intuitive given that the telemetry and logs were all available, but what I began noticing was that every tool in the stack was built to describe the incident rather than model its structure. The SIEM effectively correlated the alerts while an LLM-assisted platform generated summaries that read like a finished investigation. But no one could answer from this specific incident what else was reachable which is critical for ensuring containment. That question is a graph traversal. And until someone built the graph, that executive summary was fiction dressed as findings.
In September 2025, Shane sat down with Qevlar AI's CEO Ahmed Achchak for the SuperSOC podcast to talk about why false negatives, not false positives, are the SOC's most dangerous blind spot. One idea from that conversation has stayed with us. The industry keeps building detection and response systems around correlation, and then mistakes that correlation for causation. They spent an hour on the detection side of that problem. This piece is about what the same mistake looks like on the response side, and why it matters more as we hand more of incident response to AI.
The industry treats fluent descriptions as understanding. In incident response, understanding means building and maintaining a structural model of your environment and forcing every AI system to reason against it.
There's a line that keeps surfacing in the “AI for security” discourse, usually right before someone ships an LLM wrapper around a SIEM. If it can write an incident report, it must understand the incident. It doesn't. Writing about a thing and reasoning about its structure are two different acts, and conflating them is why so many “AI-powered SOC” products are fluent narrators of incidents they never actually modeled correctly.
Security work splits broadly into structural problems and semantic problems, or in the shorthand used here, computation and naming. Much of the incident lifecycle depends on maintaining state, validating evidence, traversing relationships, and choosing actions over a changing environment. That's computation, and it determines whether you contain a breach in twenty minutes or twenty days. Only later does the balance shift toward explaining, labeling, and communicating what the investigation found. That's naming. Treating the two as interchangeable is the expensive mistake.
MITRE's ATT&CK describes what the attacker does. Its defensive counterpart, D3FEND, is a knowledge graph, not a narrative document, because the relationships between a countermeasure, the artifact it protects, and the technique it counters are structural, and prose can't hold them precisely enough to be machine-usable. MITRE made that call years before LLMs were part of the conversation, for the same reason this piece is arguing now.
Neither ATT&CK nor D3FEND describes the responder's actual sequence, though. That comes from NIST SP 800-61 (rev. 3, April 2025), which now organizes incident response under three CSF 2.0 functions: Detect, Respond, and Recover. Respond breaks into categories including Incident Analysis (where scope gets determined) and Incident Mitigation (where containment and eradication happen), and NIST treats these as running concurrently rather than as strict phases. For a responder working a live incident, the sequence still matters: IOC correlation, alert triage, scope determination, containment, eradication, recovery, and remediation.
Every stage is a question about structure: which nodes connect to which, in what order, and with what evidence. The stages are not equally hard and not the same kind of problem. IOC correlation and alert triage are graph queries and statistical inference, important computationally routine. The distinction between graph-native and language-model reasoning gets consequential at scope determination, containment, and eradication, where a wrong structural answer has a price tag.
Scope determination asks what was used and where else it could be. CONTINUUM and CyberGFM detect lateral-movement relationships, but only across what’s already visible. What was actually used sets confirmed scope, and where the same credentials and paths remain usable sets the containment boundary. Containment requires a blast-radius model describing which nodes were reached, which remain reachable through live trust edges, and which paths to interrupt while evidence is checked. Eradication requires understanding how the intrusion occurred, including the technical and procedural conditions that made it possible, so those specific gaps in security can be addressed. All three are graph traversals and constrained-optimization problems. None of them is a writing assignment.
Speed makes the structural distinction urgent. A containment decision that arrives after the attacker’s next lateral move is a post-mortem finding. A SOC analyst racing to isolate a compromised identity before it reaches the domain controller doesn’t need a fluent summary. The analyst needs to know what’s reachable right now.
“A fluent incident summary produced after the blast-radius has already expanded is expensive decoration. The quality of the answer stopped mattering when the window closed.”
The graph is the explicit representation of that structure. It shows what's connected to what, with its own gaps and stale spots. It's a source of record, not an oracle. A GNN is a learned model operating over that structure by classifying a node, predicting a link, or scoring an anomaly. It's a prediction, not a fact. RL is a learned decision layer that selects an action from the represented state according to an objective or reward function. Graph, GNN, and RL sit on one side of the argument in this piece because none of them is a language model narrating a description instead of computing over structure.
The pipeline itself isn't linear. It's a loop: observe, decide, act, observe again against whatever just changed. RL is one way to run that decision layer when the state, actions, and objective are well defined; planning, rule engines, or constrained solvers can fill the same role. A language model with no tool access or authoritative external state can narrate successive steps, but it can't independently verify the changing environment or reliably close the action-observation loop.
Running alongside all of this is impact assessment. The industry habit for twenty years has been “crown jewel” analysis, labeling individual assets as important. That is necessary but incomplete. Business value may reside in an asset, dataset, identity, or process, while operational exposure emerges from the paths, permissions, and dependencies that make that value reachable. A low-value host can become critical when it provides a path into a payment system. The defender therefore has to track both node value and the paths carrying risk, not rank assets in isolation.
A graph that only describes the network at rest doesn't need much intelligence layered on top of it. The moment sessions authenticate, credentials get reused, and trust relationships change, keeping an accurate, current picture of what's connected to what becomes a dynamic-state problem. Once a defender has to choose among competing actions against that changing state, it becomes an optimization problem too, on both sides of the pipeline. Detection means continuously reweighing which anomalies matter given current exposure. Response means continuously resequencing which interruption shrinks the blast radius fastest.
An ungrounded language model is a poor fit for that, not because it can't be told the current state, but because it has no reliable way to compute over that state once it's told. Live topology should live as external state, queried at decision time, not memorized in the model's weights; an LLM doesn't need retraining to keep up with a graph that updates by the minute. A deterministic graph algorithm doesn't invent a path outside the graph it was given. Its result is reproducible and inspectable, although it can still be wrong if the graph, schema, or query is stale or incomplete. A GNN layered on top of that graph adds its own source of error, a prediction rather than a computed fact. RL adds a decision on top of both, a learned policy that still requires constraints and validation.
“If we involve these other dimensions, not just the two-dimensional time and place of the activity, but also the context of the user by their rights, by their privileges, by baseline activity indicated by their historical use of a resource, then if a phishing email actually is executed, that's where AI through reinforcement learning with some human feedback can improve the baseline for detections.”
None of this is a naming problem, and it isn't solved by a model that's exceptional at “what does this word mean in context.” It's solved by an explicit representation of how the environment is connected, not a description of it in prose. Even some recent graph foundation models, CyberGFM among them, borrow the transformer training paradigm and apply it to graph random walks, evidence that structure matters more than which model family processes it. A text prompt describing a network is a lossy encoding of something with an actual topology, and a model reasoning over the description is reasoning over the encoding, not the topology.
In one 2025 study, moderately disrupting a graph's real connectivity produced little or no performance change for the LLMs tested on several datasets, evidence that the models weren't reliably using the real structure to begin with. Ask an ungrounded LLM which hosts are reachable from a compromised host in three hops, and it can produce a plausible-sounding, confidently wrong answer, because it's pattern-matching on text describing a graph rather than traversing one. A confidently wrong containment boundary is the difference between isolating three hosts and isolating thirty, or missing the one that mattered.
Two incidents whose event logs read 95 percent identical can require completely different containment when one topology edge differs, for example a credential reused elsewhere, or a low-value host that quietly acquired a path into the payment system. I watched human analysts make this mistake for years before LLMs existed, treating resemblance as equivalence because the logs looked close enough. The model accelerates the same error. It has no mechanism to check whether the edge it assumes actually exists in the current graph.
The objection we hear most from LLM-forward colleagues: LLMs are deterministic, so how can they be the wrong tool for a “deterministic” problem like containment? Reproducibility of the model isn't correctness of the output. An LLM at temperature zero may repeatedly return the same wrong containment boundary, a stable bug rather than a virtue. Basic reachability (is this node reachable from that one?) is cheap; ordinary graph traversal solves it in time roughly proportional to graph size. What's actually hard is enumerating every possible simple attack path, which can scale exponentially with network size. So are the decisions built on top of that, choosing the minimum set of controls that closes an attack surface or the least disruptive containment action among several that would work. Both are NP-hard in their general form. That complexity comes from the shape of the graph, not the size of the incident report.
Two recent incidents make the point concretely:
The same gap shows up in direct benchmarks, not just postmortems. KG-LLM-Bench reports substantial degradation on relation-aggregation tasks as the number of hops and compositional complexity increase, varying by model and by how the graph is presented. Shortest-path and cycle-detection tasks show up repeatedly as among the hardest categories in graph-reasoning benchmarks, and researchers examining why have found LLMs leaning on spurious correlations, like how often a node is mentioned, rather than tracing structure.
“It's the correlation more than the causation that creates so many false positives today.”
Recognizing statistical resemblance is not the same as validating the relationships, sequence, and evidence present in a specific incident. The correlation-over-causation problem that plagues alert triage applies just as directly to containment.
As the structural investigation stabilizes, particularly once containment and recovery decisions are made, the balance of the work shifts toward explanation, reporting, and attribution. Analysis turns the structural finding into a narrative a human or a regulator can follow. Reporting translates graph state into the language of executives and compliance frameworks. Attribution has a real naming component, matching behavior to a known threat actor's name, but credible attribution remains grounded in technical and intelligence evidence, and should be expressed with calibrated confidence.
This is where LLMs become the more useful tool. Semantic retrieval and synthesis across a huge, fuzzy space of prior text, CTI reports, ATT&CK writeups, and postmortems, are tasks graph algorithms alone were never built for.
The risk here isn't structural failure; it's overconfident naming. Attribution built on semantic association alone, with no grounding in the actual event graph, produces a report that names an actor fluently with no falsifiable basis. DEFENGRAPH, a 2026 preprint, is a system built to avoid that. A knowledge graph with path retrieval constrains what the LLM is allowed to reason over. The graph doesn't just inform the output; it constrains what the model is permitted to assert.
There's a real two-tier split here (graph, GNN, and RL on one side, language model on the other), but it isn't “graph for hard problems, LLM for easy ones.”
Both tiers participate throughout an incident. The distinction is which tier has authority to assert structural facts or select consequential actions.
The failure mode to avoid on both sides is asymmetric confidence, a structural system silently blind to meaning and an LLM silently blind to whether the connections it's describing actually exist. Neither failure announces itself.
The diagnostic question isn't whether a tool “uses AI.” It's whether the topology lives somewhere real, and whether the LLM has to earn permission before it says something about it. If containment decisions come from prompting an LLM over a text description of your network, that's a graph-shaped gap dressed up as an AI capability.
The research base for this claim is accumulating. KG-LLM-Bench and NeurIPS 2023 graph-reasoning benchmarks show LLMs degrading on the tasks incident response depends on: multi-hop traversal, shortest path identification, and cycle detection. The ICML 2025 attention analysis found that LLMs processing graph-structured input attend to surface features of how the graph was described in text, not to the structural relationships themselves. These are the core operations of scope determination and containment.
Qevlar AI built its investigation platform to solve exactly this problem, contrasting it with LLM-led investigation designs in which the model itself selects the investigative path and evaluates severity. In Qevlar's internal research measuring the investigation-path variability produced by one tested LLM-agent architecture, a study of 18,000 investigation attempts across 180 real alerts, the most common investigation path held for barely 75 percent of attempts even on simple alerts; on the most complex case tested, the agent produced 90 distinct paths out of 100 runs, with the dominant path accounting for just 3 percent. Identical alerts sometimes got different severity ratings depending on which path the model took. The study measured one specific LLM-agent architecture, not every possible design, but the variance it found is hard to dismiss.
Qevlar's response is to move path selection out of the LLM. A graph orchestrator decides which steps happen and in what order, so a given alert is evaluated through the same governing logic each time. The LLM handles enrichment and reporting; by design, it doesn't render the verdict.
That architecture is meant to hold up specifically on incidents that are genuinely new. An LLM driven investigation tends to degrade exactly where the stakes are highest, since it has nothing to fall back on except resemblance. What isn't in question is that keeping the graph current is genuinely hard operational work: shadow IT, unmanaged assets, integration gaps, and telemetry that arrives hours late. A meaningful share of Qevlar's own engineering effort goes there, connecting broadly across whatever a customer has deployed and enriching every investigation against that live picture rather than a snapshot from onboarding. That unglamorous work is what decides whether any of this actually works in production.
Shane Shook, PhD, is a Venture Partner at Forgepoint Capital and has spent more than 30 years in cybercrime investigation, digital forensics, and incident response advisory for organizations worldwide. He is the principal author and executive editor of The Cybercrime Investigation Body of Knowledge (1st and 2nd editions) and writes regularly on Forgepoint's TIPS series about the architecture of AI-driven security operations. He spoke with Qevlar AI's CEO Ahmed Achchak on the SuperSOC podcast in September 2025 on shifting detection left in the kill chain.
Tim Neslony is a Venture Advisor at Forgepoint Capital and retired Air Force officer with more than 20 years in offensive cyber and national cyber policy. He began his career on keyboard as an NSA operator and developer, went on to build and command multiple U.S. Cyber Command-aligned operational units, and later served on the Joint Staff, in the Department of Defense, and at the White House as Director for Military Cyber Policy. His perspective is rooted in two decades of driving the evolution of U.S. cyber power, from early experimentation to advanced employment in conflict and crisis.
