Back to NewsroomBack to Newsroom
Netra introduces Insights to track AI agent behavior in production
Share this

Share this news via

Netra introduces Insights to track AI agent behavior in production

Netra, KeyValue’s AI agent reliability platform, has introduced Insights, an automated monitoring capability that helps teams understand how their AI agents are used and how their behavior changes in production.

By analyzing production traces, Insights identifies what users ask AI agents to do, uncovers emerging use cases and detects changes in agent behavior. It works with data Netra already captures, without requiring teams to set up monitoring rules or thresholds.

Understanding intent from production data

Teams don’t need to define intents in advance. Once around 500 traces are collected, Insights groups similar requests into clusters and assigns each a descriptive name, such as ‘refund requests’ or ‘flight bookings’.

As new traces come in, they are mapped to the closest existing intent. Requests that don’t match any existing intent are grouped into new clusters daily, helping teams discover emerging use cases automatically. Insights also tracks how each intent evolves, classifying it as new, growing, stable or declining based on its volume over the past 30 days.

Detecting changes in agent behavior

For each intent, Insights compares the past day, week and month against a 30-day baseline that refreshes daily. It tracks cost, latency, step count, error rate, tool usage and output length, and flags significant shifts with a severity level. A change in output length, for example, can signal a prompt regression or a change in model behavior. 

Overall performance metrics may appear stable even when specific workflows are experiencing issues. By breaking down behavior by intent, Insights helps engineering teams identify where attention is needed, while giving product teams a clearer understanding of changing customer needs 

After each run, Insights generates a short summary of significant changes and newly discovered intents, so teams can see what has shifted at a glance. 

Making AI agents reliable, end to end

Built on KeyValue’s experience developing more than 25 AI agents, Netra helps teams maintain reliability throughout the AI agent lifecycle. Before deployment, teams can simulate conversations, run evaluations and red team their agents. In production, they can trace requests and receive alerts when issues arise.

Insights extends these capabilities by showing how agent behavior evolves over time and identifying where performance begins to drift. 

Learn more about Netra at getnetra.ai.