A use case represents an AI-powered application or workflow being monitored. Use cases are designed to support LLM and agentic applications where multiple interaction types occur within a single workflow, and all activity is surfaced in a unified monitoring view.

Observe is where you review the AI deployments you have set up as use cases. It answers two different questions through two surfaces. The Use Case Status dashboard is the wide view: it shows every use case at once so you can scan the health of all your deployments, spot the ones that need attention, and decide where to look closer. The Use Case Analysis page is the deep view: it breaks down a single use case so you can understand what is driving its activity. The common path is to start on Status, identify a use case worth investigating, and click into Analysis.

Use Case Status

The Use Case Status dashboard displays a searchable grid of use case cards to provide an at-a-glance read on all of your deployments from one place. It is organized around two jobs, split into two tabs:

Performance: Surfaces session data for trend analysis. This is the view for understanding whether a deployment is behaving normally over time.
Triage: Surfaces alert data for active review and response. This is the view for finding what has been flagged and needs to be dealt with now.

A global timeframe selector scopes both tabs to the last 24 hours, 7 days, 30 days, or all time.

Use Case Cards

Each card summarizes one use case so you can compare deployments without opening each one. A card displays:

Use Case Name: The name assigned when the use case was created.
Deployment Type Tag: Chatbot or Agent.
Session or Alert Count: Session count on the Performance tab, alert count on the Triage tab.
Delta Indicators: A color-coded indicator of how the metric has changed since the previous equivalent period. (Ex. With the timeframe set to 30 days, it reflects the change against the prior 30 days.)
Success Rate: The percentage of sessions that finished successfully. Each session carries a status on its final event, with possible values of successful, blocked, internal error, or external error. Success rate is the number of sessions with a successful status divided by the total number of sessions. As a best practice, agentic API calls should set a session status on the last event so that success rate reflects real outcomes.
Time Series Chart: Activity over the selected timeframe. The chart can be toggled between bar and line display and expanded for a closer look.

Cards also carry a persistent set of footer metrics that differ by deployment type, because the questions you ask of a chatbot differ from those you ask of an agent. Chatbot cards show Sessions and Turns/session. Agent cards show Sessions, Autonomy, and Agents/session.

Turns/session: The average number of turns per session over the selected timeframe.
Autonomy: A measure of how independently agents operate, calculated as the average number of actions agents take before requesting human input, divided by the total number of actions.
Agents/session: The average number of agents per session.

Clicking a card opens the Use Case Analysis page for that use case.

Use Case Analysis

The Use Case Analysis page breaks down the activity of a single use case so you can understand what is driving its numbers rather than just seeing the totals. The page includes a use case selector, a timeframe filter. A View Sessions button opens the Sessions page filtered to this use case, for moving from aggregate trends into the individual interactions behind them.

Summary Metrics

Four metrics at the top of the page describe the use case at a glance:

Sessions: The total number of sessions in the selected timeframe. This module makes usage patterns and spikes easy to spot.
Success Rate: The percentage of successful sessions.
Users: The number of unique users.
Alerts Sent: The number of alerts generated.

Modules

The modules below break activity down along different dimensions. Each can be displayed in more than one visualization format.

Session Volume Over Time: Session activity across the selected timeframe. This module makes usage patterns and spikes easy to spot.
Success Rate: The breakdown of sessions by outcome, across successful, blocked, internal error, and model error states. This module shows why sessions are or are not succeeding.
Intent: The distribution of detected intents across sessions. This module reveals what users are actually trying to do.
Signal Distribution: The distribution of detected signals across categories, which surfaces what kinds of risk are present. Users can filter by one or more categories.
Top Named Entities: The most frequently detected named entities.

Signal Distribution groups signals into five categories:

DLP (Data Loss Prevention): Signals that identify sensitive, regulated, or confidential data within user communications and model responses. Includes named entities, personally identifiable information (PII), protected health information (PHI), and payment card industry (PCI) information.
Safety: Signals that identify harmful, hostile, or unlawful material.
- Toxicity: Hostile, aggressive, disrespectful, or harmful language, including hate speech, harassment, threats, and personal attacks.
- Illegality: References to illegal activities or instructions for unlawful behavior across categories such as cybercrime, fraud, drugs, violence, and terrorism.
Code: Signals related to the presence or generation of programming code. Includes code present and code requested.
Security: Signals that identify attempts to manipulate or exploit the model through adversarial inputs. Includes prompt injection, jailbreaking, role impersonation, and similar techniques.
Content: Currently contains Instructions.

Agentic Analytics

For Agent-type use cases, an Agentic Analytics subsection appears below the standard modules. Agentic workflows raise questions a chatbot does not:

How independently is the agent acting?
How are agents coordinating with one another?
What tools are the agents calling?

These modules exist to answer those questions.

Agent Activity

These modules describe what the agent is doing and how much:

Event Type Distribution: The breakdown of activity by event type, such as Agent to LLM, Agent to Tool, User to Agent, Agent to Memory, Agent to Agent, and Agent Initialized. This shows the mix of interactions taking place within the workflow.
Event Density: The number of events per session, for gauging how long sessions are. Sessions are bucketed by event count into columns from 1 to 20 or more, where the 20 or more column contains every session with 20 or more events.
Top Tool Invocations: The tools the agent calls most frequently, for understanding what actions it is actually taking.

Autonomy and Coordination

These modules describe how the agent operates on its own and with others:

Session Autonomy Score: How autonomously agents operate, shown as an overall percentage. It is calculated as the average number of actions agents take before requesting human input, divided by the total number of actions. The module displays the overall autonomy percentage, the average number of actions taken before human input, and a distribution of sessions across autonomy score ranges.
Multi-Agent Coordination: How many agents take part in sessions. The module displays the average and peak number of agents per session, the number of handoffs between agents, and a distribution of session by agent count.

Risk Distribution

These modules describe where risk and sensitive data concentrate across agent activity:

Signal Density: Events groups into four tiers based on the volume of signals they carry, showing how concentrated risk is across the workflow's events. The tiers are:
- Clean: 0 signals per 10 events.
- Low: 1 signal per 10 events.
- Elevated: 3 signals per 10 events.
- High: 5 or more signals per 10 events.
Top Named Entities: The most frequently detected named entities from all use case traffic.

Did this answer your question?