Anomaly Detection for AI Agents

AI agents operate autonomously. That is the point. But autonomy without monitoring is a liability. An agent can enter a retry loop and burn through hundreds of dollars in minutes. A prompt injection can cause an agent to switch models or call endpoints it was never intended to reach. A subtle code change can alter token consumption patterns in ways that are invisible until the invoice arrives.

Static budget limits help, but they are blunt instruments. A $100 daily budget will stop a runaway agent at $100 — but it will not tell you that the agent normally spends $5 and something went wrong at $15. By the time a hard budget limit triggers, you may have already lost 20x your expected spend.

Govyn's anomaly detection watches every agent in real time, builds a baseline of normal behavior, and flags deviations before they become expensive. It is the difference between a smoke detector and a fire extinguisher — both are useful, but one gives you a head start.

What Govyn detects

Govyn monitors five categories of anomalous behavior across all agents in your organization. Each category operates independently and can be configured with its own thresholds, time windows, and response actions.

Cost spikes. Sudden increases in per-agent or per-organization spending that exceed a percentage threshold above the rolling baseline. A cost spike might indicate a broken prompt, an unexpected input size, or an agent processing data it should not be touching.

Token usage anomalies. Dramatic changes in input token counts, output token counts, or the ratio between them. If an agent normally sends 200-token prompts and suddenly starts sending 5,000-token prompts, something has changed — either the agent logic, the input data, or both.

Request rate changes. Unexpected spikes or drops in the number of requests an agent makes per time window. A sudden burst of 500 requests per minute from an agent that normally makes 10 requests per minute is a strong signal that something is wrong. A sudden drop to zero might indicate an infrastructure failure.

Unusual model selection. Agents calling models they do not normally use. If an agent is configured to use gpt-4o-mini and suddenly starts requesting gpt-4o or claude-3-opus, that change may be intentional — or it may be a prompt injection, a misconfiguration, or a regression in the agent's decision logic.

Loop patterns. Near-identical requests sent repeatedly within a short time window. Loops are the most expensive failure mode for AI agents because they combine high frequency with high per-request cost. An agent stuck in a loop can spend thousands of dollars before a human notices.

How anomaly detection works

Govyn's anomaly detection operates in three phases: baseline establishment, deviation calculation, and threshold comparison. Each phase runs continuously as requests flow through the proxy.

Baseline establishment

For every agent, Govyn maintains a rolling window of historical behavior. The default window is 7 days, configurable per agent or per organization. Within this window, Govyn calculates statistical baselines for cost per hour, tokens per request, requests per minute, model distribution, and request similarity.

Baselines are not static averages. They account for time-of-day patterns (an agent that runs batch jobs at 2 AM will have a different baseline at 2 AM than at 2 PM), day-of-week variations, and gradual trends. The rolling window ensures that legitimate changes in agent behavior — new features, new data sources, intentional model upgrades — are absorbed into the baseline within days.

Deviation calculation

As each request is processed, Govyn compares the current metrics against the established baseline. Deviation is expressed as a percentage above or below the baseline value. For example, if an agent's baseline cost is $0.50 per hour and the current hour's cost is $1.25, the deviation is +150%.

Govyn calculates deviations on multiple time scales simultaneously — per-request, per-minute, per-hour, and per-day. This multi-scale approach catches both sudden spikes (a single expensive request) and gradual drifts (costs creeping up over several hours).

Threshold comparison

When a deviation exceeds the configured threshold for its category, Govyn triggers the configured action. Thresholds are set per anomaly type, per agent, or globally. A cost spike threshold of 200% means "alert when cost exceeds 3x the baseline." A request rate threshold of 500% means "alert when request volume exceeds 6x normal."

Thresholds include a minimum sample size parameter. Govyn will not trigger an alert unless the baseline has at least N data points, preventing false positives during the initial baseline-building period or after an agent has been idle.

Cost spike detection

Cost spike detection is the most immediately valuable anomaly category. It catches the scenario every team fears: an agent that suddenly starts spending 10x or 100x its normal cost.

Consider a concrete example. Your research-agent normally costs $5 per day. It processes customer support tickets, generates responses, and logs results. One Monday morning, a code deployment changes the prompt template, accidentally including the entire ticket history instead of just the latest message. Token counts jump from 200 per request to 8,000 per request. Cost per hour jumps from $0.20 to $8.00.

Without anomaly detection, you discover this on your next invoice — $240 for a single day instead of $5. With Govyn's cost spike detection configured at a 200% threshold, you receive an alert within the first hour. The alert tells you which agent deviated, by how much, and links directly to the activity feed where you can inspect the requests.

Cost spike detection operates on rolling hourly and daily windows. The hourly window catches acute spikes. The daily window catches sustained increases that might individually stay below the hourly threshold but accumulate to a significant overspend.

# Cost spike detection configuration
policies:
  - name: cost-spike-alert
    type: anomaly
    rule:
      category: cost_spike
      threshold_percent: 200    # Alert at 3x baseline
      window: 1h                # Evaluate over 1-hour windows
      baseline_days: 7          # Use 7-day rolling baseline
      min_samples: 10           # Require 10+ baseline data points
      action: notify            # notify | block | log
      channels:
        - dashboard
        - email
        - webhook:slack

Loop detection

Loop detection catches the single most expensive failure mode in autonomous agent systems: an agent sending the same or nearly identical request over and over. Loops happen more often than most teams expect. A retry mechanism without backoff, a decision loop that never converges, a tool call that always fails and always retries — these patterns can generate hundreds or thousands of requests in minutes.

Govyn identifies loops by comparing request similarity. When an agent sends N requests within M seconds where the request payloads are more than X% similar (measured by normalized edit distance on the message content), the proxy flags it as a loop. The default configuration is 5 similar requests within 60 seconds at 90% similarity, but all parameters are configurable.

When a loop is detected, the configured action is taken. The most common configuration is block — immediately reject further similar requests and notify the team. This stops the cost from accumulating while preserving the agent's ability to make different requests.

# Loop detection configuration
policies:
  - name: loop-detection
    type: anomaly
    rule:
      category: loop
      max_similar: 5            # Block after 5 similar requests
      window_seconds: 60        # Within a 60-second window
      similarity_threshold: 0.9 # 90% payload similarity
      action: block             # Stop the loop immediately
      channels:
        - dashboard
        - webhook:pagerduty

Loop detection is available in the open-source self-hosted proxy. The cloud dashboard adds visualization of detected loops, historical loop frequency, and one-click investigation via session replay.

Behavioral anomalies

Beyond cost and loops, Govyn detects subtler behavioral changes that may indicate problems before they become expensive.

Model switching. If an agent that exclusively uses gpt-4o-mini suddenly starts calling gpt-4o, Govyn flags the change. This can catch prompt injections that manipulate the model parameter, configuration errors that override the intended model, and agent logic regressions that change model selection criteria. You can configure a model allowlist per agent so that any request for an unlisted model triggers an immediate alert.

Token count shifts. Changes in the distribution of input and output token counts can indicate data pipeline issues. If an agent's average input token count doubles, it might be receiving larger payloads, concatenating context incorrectly, or failing to truncate conversation history. Govyn tracks token count distributions per agent and flags statistically significant shifts.

Timing anomalies. Agents that normally operate during business hours suddenly making requests at 3 AM may indicate unauthorized usage, a misconfigured cron job, or a compromised API key. Govyn can establish time-of-day baselines and flag out-of-pattern activity.

Error rate spikes. A sudden increase in error responses (4xx or 5xx from the upstream provider) can indicate issues with the agent's request format, rate limit exhaustion, or provider outages. Govyn tracks error rates per agent and alerts when they exceed the baseline by a configurable margin.

# Behavioral anomaly detection
policies:
  - name: behavior-monitoring
    type: anomaly
    rule:
      category: behavioral
      monitors:
        - metric: token_count_input
          threshold_percent: 150
          window: 1h
        - metric: token_count_output
          threshold_percent: 200
          window: 1h
        - metric: error_rate
          threshold_percent: 300
          window: 30m
        - metric: model_distribution
          alert_on_new_model: true
      min_samples: 20
      action: notify
      channels:
        - dashboard
        - email

Alert configuration

Anomaly detection rules are defined in YAML, the same format used for all Govyn policies. Each rule specifies the anomaly category, detection parameters, the action to take, and the channels to notify. Rules can be scoped to specific agents, agent groups, or applied organization-wide.

Every rule supports three actions:

• notify — send alerts through configured channels but allow the request to proceed. Use this for monitoring and early warning without disrupting agent operations.
• block — reject the request and send alerts. Use this for critical anomalies like loops or extreme cost spikes where continued operation would cause harm.
• log — record the anomaly in the action log without sending notifications. Use this for low-severity anomalies or during initial tuning to understand your baseline before enabling alerts.

Rules are evaluated in priority order. A higher-priority rule's action takes precedence. You might configure a notify rule at 150% cost deviation and a block rule at 500% cost deviation — the agent gets a warning at 2.5x baseline and is stopped at 6x baseline.

# Layered anomaly detection with escalating actions
policies:
  - name: cost-warning
    type: anomaly
    priority: 10
    rule:
      category: cost_spike
      threshold_percent: 150
      window: 1h
      action: notify
      channels:
        - dashboard

  - name: cost-critical
    type: anomaly
    priority: 20
    rule:
      category: cost_spike
      threshold_percent: 500
      window: 1h
      action: block
      channels:
        - dashboard
        - email
        - webhook:pagerduty

  - name: agent-specific-threshold
    type: anomaly
    priority: 15
    agents:
      - billing-processor
      - payment-agent
    rule:
      category: cost_spike
      threshold_percent: 50    # Tighter threshold for financial agents
      window: 30m
      action: notify
      channels:
        - dashboard
        - email

Alert delivery channels

Govyn delivers anomaly alerts through multiple channels so your team sees them wherever they work. Each alert rule can specify one or more delivery channels.

In-dashboard notifications. Every alert appears in the Govyn dashboard notification center. Notifications include the anomaly type, affected agent, deviation magnitude, timestamp, and a direct link to the relevant activity feed. Dashboard notifications are always enabled — they cannot be turned off.

Email alerts. Govyn sends email notifications to team members based on their role. Organization admins receive all alerts by default. You can configure additional recipients per rule or globally. Emails include a summary of the anomaly and a link to the dashboard for investigation.

Webhook notifications. For integration with external tools, Govyn sends HTTP POST requests to configured webhook URLs. The payload includes structured JSON with the anomaly type, agent identifier, baseline value, observed value, deviation percentage, and timestamp. Common integrations include:

• Slack — send alerts to a Slack channel via incoming webhook URL
• PagerDuty — trigger incidents via the PagerDuty Events API
• Opsgenie — create alerts via the Opsgenie API
• Custom endpoints — any HTTP endpoint that accepts JSON POST requests

All webhook URLs are validated with SSRF protection. Govyn rejects webhook URLs that resolve to private IP ranges (RFC 1918), localhost, link-local addresses, and cloud metadata endpoints. This prevents attackers from using webhook configuration to probe internal networks.

# Webhook alert payload example
{
  "type": "anomaly_detected",
  "anomaly": {
    "category": "cost_spike",
    "agent": "research-agent",
    "baseline_value": 0.42,
    "observed_value": 3.15,
    "deviation_percent": 650,
    "window": "1h",
    "threshold_percent": 200,
    "action_taken": "block"
  },
  "timestamp": "2025-03-15T14:23:00Z",
  "dashboard_url": "https://app.govynai.com/dashboard/alerts/abc123"
}

Responding to anomalies

Detecting an anomaly is the first step. Investigating and resolving it requires a workflow that moves from alert to root cause to fix. Govyn provides the tools for each stage.

Step 1: Receive the alert. An anomaly notification arrives in the dashboard, your email, or your Slack channel. The alert identifies the agent, the anomaly type, and the severity (deviation magnitude).

Step 2: Check the activity feed. Navigate to the activity feed and filter by the affected agent. Look at the recent requests — their token counts, costs, models used, and policy evaluation results. The activity feed shows the raw data behind the anomaly.

Step 3: Review session replay. If the anomaly involves a specific session, open session replay to see the complete conversation flow. Session replay shows every request and response in sequence, making it easy to identify where the agent went off track — a bad tool call, a hallucinated API endpoint, or a prompt injection.

Step 4: Identify the root cause. Common root causes include prompt template changes that increased token counts, new data sources that changed input sizes, agent logic regressions that altered model selection, broken retry mechanisms that caused loops, and rate limit changes from upstream providers that triggered error cascades.

Step 5: Adjust policies. Based on the root cause, update your policies. You might tighten a budget limit, add a model allowlist, adjust loop detection parameters, or create a new anomaly rule with a lower threshold for the affected agent. Deploy the updated YAML and the changes take effect immediately.

Anomaly detection vs static rules

Static rules and anomaly detection serve different purposes. Understanding when to use each — and how they complement each other — is key to effective AI agent governance.

The recommended approach is to use both. Set static budget limits as hard ceilings — the absolute maximum an agent is allowed to spend in a given period. Layer anomaly detection on top for early warning — catch problems at 2x baseline instead of waiting for the 10x budget limit. Together, they provide both sensitivity and safety.

Dashboard anomaly view

The Govyn dashboard provides a dedicated view for anomaly monitoring and investigation. The anomaly view is available on Team plans and above.

Notification history. A chronological list of all anomaly alerts with filters for anomaly type, agent, severity, and date range. Each notification shows the anomaly category, the affected agent, the deviation magnitude, the action taken, and the timestamp. Click any notification to jump to the relevant activity feed entry.

Alert timeline. A visual timeline showing anomaly events plotted against agent activity. The timeline makes it easy to correlate anomalies with deployment events, configuration changes, or external factors. Multiple agents can be overlaid on the same timeline for cross-agent pattern analysis.

Baseline charts. Per-agent charts showing the current baseline values for cost, token counts, and request rates alongside the actual observed values. Baseline charts help you understand what "normal" looks like for each agent and verify that your thresholds are set appropriately. If the baseline and actual values are consistently close, your thresholds may be too tight. If there is a wide gap, they may be too loose.

Alert rule management. Create, edit, and test anomaly rules directly from the dashboard. The rule editor shows a preview of how the rule would have performed against historical data — how many alerts it would have generated in the past 7 days, which agents would have been affected, and whether any critical events would have been missed.

Frequently asked questions

Enable anomaly detection on Team plan Learn more about Govyn