See every step

AI agent observability from prompt to tool result

Follow reasoning, sub-agent delegation, tool calls, results, errors, and completion in real time—then retain an operational record for debugging and review.

  • AI agent observability
  • agent observability best practices
  • LLM observability
  • AI audit logs

The execution protocol is the trace

AI agent observability starts with a typed event stream

An agent that only returns a final answer is a black box. SyntheticBrew streams 13 typed SSE event kinds while the agent works — the same events power your product UI, the admin dashboard's tool-call log, and the persisted session log, so everyone debugs the same run. SSE keep-alive signals keep long-running streams open through proxies and load balancers.

EventWhat your UI shows
thinkingA live reasoning indicator — the agent is working, and on what.
message_deltaTokens streaming into the reply as they are generated.
messageA completed assistant message.
tool_callWhich tool the agent invoked, with its arguments.
tool_resultThe outcome and status of that tool call.
confirmationAn approval prompt before a gated, consequential action runs.
user_input_requiredThe agent is paused, waiting on the user.
agent_spawnA sub-agent was delegated part of the task.
agent_resultThat sub-agent finished and handed its output back.
structured_outputA typed payload you can render as native UI — tables, cards, forms.
agent.state_changedAgent status transitions during the run.
doneThe run is complete, with its session id.
errorA failure, with detail — never a silent hang.

The operating checklist

AI agent observability best practices

What separates an agent you can operate from an agent you can only restart:

  • Stream progress, not spinners. Render the typed events as visible steps — users trust an agent they can watch reason, delegate, and call tools.
  • Persist the full event log. When a production answer looks wrong, go back to the recorded run — every reasoning step, tool call, and result — instead of guessing from the final message.
  • Audit every consequential action with actor context. Record who (or what) acted, on which resource, in which session — so a review question has an answer, not a shrug.
  • Alert on circuit-breaker state and tool failure rates. Use the Prometheus tool-call metrics (Enterprise plans), labeled by tool and status, to catch a degrading MCP server before your users do.
  • Correlate by session id. Event log, audit log, and tool-call log all carry the session id, so one identifier walks you across every record of a run.

These practices are what regulated deployments audit against — see how they apply in AI agents for banking workflows and across the enterprise platform.

Production signals

LLM observability with Prometheus metrics and health checks

The engine emits the numbers your existing monitoring stack expects — no sidecar, no agent-of-the-agent. The Prometheus endpoint is available on Enterprise plans; health checks and the event, tool-call, and audit logs ship on every plan.

  • Six metric families cover HTTP traffic and latency, active sessions, tool calls by tool and status, and LLM requests by provider and model.
  • Health endpoint at /api/v1/health reports status, version, uptime, agents count, and a database ping — returning 503 when degraded, so your load balancer reacts automatically.
  • Resilience is observable. Every tool call runs under a 30-second timeout; circuit breakers open after 3 failures in a 60-second window and retry after 120 seconds (closed → open → half-open, admin-resettable), and an LLM retry wrapper absorbs transient provider errors.
Prometheus metricsscrape targets
# Scraped by Prometheus
syntheticbrew_http_requests_total
syntheticbrew_http_request_duration_seconds
syntheticbrew_active_sessions
syntheticbrew_tool_calls_total{tool,status}
syntheticbrew_llm_requests_total{provider,model}
syntheticbrew_llm_request_duration_seconds

# GET /api/v1/health -> status, version, uptime,
# agents count, DB ping (503 when degraded)

Operational evidence

AI audit logs with actor, action, and resource

Streaming tells you what is happening; the audit trail tells you what happened. Every consequential operation is recorded with actor type and subject, action, resource, a structured details JSON payload, session id, and task id.

  • A queryable tool-call log filters by session, tool, agent, status, user, and time — “show every failed call to the billing tool this week” is a query, not an archaeology project.
  • The persisted session event log captures the full run — every reasoning step, tool call, and result — and the admin dashboard makes every tool call across every session searchable and drill-in-able.
  • Observability and agent security share the same records: the log that helps you debug is the log that answers your auditor.
Audit log: actor, action, and resource for every consequential operation — with session and task ids

Questions

AI agent observability: common questions

What exactly is traced for each agent run?

Everything the run emitted: reasoning steps, streamed message deltas, every tool call with its result and status, sub-agent delegation and results, confirmations, state changes, errors, and completion. The full session event log is persisted, and the audit log adds actor type and subject, action, resource, a details JSON payload, session id, and task id for consequential operations.

Can I inspect a past agent session?

Yes. Every session is persisted with its message history and event log, and the dashboard gives you a filterable tool-call log — which tool ran, with what arguments, what came back, and whether it succeeded — plus the audit trail. There is no timeline-scrubber replay UI: the event log lives in your own PostgreSQL, so anything the dashboard does not surface you can query directly.

Does SyntheticBrew integrate with Prometheus and Grafana?

Yes, on Enterprise plans. The engine exposes Prometheus metrics — HTTP request counts and latency, active sessions, tool calls labeled by tool and status, and LLM requests labeled by provider and model — so you can build Grafana dashboards and alert rules on the monitoring stack you already run. Health checks, the session event log, the tool-call log, and the audit trail are included on every plan.

Can I get event and audit logs out of the platform?

Yes. The session event log, tool-call log, and audit trail are all readable through the API, so you can pull them into your own warehouse or SIEM on whatever schedule you want rather than relying on an export button.

See every step your agents take

Run an agent, open the tool-call log, and read every call it made — then wire the metrics into the monitoring you already trust.