Evaluation and Observability

Cost Tracking

Cost tracking monitors and attributes the token and API expenditure of agent systems, giving you financial visibility into what individual tasks, workflows, and users actually cost to serve. In agentic systems, costs are unpredictable in a way that standard API usage is not: reasoning depth, tool call count, and context window size all multiply against each other, so a task that takes 3 tool calls at 2k tokens each costs an order of magnitude less than one that takes 40 calls at 8k tokens each, and neither outcome is knowable in advance. Effective cost tracking aggregates at multiple levels from per-call token counts to per-task totals to per-workflow trends, giving you the data needed to distinguish normal variation from a runaway loop, target optimization efforts, and set defensible prices.

connected to