Track five signals: p95 end-to-end agent run latency, task success rate, average tokens per completed task, tool call error rate, and retry or loop rate. Token count per task serves as a cost proxy and catches prompt bloat or reasoning loops before they appear on the invoice. All five require span-level granularity; aggregate metrics will not surface the outlier runs that drive most of the cost and most of the user complaints.