Metering and auditing every LLM, MCP and agent call
Sam
Summary
I present the Gravity Agent Management Gateway to show how we can stop AI projects from stalling after six months by providing a single control plane that manages access, policy enforcement, cost budgeting, and observability for LLMs, tools, and agents. I demonstrate a self‑service subscription flow where a user discovers a resource in the developer portal, requests access, and a human approves the subscription, automatically issuing an API key. The gateway catalogs models, MCP servers, and tools, giving each a reusable identity. I then walk through the LLM proxy stack—cost rate‑limiting, content guardrails, PII filtering, and semantic caching—explaining how policy order protects downstream calls and how budget breaches block only the offending consumer. Observability dashboards expose token usage, spend per consumer, and model performance, and I illustrate per‑team budgeting in AI workspaces. Next, I integrate an external Azure AI Foundry agent that calls a Gravity‑registered tool, showing the approval workflow and end‑to‑end logging. Finally, I use MCP Studio to build a custom endpoint that aggregates multiple tools, configure fine‑grained authorisation, and attach a Stripe MCP server for billing customers. The demo concludes with a recap of full visibility, governance, and compliance (GDPR, EU AI Act) across the entire agentic stack.