Paid: Tracking OpenAI Customer Costs

TLDR: in 5 lines of code, Paid allows you to completely understand how much each customer is costing in underlying model costs. In this talk, I plan to demo that solution and show how we built it.

Before:

openai_client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))  
run_agent(openai_client)  

After:

paid_client = Paid(token=os.getenv('PAID_API_KEY'))  
paid_client.initialize_tracing()  
openai_client = PaidOpenAi(OpenAI(api_key=os.getenv('OPENAI_API_KEY')))  
  
paid_client.capture("CUSTOMER_ID", lambda: run_agent(openai_client))  

The above is all you would need to understand exactly how much the customer is costing in terms of OpenAI.

As more and more companies are basing their products on commodity AI models. It is apparent that some customers will be responsible for more token usage than others (for example, a chatty customer speaking to a voice agent). Paid has a solution around this where we use OpenTelemetry to record how many tokens are used per customer. We also have agents scraping the current token price off the web to attribute customers to AI Model costs.

I will be demoing the solution as well as talking about how it works under the hood and how you can apply traditional telemetry techniques to new AI agent workflows.