Having trouble setting limits or tracking your customers usage? We’ve got you.
Introducing Usage Tiers.
The goal of this feature is to allow you to configure different usage tiers that can be applied to customers.
- Create named tiers, for example “Free Trial,” “Standard,” “Professional,” “Enterprise”
- Set call and token limits for each of the four model tiers (standard, premium, super premium, reasoning)
- Decide how to handle when customers hit their limit:
- Block: Send them a message and not respond
- Allow: Let them make the calls but keep track of the excess call and tokens for billing separately
- Restrict: Route the calls to a different model group, Eg. If the call was to premium models, route to standard
- Set replenishment schedules
- Usage Tiers are versioned. You can edit if there is a customer assigned, but you can clone and edit, but you get a new version number.
You setup and access these tiers through the Admin -> Usage Tiers
And when you create, or edit a usage tier BEFORE it has any customers you can enter the following values. Once a customer is assigned a usage tier, you MUST clone it to a new version number. This is to avoid disrupting the limits of existing customer on that tier. You need to migrate them to the new tier from the customers page.
Making a Usage Tier the Default for New Customers
In the Project Settings accessible from the Home page, you can set what tier first time callers get assigned to. For new projects, we create and assign the Default New Customer Toer (v1) when we setup your project. Existing customers must create that tier and assign it manually so we don’t disrupt any of your current customer.
Managing Customers
The Customers list now groups customers by usage tier.
There is a new tab in the Customers view when you select a customer. USAGE & LIMITS. It has all of the current tier, call use details, and replenishment options. If you want to move a customer to a new tier, click on the CHANGE TIER button.
Team LLMAsAService
Leave a Reply