Use case: “LLM-Ops” for Applications
Goal
Integrate and operate reliable LLM features in applications at scale.
How
Quickly build against our AI Gateway and benefit from a variety of safety, scalability, reliability and cost saving features. If you are already using LLM models in your products then with a few lines of code you get production hardened systems that are easier to operate, observe, and rollout that next model of LLM enabled feature.
Escape the proof of concept purgatory...
Our value prop is simple: We handle the hidden features needed to deliver reliable, and valuable LLM features to your customers
Getting started is fast and easy...
Step 1: Get started in minutes. Sign up for a trial account and call our API where you would have called any of the vendor APIs. Or, use our open source pre-built Chat Panel controls or iFrame embed snippets. You have a choice of using our vendor API keys (for central billing) or your own. Add multiple models for redundancy and cost control.
Step 2: Monitor your call errors, conversations and model costs within the dashboard; Respond to feedback and calls to actions that customers submit.
Step 3: Start building more advanced Agents to solve more customer centric features.
Control panel dashboard
Manage and observer all LLM use from the control panel
Agent UI Builder
Build "agents" to solve specific customer or internal use cases
Multiple Models and Model Groups
Create groups of different model strength, response time and model versions
Safety and PII Routing
Personal Information, Prompt Safety & Smart Model Routing configuration
Full model and customer costs
See real-time customer and vendor costs for LLM features. Set limits.
Call Trace Logging
Diagnose model calls through call tracing
Want to see it in action?
Frequently Asked Questions...
Depending on your specific coding framework you have many options for making calls to our API:
- A plain https post fetch call to our chat.llmasaservice.io endpoint
- our useLLM hook from our NPM package for React/Next.js. This hook allows you to make direct streaming or non-streaming text completion calls to and of your model groups or models.
If you want prebuilt customizable fully features chat panels, you can use our NPM UI libraries package. This gives you all of the analytics, conversations, call to actions, email features without you coding anything You just add an <AgentPanel … /> or <ChatPanel … /> components.
The built in Agent Panel is fully customizable via CSS (all colors and layout). Custom CSS files can be linked in the control panel, full documentation can be read here. On the rare occasion complete interactive control of the Agent Interface is needed, our panel is Open Source on GitHub and can be forked and customized.
Yes, see our knowledge base Developers section at help.llmasaservice.io
There are a lot of client side helper libraries and APIs that make calling specific vendors APIs easier. We operate differently, we are a gateway. We add a lot of features in addition to just making calls to the vendor models. We redact PII, we block banned phrases, we look at the prompt complexity and make a decision about what model strength to call, we logs the conversations and calls, we have prompt libraries and RAG ingestion / querying, we route based on geographic restrictions, plus more. Its these features in combination that make your LLM feature respond reliably in production.