Concepts

Arbio routes requests based on the rules you specify. You can create multiple rules and customize how this works using the Arbio web application.

The key concept is that all changes to routing occur in the Arbio platform. With no changes to your code

Terminology

• Providers - Companies like OpenAI, Grok, Microsoft Azure that provide API access to LLMs.
• LLM Models - Examples are gpt4-o, llama3, etc.
• Rule - Specifies a route from your application to a Provider's LLM Model.

LLM Conversion Routing

You can convert from one Provider or LLM Model to another without changing code.

The example below shows how to specify this using web-based Rule editor

JWT Routing

Arbio is able to optionally accept signed Oauth bearer tokens (JWT) instead of API tokens. This allows applications that prefer to use JWT for auth to avoid sharing tokens with their users.

Load Balancing

Arbio's default behavior is to load balance requests across providers as specified in your rules.