Integrations
LlamaIndex
Use RouterBench as the LLM backend for LlamaIndex — one endpoint for every model.
LlamaIndex
RouterBench is OpenAI-compatible, so it drops straight into
LlamaIndex as the LLM. Point the OpenAILike class
at your RouterBench endpoint and everything downstream — RAG, agents, query engines —
routes through RouterBench (smart routing, guardrails, observability, structured
output).
Install
pip install llama-index-llms-openai-likeUsage
import os
from llama_index.llms.openai_like import OpenAILike
llm = OpenAILike(
model="routerbench/auto", # smart routing; or "claude-sonnet-4-6", "gpt-5"
api_base="https://api.routerbench.com/v1",
api_key=os.environ["ROUTERBENCH_API_KEY"],
is_chat_model=True,
)
print(llm.complete("What is RouterBench, in one line?"))Use it anywhere LlamaIndex expects an LLM:
from llama_index.core import VectorStoreIndex, Settings
Settings.llm = llm
index = VectorStoreIndex.from_documents(documents)
response = index.as_query_engine().query("Summarize the key risks.")Local & self-hosted models
Point model at a custom provider to run through
your own Ollama / vLLM / LM Studio endpoint via RouterBench:
llm = OpenAILike(model="custom/ollama-local/llama3.1", api_base="https://api.routerbench.com/v1",
api_key=os.environ["ROUTERBENCH_API_KEY"], is_chat_model=True)Get an API key at app.routerbench.com. For embeddings, use
OpenAILikeEmbedding pointed at the same api_base.
How is this guide?