Intelligent Routing
Not every prompt needs a frontier model. The Agent Broker routes each query to the optimal execution target — cache, SLM, RAG, or LLM — based on intent and context.
Routing Principles
The Agent Broker acts as an intelligent triage layer between user prompts and underlying execution engines. It dynamically evaluates every request to determine the most cost-effective, accurate, and fastest resolution path.
A lightweight, high-speed routing model parses the query. It strips PII, detects the core intent, and scores the cognitive complexity required to generate a correct answer.
Instead of defaulting to frontier models (like GPT-4 or Gemini 1.5 Pro), the router maps the prompt to the precise execution tier needed—be it a zero-token cache hit, a fast local SLM, or a specialized RAG pipeline.
The broker orchestrates connections to internal APIs, vector databases, and enterprise knowledge graphs. It extracts strictly the required context fields, preventing massive token dumps into the LLM.
Every prompt passes through this pipeline before execution.
Scrubs PII, credentials, and secrets at the edge before routing.
Checks the organization policy knowledge base to instantly address queries (e.g., policy FAQs) without generative LLMs.
Lightweight 1B–3B router model scores complexity in <20ms.
Filters relevant fields, preventing megabyte doc dumps.
Four Tiers
The context-aware router categorizes incoming requests into one of four execution tiers to balance cost, latency, and capability.
Deterministic lookups. Zero LLM tokens consumed.
Fast, local models for classification and extraction.
Retrieval-grounded generation from enterprise data.
Advanced reasoning and multi-agent orchestration.
Interactive Simulator
Select any workflow from the library to watch the broker analyze intent, classify complexity, and route to the optimal tier.
Financial Impact