Smart Routing: Matching Requests to the Right Model

Not every prompt needs your most capable (and most expensive) model. A classification step, a JSON extraction, or a short FAQ answer can run on a smaller model at a fraction of the cost. The hard part is deciding per request without writing brittle rules everywhere.

Yaha's smart routing evaluates each incoming request and selects the most cost-efficient model from your project's allowlist while respecting quality constraints you define in the console.

Why static routing breaks down

Hard-coded if prompt.length < 200 rules drift quickly. Product teams add new flows, agents send multi-turn context, and what looked "simple" yesterday carries hidden complexity today.

Semantic routing looks at the meaning of the request, not just character counts. That lets one policy serve chat apps, batch jobs, and coding agents without forked logic in every service.

How it fits your allowlist

Smart routing never sends traffic to models you have not approved. You configure an allowlist per project (for example a small fast model, a mid-tier model, and a stronger one for hard cases), and routing picks the best fit within that set.

This matters for compliance and procurement: routing optimizes inside your approved catalog, not around it.

Cost vs quality in practice

Consider a support bot with two tiers of traffic:

• Tier 1: password resets, order status, hours of operation, routed to a fast, cheap model
• Tier 2: disputed charges, policy exceptions, escalated to a stronger model

Without a gateway, engineers encode that split in application code and redeploy when pricing or models change. With Yaha, you adjust routing policy in the console and every client picks it up on the next request.

Observability closes the loop

Every routed request is logged with the model chosen, latency, and estimated cost. You can review spend in the console and tune policies with data instead of guesses.

Getting started

Enable smart routing under Projects → Intelligence in the console, set your allowlist, and send traffic through the gateway. No client changes beyond using your Yaha project key.

curl https://gateway.yahagateway.io/v1/chat/completions \
  -H "Authorization: Bearer yaha.your-project-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Summarize this invoice line item"}]
  }'

Use "model": "auto" (or your project's configured routing alias) to let the gateway choose.

Smart routing is not about always picking the cheapest model. It is about not overpaying by default. Configure your floor, set your ceiling, and let the gateway handle the middle.

Explore routing in your project: Open the console.