, 8 min read
How I used TypeSafe's new System One model, Jev, to pick the right model for every message in PINTER, the trade-offs I chose, and what I would change next.
From the project PINTER
TypeSafe AI released Jev on 15 September 2026. It is their first "System One" model, and it does not write text. A week later it was the routing step in PINTER, the AI agent platform I build at Pupuk Indonesia. This post covers why I picked it, how the router works, and where it still falls short.
The problem
In PINTER, each agent was tied to one model. That forced a bad choice. Put a strong model behind everything and simple questions cost as much as hard ones. Put a cheap model behind everything and hard questions get weak answers.
Most messages are simple. A greeting, a lookup, a short rewrite. A few need real reasoning. I wanted each message to go to the model that fits it, decided per message, without the user doing anything.
Why not ask an LLM to pick
The common trick is to prompt a language model: "Here are the models. Reply with the name of the best one." It works in a demo. In production it has three problems.
- The answer is text. You parse it and hope it matches one of your options.
- There is no confidence. A guess and a sure answer look the same.
- You pay for a generation step to get one word back.
A router needs a choice from a fixed list, a number that says how sure it is, and a clean way to fail.
What a System One model is
Jev is built for that shape of problem. You send it a state, which is any text, and a set of typed questions. It answers each question with a structured value, not prose. There are three question types:
| Type | Asks | Returns |
|---|---|---|
| Choice | Which option fits? | choice, probabilities, confidence |
| Score | How does this rate on a rubric? | score, probabilities, confidence |
| Noul | Is this statement true? | a value from 0 to 1 |
A Choice question is a router. The options are my routes, and the confidence tells my code whether to trust the answer. Questions in one call run in parallel, so adding more later barely changes the response time. Input costs $0.042 per million tokens, and there are no output tokens to pay for.
How the router works
PINTER already had virtual models: a name that points to real models. combo tries models in order until one works. round_robin spreads load. I added a third type, smart_router.
An admin sets it up with:
- Routes. Each route has an intent key, a plain description of what belongs there, and the target model.
- A fallback model for anything the router cannot place.
- A confidence threshold, 0.65 by default, adjustable from 0.10 to 0.95.
My router, smart-pinter, has four routes. Cheap text goes to a fast model, tool-heavy steps to a low-latency one, images to a vision model, and hard reasoning or code to the strongest model.
When a message comes in:
- PII in the message is masked before anything else sees it.
- The router sends the latest user message to Jev as one Choice question. The route descriptions are the options.
- Jev returns the chosen intent, its confidence, and the probability of every option.
- If the confidence meets the threshold, the message goes to that route's model. Otherwise it goes to the fallback.
- The agent runs on the chosen model and streams its answer.
- The decision streams to the chat as well, so the user sees it.
The request to Jev looks like this. The descriptions are shortened.
{
"state": "<latest user message, PII masked>",
"model": "jev-latest",
"questions": {
"intent": {
"type": "choice",
"instructions": "Determine the optimal handling tier for this request",
"criteria": {
"general_low_cost_text": "High-volume, non-critical text tasks",
"low_latency_tool_calls": "Text-only tasks that need fast responses",
"multimodal_vision_processing": "Prompts with images, charts, or scans",
"advanced_reasoning_and_code": "Complex coding and multi-step reasoning"
}
}
}
}
The decision itself is plain code:
choice, confidence, probabilities = await evaluate_choice(prompt, criteria)
if choice and confidence >= threshold:
route = find_route(routes, choice) # case-insensitive match on intent key
if route:
return route.target_model_id
return fallback_model_id
That split is the point. Jev makes the judgment. My code decides what to do with it, and the threshold is a setting, not a prompt.
Design decisions
It fails open. If Jev times out after 5 seconds, returns an error, or there is nothing to classify, the message goes to the fallback. The client never raises. A broken triage step should never break the chat.
The threshold is a dial. A high threshold sends more traffic to the fallback. A low one trusts Jev more. Admins tune it per router without touching code.
Only the latest message is classified. This keeps the call small and the result predictable. The cost is that a short follow-up like "and the second one?" is judged without its context.
Private data stays masked. Jev receives the message after PII is replaced with tokens.
Keys live in the vault. The Jev API key can come from a registered model, so it is not stored in the router config. When the config is read back, the key is masked.
No nesting. A route must point to a real model, not another virtual one. That keeps every decision one hop deep and easy to explain.
The decision is visible. Each answer carries a Smart Routing card: the intent, confidence against the threshold, and a bar for every option. It is saved with the message, so it shows up in history and shared chats too.
What a fallback looks like
Not every message fits one route. I asked for a map of fertiliser demand across Indonesia plus a chart of sales trends. Jev's best guess scored only 0.14, far below 0.65. My guess is that the request mixes data lookup, tool calls, and visual output, so no single description matched it well.
The fallback model handled it. It listed the datasets, queried them, and drew both visuals.
This is the case the fallback exists for. It gets every message the router is unsure about, so it has to handle any kind of request, including tool calls. The low score is also a signal: requests like this may deserve their own route.
One bug worth sharing
The first version never matched a single route. Jev was answering correctly. The problem was the config encryptor. It encrypted every field whose name contained "key", to protect secrets. intent_key matched that rule, so every route name was stored encrypted and never equal to Jev's answer. The fix was a short list of exempt field names, plus a test that route names stay readable.
A rough cost estimate
I have not measured savings yet, but list prices give a first idea. The assumptions:
- 3,000 input and 600 output tokens per message.
- A traffic mix I guessed: 40% cheap text, 20% tool calls, 10% vision, 15% hard reasoning, 15% fallback.
- Public list prices on 26 September 2026, off-peak for DeepSeek.
| Setup, per 10,000 messages | Cost | Smart router compared |
|---|---|---|
| Smart router, Jev included | $21.06 | - |
| Everything on Gemini 3.8 Flash | $45.00 | 53% cheaper |
| Everything on Deepseek V4 Pro | $31.68 | 34% cheaper |
| Everything on Gemini 3.5 Flash Lite | $24.00 | 12% cheaper |
| Everything on DeepSeek V4.1 Flash | $8.10 | 2.6x more expensive |
Three things stand out.
- Jev is close to free. It adds $0.17 per 10,000 messages, under 1% of the bill. It only reads the latest message and has no output tokens.
- The saving depends on the baseline. Against the strongest model, the router saves about a third. Against the cheapest model, it costs more. What it buys there is quality on the hard messages.
- The fallback is the biggest lever. At these prices Gemini 3.5 Flash Lite costs more per message than Deepseek V4 Pro, because its output tokens cost more. With 15% of traffic landing there, the fallback choice moves the bill more than any route.
The dollar amounts are small at this scale. The ratio is what matters, and it only holds if the guessed mix is close to real traffic.
What is not measured yet
I would rather say this plainly than guess.
- Latency. I have not benchmarked the extra time the triage step adds.
- Cost. The estimate above uses a guessed traffic mix. I have not compared real spend before and after routing.
- Accuracy. Routing has not been scored against a labelled set of real messages.
These are the next things to measure, and they decide whether the thresholds are right.
What I would change next
- Retry on the fallback in chat. The OpenAI-compatible endpoint already tries the fallback when the chosen model fails. The chat path does not yet. Both should behave the same.
- Track the fallback rate. A fallback today only writes a warning to the log. A rising fallback rate is the first sign that routes or thresholds need work, so it should be a metric with an alert.
- Ask more than one question. Jev evaluates questions in parallel, so one call could return a Choice for the topic and a Score for complexity. My code would combine them. Changing priorities would mean changing a number, not rewriting a prompt.
- Give follow-ups some context. Classifying the last two or three messages together would fix the "and the second one?" case.
Takeaway
Choosing a model is a decision, not a conversation. Once I treated it that way, the router became small: one typed question, one number, and a few lines of code that decide what to trust. Jev is new, but that idea is the part I would keep with any tool.