Jev vs Claude for Java Intent Routing: Up to 7x Faster, 9 to 21x Cheaper, Not More Accurate

On 385 Banking77 messages Jev routed 84.9% correctly: a statistical tie with Claude Haiku 4.5, three points behind Sonnet 5 and Opus 5. It answered in 300 ms against 0.8 to 2.2 seconds, and cost $0.18 per 1,000 decisions against $1.59 to $3.89. Nobody broke a label. What Jev buys is speed, price and a usable probability.

What is Jev, and what does it return?

Jev is the first "System One" model from TypeSafe AI (TypeSafe below), launched on September 15, 2026. It doesn't generate text. It takes a piece of state and a set of typed questions and returns typed answers with probabilities: "System One models are built to make fast, structured decisions that software can use directly" (TypeSafe docs).

There are three question types. A choice picks one of up to 255 named options and returns a probability for each. A score returns a probability-weighted position on 2 to 10 levels you describe, counted from zero. A noul returns the probability that a yes/no condition holds (API reference). Input costs $0.042 per million tokens, output is free, and a request may carry 64k tokens, 32k of them state (models page).

Two facts matter for a Java team. There is no Java SDK, only Python and JavaScript, so you call POST https://api.typesafe.ai/v1/systemone yourself. And founder Diogo Almeida's launch post promises "70ms-500ms" responses and a model that "can't hallucinate", while its FAQ leaves "How does Jev perform against public benchmarks?" unanswered (launch post). So I ran one. LangChain, which built routing middleware on Jev two days after launch, set the scope:

"Jev isn't a drop-in replacement for an LLM... it can handle classification tasks we often use LLMs for today, without the same latency and cost." (Sydney Runkle and Hunter Lovell, LangChain, September 17, 2026)

How did I test it?

The task is intent routing on Banking77 (PolyAI, CC BY 4.0): 77 support intents, 3,080 test messages with a median length of 45 characters, from Casanueva et al., 2020 (arXiv:2003.04807). I took 5 messages per intent, 385 in total, and ran Jev on the full 3,080 as well. Every model saw the same 77 labels with the same two training examples per label, one message per call.

Jev was called from plain Java 21, java.net.http.HttpClient plus Jackson, one choice question with 77 structured options. The Claude arms went through the official Anthropic Python SDK against the Messages API: Haiku 4.5 with the provider's JSON schema, Haiku 4.5 with format instructions in the prompt plus a parser and a three-attempt validate-and-retry loop (what Spring AI's validateSchema() does), Sonnet 5 and Opus 5 with the schema, Opus at low effort. Latency is a separate run of 60 sequential calls per model, timed client-side on the same machine in Poland. Cost is billed tokens times list price.

Two mistakes first. Without an API key at the start, I pushed the Claude arms through Claude Code subagents in batches of 50 messages per JSON array; that produced a failure I'll come back to, and I redid everything per call. And my 256-token output cap was too low for Sonnet 5, which thinks before it answers: two calls returned no JSON. My harness, not the schema; I raised the cap to 1,024 and reran those two.

What do the numbers say?

Jev ties Haiku on accuracy, trails Sonnet and Opus by three points, and beats all three on latency and cost by a wide margin.

Paired on the same 385 messages, Jev versus Haiku is not significant: 23 messages only Jev got right against 11 only Haiku got right, exact McNemar p = 0.058 for the schema arm and 0.15 for the prompt arm. Sonnet (p = 0.017) and Opus (p = 0.004) are distinguishable from Jev. On the full 3,080 Jev scored 85.2%, 95% interval 83.9 to 86.4, at p50 288 ms under six concurrent calls. Claude prices: Haiku $1 in / $5 out per million, Sonnet $2 / $10, Opus $5 / $25, cache reads at 10% (Anthropic pricing, September 2026).

Did prompt-and-parse break any labels?

Not once per call. Across 1,540 per-call Claude requests, four arms times 385 messages, every answer was one of the 77 labels, and the retry loop never fired. On a 77-way enum with the label list in front of it, Haiku 4.5 held the schema whether I enforced it or merely asked for it.

The break came from batching. In my first pass, 50 messages per JSON array, Haiku tidied the label reverted_card_payment? to reverted_card_payment three times out of five, and three message ids simply vanished from three different arrays. Opus made neither mistake. Batch a structured-output call and you get exactly the failure the frameworks warn about; call per message and, at this size, you don't.

That warning is real, just rarer than I expected. Spring AI 2.0.1's reference: "The AI Model is not guaranteed to return the structured output as requested" (Spring AI docs). Christian Tzolov, who built the retry loop, on its limit:

"The model is asked to produce JSON matching the schema, not forced to. Most of the time it complies. Sometimes it doesn't." (Christian Tzolov, Spring team, June 23, 2026)

LangChain4j ranks its strategies by reliability and calls the prompting-only fallback "quite unreliable" (LangChain4j docs). One correction to that page: its provider list omits Anthropic, but langchain4j-anthropic 1.20.0 ships native JSON-schema output for Anthropic, opt-in through supportedCapabilities(RESPONSE_FORMAT_JSON_SCHEMA) on the model builder; I checked the jar. Without the flag an AiService returning an enum runs on prompt instructions plus a parser, the path my prompt arm tested.

Jev's choice can't return anything outside the keys you sent, the probabilities sum to one, and the answer is the decision rather than text about it. I think that guarantee is worth having, but on this data it's insurance against a tail, not a fix for a measured rate. I hand-rolled a validate-and-retry loop before Spring AI shipped one; here it would never have fired.

Is Jev really 400x cheaper?

Measured against the model you'd actually use, 9x to 21x, and the reason is caching. LangChain's post repeats the vendor's "up to 200x faster inference and 400x lower cost" (LangChain). Jev has no prompt caching and re-sends the 77 options with their examples on every call: 4,350 input tokens, $0.18 per 1,000 decisions. Sonnet and Opus cached the label prompt from the second call and cost $1.59 and $3.55 per 1,000, 8.7x and 19x. Haiku never cached: its minimum cacheable prompt is 4,096 tokens and my label prompt was 2,887 to 3,788, so it paid full price every call, $3.01 to $3.89 per 1,000, 17x to 21x (prompt caching docs). Cheapest LLM on the list, most expensive per decision.

Latency held closer to the claim: 300 ms median and 364 ms p95 for Jev on sequential calls from Poland, inside the promised 70 to 500 ms band. Haiku answered in 770 to 968 ms, Sonnet in 2.1 s, Opus in 2.2 s, so 2.6x to 7.5x slower than Jev, not 40x to 200x. Every model's first call cost about a second more for TLS.

Is the confidence value usable?

As a ranking, yes; as a probability, less so. Of 385 answers, 306 came back with a top probability of 0.9 or more, and 93.8% of those were right while the bin's mean confidence was 0.991. The middle bins are worse: between 0.6 and 0.8 the gap between stated and actual reaches 0.27, on 17 to 19 messages each. Jev is overconfident across most of the range, most consequentially at the top; expected calibration error is 0.078 on the sample and 0.073 on the full 3,080. TypeSafe's docs say calibration "applies to prediction groups, not individual answers" (System One concepts), and that matches what I see.

The ranking works. The 79 answers below 0.9 (20.5% of the sample) were right 50.6% of the time; on those same 79 messages Haiku scored 50.6% and Opus 65.8%. The low tail is hard for every model. Stack the two result sets and you get the escalation curve a Java service would actually run: keep Jev's answer above a threshold, send the rest to a bigger model, with every cost figure below billed, not estimated.

Sending one message in ten to Sonnet recovers most of the gap for a fifth of Sonnet's price. Sending them to Haiku doesn't help: 86.0% at 0.7, then back toward Haiku's own 81.8%.

The rest is mostly the dataset, with one caveat. Thirty-nine messages were wrong for Jev, Haiku and Opus alike, and on 25 of them all three chose the same label. "Why isn't my transfer going through? I get a message saying it's not possible" is labelled beneficiary_not_allowed; nothing in the text says so. But all three models saw the same thin description of every label, its name and two examples, so part of that agreement may be my criteria rather than Banking77's ambiguity. Either way the probability finds those cases, and no model in this run fixed them.

Where does Jev plug into Spring AI and LangChain4j?

Four seams, all of them interfaces the frameworks already expose: a CallAdvisor guard and a DocumentPostProcessor reranker in Spring AI, a ScoringModel reranker and an InputGuardrail in LangChain4j. I compiled all four against Spring AI 2.0.1 and LangChain4j 1.20.0, the current releases on September 22, 2026 (Spring AI releases, LangChain4j releases). They compile; I didn't run them against live traffic, and that's a lower bar.

The client is the call from the benchmark, trimmed but with the status check kept, because without it a 401 quietly becomes an empty intent:

ObjectNode body = JSON.createObjectNode();
body.put("state", customerMessage);
body.put("model", "jev-latest");
body.putObject("questions").set("intent", question); // {"type":"choice","instructions":..,"criteria":{label:{what,examples}}}

HttpRequest req = HttpRequest.newBuilder(URI.create("https://api.typesafe.ai/v1/systemone"))
    .header("Authorization", "Bearer " + apiKey)
    .header("Content-Type", "application/json")
    .timeout(Duration.ofSeconds(10))
    .POST(HttpRequest.BodyPublishers.ofString(JSON.writeValueAsString(body)))
    .build();

HttpResponse<String> resp = http.send(req, BodyHandlers.ofString());
if (resp.statusCode() != 200) throw new IOException("Jev HTTP " + resp.statusCode());   // retry 429/5xx, fail the rest
JsonNode answer = JSON.readTree(resp.body()).path("answers").path("intent");
String intent = answer.path("choice").asText();          // always one of your labels
double pTop   = answer.path("probabilities").path(intent).asDouble();

The key lives in a property, @Bean JevClient jevClient(@Value("${jev.api-key}") String key), never in code. In Spring AI the natural seat is a CallAdvisor in front of the model: one noul (is this in scope?) and one score (how urgent?) about the user text in a single request, short-circuit the chain below a threshold without spending an LLM call, otherwise pass both numbers along in the request context. Score levels count from zero, so a three-level urgency comes back in the range 0 to 2.

Decide the failure policy per seat before you ship. Mine: the guard and the Spring reranker fail open with a warning in the log, because a Jev outage must not take the assistant down; the LangChain4j ScoringModel fails closed, because a reranker that silently returns the wrong order is worse than an exception; the InputGuardrail returns failure() so LangChain4j retries it. JevClient retries 429 and 5xx three times and throws on anything else. Above 255 options you shard a choice into a hierarchy, which the docs cover and I didn't test.

What would I use it for, and what not?

Where I'd put it first: in front of the LLM, as the router and the guard, wherever the decision is one of N and the LLM call is the expensive part. Any bulk classification job with a fixed label set is the same shape.

Where I wouldn't: anything with numbers. TypeSafe's own limitations page says "jev-1.13 does not count reliably", "reads dates as text, not as ordered quantities", answers literally, and "does not treat [state] as hostile by default" (jev-1.13 limitations). Do the arithmetic in code and hand Jev the semantic part. It's English-first, and it can't write a sentence, so the LLM stays in the loop for anything a human reads.

What this means in practice

If a decision in your Java service is "one of N" or "how much", the case for a typed model is latency and price, not correctness: in this run Jev was 2.6x to 7.5x faster and 9x to 21x cheaper per decision, and no more accurate than Haiku. Don't buy it to fix broken enums; per call, on 77 labels, Claude didn't break any. Do the cost math with your real prompt against each model's cache floor, because Haiku under 4,096 tokens costs more per decision than Opus with a cached prompt. And use the probability for the thing it's good at: routing the 10 to 20% hard cases to Sonnet, where a fifth of Sonnet's price buys most of its accuracy.

The framework choice itself hasn't changed: Spring AI vs LangChain4j covers it, and Spring AI 2.0's reliability plumbing is what you'd be putting Jev in front of.

I have no affiliation with TypeSafe. The Claude calls for this piece cost about $5, paid by me; Jev was free under its preview, and would have been about 70 cents at list price.

FAQ

What is a System One model? A model trained to return a typed decision with calibrated probabilities instead of generating text. TypeSafe's Jev, released September 15, 2026, answers three question types: a choice among named options, a score on described levels, and a yes/no probability. Your code branches on the values.

Does Jev have a Java SDK? No. As of September 22, 2026 TypeSafe ships Python and JavaScript SDKs only. From Java you call POST https://api.typesafe.ai/v1/systemone with a bearer token and a JSON body of state plus questions; the client in this article is about 110 lines on java.net.http and Jackson, retries included.

Can Jev replace structured output in Spring AI or LangChain4j? Only for decisions, not for content. Where the output is a label, a rating or a probability, a choice, score or noul returns it typed by construction, 7x faster than Sonnet 5 or Opus 5 in my run. Where the output is text or a record with free-form fields, you still need an LLM.

When should a Java service escalate from Jev to an LLM? When Jev's top probability is below a threshold you set on your own labelled data. Here a threshold of 0.7 sent 10.4% of messages to Claude Sonnet 5 and raised accuracy from 84.9% to 87.5% at $0.36 per 1,000 decisions, against 88.1% at $1.77 for Sonnet alone. Escalating to Haiku 4.5 did not help.

---

Sources

1. TypeSafe docs: Introduction — definition of System One models and what Jev returns (Tier 1, vendor's own product docs).
2. TypeSafe docs: API reference — endpoint, question types, option and level limits (Tier 1).
3. TypeSafe docs: Models — jev-1.13.0, $0.042 per million input tokens, 64k/32k budgets, English-first (Tier 1).
4. TypeSafe: Introducing System One Models & Jev — launch post by Diogo Almeida, September 15, 2026; latency and "can't hallucinate" claims; benchmark FAQ (Tier 3, vendor claims).
5. TypeSafe docs: jev-1.13 limitations — counting, dates, literal reading, adversarial state (Tier 1, self-reported).
6. TypeSafe docs: System One concepts — calibration applies to groups (Tier 1).
7. LangChain: Building a harness with Jev — Sydney Runkle and Hunter Lovell, September 17, 2026 (Tier 3, partner).
8. Spring AI reference: Structured Output Converter — "not guaranteed" statement, version 2.0.1 (Tier 1).
9. Spring: Self-Correcting Structured Output in Spring AI 2.0 — Christian Tzolov, June 23, 2026; validateSchema, 3 attempts (Tier 1).
10. LangChain4j docs: Structured Outputs — strategy ranking, provider list, fallback (Tier 1); Anthropic JSON-schema support verified on the langchain4j-anthropic:1.20.0 jar.
11. Spring AI releases and LangChain4j releases — 2.0.1 (2026-08-21) and 1.20.0 (2026-09-04) (Tier 1).
12. PolyAI Banking77 — dataset, CC BY 4.0; Casanueva et al. 2020 (Tier 1).
13. Anthropic pricing — Haiku 4.5, Sonnet 5, Opus 5, cache read rates, September 2026 (Tier 1).
14. Anthropic: prompt caching — minimum cacheable prompt length per model (Tier 1).
15. Own measurements, September 22, 2026 — Jev from Java (385 + 3,080 + 60 sequential calls), four Claude arms through the Anthropic SDK (385 + 60 sequential each), paired tests and billed-token costs; raw rows and code kept locally (owner data).