Jev, “a new class of frontier models built to make fast, structured decisions”, was released just a few days ago (to a lot of fanfare), and I was teeming with excitement over how it could solve a problem I’ve run into many times with chat-based LLMs: reaching for a text-generation model when what I actually need is a structured decision.
I got early access and quickly went through their documentation. And I happened to have a perfect candidate already sitting in production, facing this very issue every day. I run an MLB (Major League Baseball) moneyline model. Every game day, it produces a win probability for each game, compares that to the best available odds, and sizes a bet.
On top of the ML-model-based recommendation, I had built an LLM-based arbiter. Its job is to read the pre-game news (injured lists, yesterday’s bullpen usage, etc.) and grade the bet UPGRADE / SUPPORT / DOWNGRADE / REJECT. This is useful because there’s no easy way to incorporate last-minute news and changes into an ML model built on historical team stats and game results. The LLM-arbiter helps fill this gap.
Now even though I am using an LLM for this purpose, this is really a multi-class classification problem! The objective is simply to pick the best label based on the input data (news highlights + prompt).
This general purpose chat model was working fine, but it was using 700-800 output tokens per game (reasoning, a narrative, per-factor adjustments in pp) just to arrive at one of four decision values. And I had to add a tool call and demand a strict JSON schema to enforce reliability and consistency of output. The failure rate was quite low (0.4%, mostly due to schema-coercion), but the point is: it’s a lot of overhead, unnecessary latency, and cost to get the model to simply select the best option.
Jev is built for the thing I actually wanted. It’s what TypeSafe calls a System One model (HT Daniel Kahneman’s Thinking Fast and Slow!). You pose typed questions against a state, and it returns the structured result directly. There’s no text generation, and no parsing. There are three primitives: Choice (pick from a list), Score (rate against a rubric), and Noul (is this statement true?). My problem was a four-way pick, so I used Choice, and I get back the selected option plus a calibrated probability for each option.
To quantify the improvement, I back-tested both approaches (chat model versus Jev) on ~400 games:
- Cost: went down from $7.33 to 3 cents, which is about 230× cheaper per game!
- Latency: 170ms median, 274ms p95, which is small enough to not even think about it any more.
- Every answer now comes with a probability distribution over the four labels, so I have a confidence number I can threshold on. I never had that before.
- No more type errors!
- The ROI on bet improved by +5.05 pp when compared to using a generative LLM. But the confidence interval runs from -2.1 pp to +12.1 pp — it spans zero, so I can’t call this a real effect yet.
What I lost is the explanation. The chat-based model was able to provide justification for its decision. Jev can’t directly do this.
What I described here is a classification use case, but Jev can be used for many other problems. If you’re using a chat model to do something when the model doesn’t need to explain itself or draft something for you, consider replacing it with Jev. If you just want the model to evaluate the input and simply make a decision, Jev is a better fit than a chat model. Jev doesn’t replace chat models; it complements them by handling problems they were not well-suited for in the first place.
