# What is a decision model? Answers and evidence > Decision models return structured answers or probabilities. Jev accepts 3 question types: Choice, Noul and Score, with no generated text. Updated 2026-11-04 (Australia/Adelaide) Decision models return structured answers, such as categories, scores or yes/no probabilities. Jev returns these without generating text; some alternatives generate an option letter. [Compare decision models]() [Definition source: Simon Willison]() ## How decision models work You send the context your app has, such as a message or an agent's history. A game state or web page can also be input. For a Choice question, you supply the allowed options and receive probabilities for them. You can use those probabilities to decide which answers to act on and which to send to a person or a bigger model. [Source]() ## Decision models and generative LLMs A generative LLM can return structured output that your code reads. It generates its answer one token at a time. [Jev]() returns structured answers without generating text. Together reports that [Tev1]() keeps a text-generation head and is trained to return one option letter. [Source]() · [Source]() ## Choice, Noul and Score [Jev](), and the models built to its API, take three question types. Choice returns probabilities for the options you supply. A Noul question returns a yes/no probability; the name comes from the Bernoulli distribution. A Score question returns a value along your defined scale, which can fall between the numeric levels you supply. [Source]() ## Decision models and System One models TypeSafe calls [Jev]() a "System One model", after Daniel Kahneman's split between fast, intuitive thinking and slow, deliberate reasoning. Liquid AI uses "decision models" for [d1](). [Source]() · [Source]() ## Frequently asked questions ### When is a decision model good enough to replace an LLM call? Whether a model can replace an LLM call depends on the task and error budget. In Vals AI's claim-verification test, [Jev]() tied GPT-6-class models at 0.975. With a 1% error budget, [Jev]() automated 95% at 1.6% error, over budget; [Mercury Decide]() automated 94% at 0.8% error. In a study of conflicting retrieved sources, sending each decision model's least confident quarter to Gemini 3.8 Flash scored no differently from Gemini 3.8 Flash, statistically, for 8 of 11 models, at 25 to 37% of its cost. A 16-day Claude Code field report had mixed results. Of 240 scope judgments, 172 were uncertain or lacked evidence. The team removed [Jev](); token savings were never measured. [Source: Vals AI · Jev evaluation 6 Oct 2026, Mercury Decide evaluation 8 Oct 2026]() · [Source: Khatri, Jain, Kapoor, Sah · Do We Need an LLM to Know Why Sources Conflict? · arXiv 2610.36915 · 27 Oct 2026]() · [Source: sash-ua · Accurate and Useless · 30 Oct 2026]() ### Why does the answer change when I reword the question? The linked studies tested option lists and option order. They did not systematically test rewording. Across 1,700 questions, shuffling the options moved [Jev]()'s accuracy by under two points but grew the set of questions it ever got wrong by 27 to 29%, more than plain repetition. A small hand-written field guide offers thin evidence: [Jev]() often confidently chose the nearest option when the correct answer was missing. In a replay after the locked tests, adding a "none of these" exit raised correct handling from 11 of 14 requests to 13. [Source: Song, Zheng, Ma, Zhou · Understanding Jev-Style System One Models: Does Structure Reduce Selection Bias? · arXiv 2610.35826 · 26 Oct 2026]() · [Source: WeBuildDatabases · Jev, job by job · 29 Oct 2026]() ### Can I trust the probabilities, including on coin-flip questions? None of these studies tested calibration on coin flips. On other tasks, the results varied. In SILT's preregistered test of 799 unpublished yes/no questions about coding-agent work and destructive shell commands, [Decider]() V1.1-27B was the only one of seven models to pass SILT's chosen calibration rule. [Jev]()'s result was inconclusive. So were the results for OpenAI's [Decisions API]() and [Laya](). A separate banking-intent test found [Jev]()'s choice answers well calibrated but its yes/no answers underconfident on true cases. On MedQA, answers assigned at least 0.90 probability were still wrong 1.71% to 12.77% of the time, depending on the model. [Source: SILT · Do Decision Models Know When They're Wrong? · 11 Oct 2026]() · [Source: ankur287 · jev-open-calibration, code and data for a Towards Data Science article · Oct 2026]() · [Source: Freyer et al. · Decision models vs generative LLMs on medical questions · arXiv 2610.38448 · 28 Oct 2026]() ### How cheap is a moderation check, and where should the threshold sit? The reports give workload-specific thresholds. Squigdle's developer reported 6 cents for about 1,000 doodle checks, about $0.00006 each, using 512×512 images plus titles and the game's own thresholds. The reported misses were occasional false alarms. This single developer's live-game report gives little evidence about other workloads. In a separate moderation simulation, a fine-tuned [Laya]() router used a threshold set with labelled calibration data. It needed one extra moderator instead of three at 100 extra posts an hour. [Source: James O'Halloran · Moderating 500 daily anonymous doodles with Jev · 30 Oct 2026]() · [Source: Enayat Hassani · Exposure-aware routing · 29 Oct 2026]() ### Can a decision model sit in front of risky actions as a guardrail? Two tests point both ways. In a USC input-gate test, [Jev]() reduced answered harmful-labelled prompts from 527 to 17 out of 720, but refused 27 of 250 safe prompts, against 3 for the LLM alone. The study counted answered prompts as a proxy for harm; it did not check the completions. In Datadog's prompt-injection detector test, [Jev]() caught 82.90% of labelled attacks; prepending a fake policy inside the untrusted text reduced that to 3.88%. That study distinguishes a classifier bypass from a full agent compromise. [Source: Cognetti, Krishnamachari (USC) · Jev as an LLM Safety Guardrail · 2026, code and tables]() · [Source: Datadog Security Labs · 15 Oct 2026]() ### Do decision models work outside English? Performance depends on the language and task. Šuppa and Kopčan's XDecideBench tests nine suites across 122 languages. The English-best model, picked in hindsight, loses significantly in 56 of 232 suite-language cells. A threshold fitted on English to admit 5% errors instead admits 7.8% to 12.4% on German and French XNLI. Check the model's accuracy and confidence threshold on labelled examples from the language you plan to use. [Source: Marek Šuppa, Jaroslav Kopčan · XDecideBench · arXiv 2611.01370 · 01 Nov 2026]() [Jev alternatives]() [Decision model pricing]() [JevBench scores]()