# JevBench scores and methods for decision models > JevBench independently evaluates Jev-compatible models. Its 1 Nov 2026 board reports 337 measured models and 217 ranked on the open-weights board. Updated 2026-11-04 (Australia/Adelaide) JevBench is Benchmark Heaven's independent evaluation of Jev-compatible decision models, run by Florian Standhartinger. It measures decision quality and calibration alongside speed and cost, with separate boards for self-hosted systems and hosted APIs. [Source]() · [Source]() · [Source]() ## Board published 2026-11-01 (v1.8.6) The board published 01 Nov 2026 reports 337 measured models and 217 ranked systems on its open-weights board. [Jev]() 1.13.0 is an unranked reference at 77.1 Capability. The first four rows have open-weight status Unknown on this board; check each model's licence before choosing how to run it. | Rank | Model | Capability | Board's open-weight status | | --- | --- | --- | --- | | 1 | [Baseten Mammoth-1 (27B)]() | 85.2 | Unknown | | 2 | [KnowLine 31B-Gen1]() | 85.2 | Unknown | | 3 | [Fastino GLiDE 1.2]() | 84.9 | Unknown | | 4 | H2O-Lightning-31B v1.0.7 | 84.3 | Unknown | | 5 | [Perplexity Decider v1.1 27B]() | 82.7 | Yes | | 143 | [SemIf (formerly OpenJev), Qwen3.5-4B]() | 55.4 | Yes | [Source]() ## What JevBench measures Each request supplies application state and a bounded question. Intelligence measures chance-corrected decision accuracy, with tier weighting and a public/sealed gap adjustment. Calibration scores the probability distribution. Capability is their arithmetic mean, with ranks restricted by cost and median-latency caps. The hosted API board leads with Composite, a weighted harmonic mean of Intelligence, Calibration, Speed and Cost with penalties. It also reports dollars per 1,000 decisions and p50 latency. [Source]() ## Where SemIf, OpenJev and TypeSafe's Jev sit [SemIf]() is the project formerly called [OpenJev](), independent of TypeSafe. Its Qwen3.5-4B row ranks #143 on the 01 Nov 2026 v1.8.6 board, with 55.4 Capability, 27.1 Intelligence and 83.6 Calibration. [Jev]() 1.13.0 scores 77.1 Capability as the unranked reference. TypeSafe's [Jev]() 1.14.0 appears as a second unranked reference here and is ranked on the API board. These scores describe the recorded versions and setups. [Source]() ## Timing and cost rules Method v1.7 counts a decision taking more than 10 seconds as wrong and gives it zero calibration credit. Exactly 10 seconds passes. Self-hosted timings are measured over loopback; ranking uses an estimated response latency, including network and serving overhead, of twice measured p50 plus 150 ms. That estimate has not yet been validated against public endpoints. From v1.8, self-hosted cost uses estimated GPU time, capped at the previous list-price estimate. Hosted rows retain published API prices. [Source]() ## Artificial Analysis Decision Index Artificial Analysis runs a separate evaluation. Its v1.0 index, launched 13 Oct 2026, combines four evaluations covering knowledge and hallucination, reasoning, tool calling and safety classification. Cost, speed and calibration are reported alongside it. Subquadratic's 22 Oct release post quotes 65 for [SubQ Snap]() 1.1 on that index. Its scale and tasks differ from [JevBench]()'s, so scores cannot be compared directly. [Source]() · [Source]() ## Jev Decision Index The community Jev Decision Index, hosted by multimodalart on Hugging Face, has its own tasks and scoring rules. For the v0.4 text board, Full score combines 60% private tests of the same skills and 40% private tasks from new domains, each mapped onto the public scale. Public text results are displayed separately and excluded from Full score. Scores and ranks cannot be carried between this index, Artificial Analysis and [JevBench](). [Source]() [Jev alternatives]() [Published model prices]() [What is a decision model?]()