Decision models vs classifiers, embeddings and rules

What each approach needs

A supervised classifier learns task labels from examples. BERT's paper describes fine-tuning a pretrained encoder with an added output layer. Jev's Choice API instead takes the question and option descriptions with each request. Runtime labels also predate Jev: Yin, Hay and Roth studied classification of unseen labels through textual entailment. A runtime-label interface does not remove the need for labelled examples when you measure accuracy or fit calibration.

Source: arxiv.org · Source: arxiv.org · Source: docs.typesafe.ai · Evidence: ankur287 · jev-open-calibration, code and data for a Towards Data Science article · Oct 2026 · Source ↗

Classic classifiers: the Banking77 result

In ankur287's independent test, logistic regression with temperature scaling tied Jev's Choice accuracy on seven banking intents at 0.975. Jev's calibration error was 0.014 against 0.022, with the difference within noise. On all 77 intents, the trained classifier scored 0.888 accuracy against Jev's best tested 0.859 with three-shot criteria. The classifier used labelled Banking77 training data. These results do not establish how either approach would perform on a different label set.

Evidence: ankur287 · jev-open-calibration, code and data for a Towards Data Science article · Oct 2026 · Source ↗

Embeddings plus a trained head

Voyage embedded documents, combined the embeddings with raw fields, and trained ridge regression on labelled documents. Across 29 classification tasks, its experimental voyage-json baseline averaged 88.3 AUROC against Jev's 77.6. VENDOR-REPORTED: Voyage evaluated its own models. This comparison gives the embedding baseline task-specific training examples while Jev labels documents without learning from that collection. It is evidence for using existing labels, not a matched zero-shot comparison.

Evidence: Voyage AI blog · How to create your own Jev using Voyage models · 27 Oct 2026 · Source ↗

Rules: a paging example

Rules need an explicit condition to test. Coroot's burn-rate rule used a fixed one-second service-level objective and a three-minute hold, scoring 30 of 31 synthetic scenarios correctly. Tuned Jev 1.13 scored 29, Tev1-4B 28, and untuned Jev 27. VENDOR-REPORTED: these are Coroot's measurements. Model thresholds were tuned on the same scenarios, which were designed to be obvious under that fixed objective.

Evidence: Coroot · Anomaly detection for alerting · 03 Nov 2026 · Source ↗

Reranking has its own evidence

Dinzinger and colleagues compared six Qwen backbones on ten TREC DL and BEIR collections. Reranker fine-tuning beat decision fine-tuning by 5 to 29 nDCG@10 points up to 9B parameters; at 27B they tied. Decision fine-tuning never produced the better reranker in that study. Its calibration advantage also depended on the question wording.

Evidence: Dinzinger, Ghosh Dastidar, Granitzer · System One models Are Not Better Rerankers · arXiv 2610.32318 · 23 Oct 2026 · Source ↗

Latency, cost and calibration can change the choice

In Lucca's 231-email Italian and English comparison, logistic regression trained on about 200 labels had the highest accuracy on every field, with some statistical ties. Yet its composed actions ignored 18% of emails needing action, against 4% for the best generative model. A cost-chosen threshold reduced the classifier's misses. Per-field accuracy alone did not choose the best action policy.

Evidence: Federico Lucca · Measure Before You Train · arXiv 2610.32955 (v2) · 23 Oct 2026 · Source ↗

Measure the operating point

The banking test fitted temperature scaling on out-of-fold predictions. Its same 0.90 threshold let three open backends automate 0.2%, 21.1% and 78.9% of traffic. Vals AI reports $0.025 per 1,000 claims for Jev and median latency near 0.1 seconds in its long-document timing test. Those figures cannot establish a cost or latency ranking against the banking classifier, Voyage's embedding baseline or Coroot's rules on another workload. There is no matched Jev-versus-XGBoost result in these evidence cards.

Published decision model prices · Test on your own workload

Evidence: ankur287 · jev-open-calibration, code and data for a Towards Data Science article · Oct 2026 · Source ↗ · Evidence: Vals AI · Jev evaluation 6 Oct 2026, Mercury Decide evaluation 8 Oct 2026 · Source ↗