# Deterministic decision models and repeated answers

> Deterministic decision models need checks of repeated answers and batching. See a Jev repeat test, Finic's stated contract and what to record.

Updated 2026-11-05 (Australia/Adelaide)

Decision models can return probabilities without generating text, but that alone does not guarantee identical answers on repeated calls. Determinism depends on the model and serving contract; a stored response lets you replay a past decision without calling the model again.

## Repeated Jev requests can change the top answer

In pgdocket's dated [Jev](<https://decisionmodels.ai/models/jev>) 1.13 test, 2,000 consumer complaints went through three identical sets of requests. Only 27.5% of rows kept identical probabilities across all three runs; 95.2% kept the same top answer. The 4.8% that changed are enough to matter when reproducing an individual decision. This test does not establish the behaviour of every model or current [Jev](<https://decisionmodels.ai/models/jev>) release.

[The pgdocket project](<https://decisionmodels.ai/projects#project-pgdocket>) · [Repeated-call evidence](<https://decisionmodels.ai/#pgdocket-repeat>) · [Jev model facts](<https://decisionmodels.ai/models/jev>)

[Source](<https://github.com/ElyasEbrahim/pgdocket/blob/master/docs/phase0/FINDINGS-jev-1.13.md>)

## Batching and option order are separate checks

pgdocket reports about 16.5% of top answers changing when the same rows are rebatched. In a separate 1,700-question study, option shuffling barely moves overall [Jev](<https://decisionmodels.ai/models/jev>) accuracy, by under two points, while the set of questions it ever gets wrong grows by 27% to 29%, beyond plain repetition. Stable aggregate accuracy can therefore coexist with changing individual answers.

[Compare decision models](<https://decisionmodels.ai/compare>)

[Source](<https://github.com/ElyasEbrahim/pgdocket/blob/master/docs/phase0/FINDINGS-jev-1.13.md>) · [Source](<https://arxiv.org/abs/2610.35826>)

## A vendor can promise identical requests will repeat

Finic's [Grayson](<https://decisionmodels.ai/models/grayson>) documentation says the same request always receives the same answers. Wording, option order, data, context and key order are part of the request. It separately says changing another question can move numerical answers slightly, under one percentage point on average. That is a vendor statement; the registry has not reproduced it.

[Grayson API and pricing](<https://decisionmodels.ai/models/grayson>)

[Source](<https://docs.finic.ai/concepts/ask>)

## Store the response when a decision must be replayed

pgdocket stores versioned decisions because a fresh model call may differ from the original. Finic also tells users to retain their own request and response for audits, including its answer and request identifiers. Keep wording, option order, model identity and batching with that record, and measure repeated calls before interpreting a change as an improvement.

[What is a decision model?](<https://decisionmodels.ai/what-is-a-decision-model>) · [Decision model projects](<https://decisionmodels.ai/projects>)

[Source](<https://github.com/ElyasEbrahim/pgdocket/blob/master/docs/phase0/FINDINGS-jev-1.13.md>) · [Source](<https://docs.finic.ai/concepts/ask>)