RESEARCH PREVIEW
Scantron Preview decision model
Scantron Preview scores proposed agent actions with fixed security questions. Its research card reports injection results and weak authorization accuracy.
By aidendle94 · Hugging Face package release: 4 Nov 2026
A research d1-2B fine-tune for proposed agent actions, followed by a 10,240-weight output-head edit. Fixed questions cover risk, review, injection and authorization. The card corrects an earlier evaluation whose question selection revealed labels. Long states are truncated at 8K tokens, and authorization remains weak.
Quick answers
What is Scantron Preview by aidendle94?
Scantron Preview: A research d1-2B fine-tune for proposed agent actions, followed by a 10,240-weight output-head edit.
Who makes Scantron Preview?
Facts
- Size
- 1.92B parameters with an untied output head
- Inputs
- JSON action, user request and surrounding context; fixed trace or command question sets
- Context
- 128K architecture context; states trained and evaluated only up to 8K tokens
- Licence
- LFM Open License v1.0
- Access
- Weights, custom Transformers code, calibration, questions and edit provenance
Claims and sources
- The maker reports 0.971 AUROC on 2,808 LlamaFirewall traces and 0.736 AUROC for review on 822 ATBench traces. Authorization balanced accuracy is 57.6% on 621 AuthorizationBench items and 32.5% on 175 Mailu actions across three classes. All traces receive the same three questions; 451 LlamaFirewall traces are truncated. These results do not establish a production safety control. vendor reportedsource
Links
Source: Card, weights and provenance →
Related projects
No related projects are listed in the registry.