Decision Models / UltraBERT Curation
OPEN
UltraBERT Curation
By Daniel van Strien · Released 28 Oct 2026
A 380M-parameter decision model for data curation: give it a web page, prompt, maths problem or library record and typed questions (topic, format, educational value, difficulty) and it returns a probability for every option. The release centres on its recipe: the data-building and training scripts are public, with source and labeller/licence metadata where known, and 516,964 of the 535,730 training rows are released (web text omitted).
Facts
- Size
- 380M parameters, UltraBERT-380M encoder plus a GLiClass scoring head
- Licence
- Apache-2.0
- Open data
- 516,964 of 535,730 training rows released, plus 1,020,000 FineWeb pages scored with it
Claims and sources
Links
Source: Hugging Face → Source: Recipe →
Related projects
No related projects are listed in the registry.