Research
The rules behind every number.
A result is only useful if the next one can be compared with it. These are the rules we keep.
A fixed exam
Each model has one test set that doesn't change between versions. If the test changes, old and new numbers aren't compared.
95% intervals
Our main reading results carry 95% intervals. When two intervals overlap, we don't call it a win.
Sets it never saw
Where we can, we test on data the model never saw in training, and say which set it did see.
Bake-offs first
Open and commercial models are tested on the same data before we build or fine-tune anything.
Handwriting on its own model
Handwriting has its own model and print has its own, so it can't degrade printed reading.
Keep the history
Each run is kept with its baseline, so a change can be traced back to what caused it.