Reading Arabic handwriting
Ruq‘ah, Ta‘liq, Nastaliq, Naskh, and Maghrebi. A model that reads historical Arabic handwriting, scored on the same lines in every version.
The test
A fixed exam: the same lines and the same score, for every system and every version.
- Scripts
- Five historical Arabic scripts: Ruq‘ah, Ta‘liq, Nastaliq, Naskh, and Maghrebi.
- Unit
- A handwritten line, compared with its true text.
- Metric
- Character error rate, with a 95% interval for every system.
- Rule
- A new version is only compared on the same exam.
The result
Six systems on the same exam.
| System | Character error rate | 95% interval |
|---|---|---|
| Kindi v0.5 preview | 8.83% | 8.45–9.26 |
| HATFormer | 21.46% | 20.89–22.04 |
| sdkv2 | 37.36% | 36.81–37.93 |
| Mistral OCR v4.1 | 39.49% | 38.44–40.57 |
| Qari-OCR v0.3 | 47.93% | 47.01–48.88 |
| Tesseract | 62.83% | 62.22–63.49 |
0%10%20%30%40%50%60%70% |
The whiskers at each bar show the 95% interval.
Our upper bound, 9.26%, sits under HATFormer's lower bound, 20.89%. The intervals don't overlap.
Script by script
The gap to HATFormer, one script at a time, on a wider test of 8,136 lines that adds Oriental and modern hands.
| Item | HATFormer | Kindi v0.5 preview |
|---|---|---|
| Oriental | 32.0% | 3.7% |
| Ruq‘ah | 17.9% | 5.8% |
| Nastaliq | 20.7% | 8.2% |
| Ta‘liq | 13.4% | 8.8% |
| Naskh | 21.2% | 9.8% |
| Maghrebi | 33.9% | 10.4% |
| Modern | 41.4% | 15.3% |
Character error rate, lower is better.
The biggest gain is in Oriental: from 32.0% to 3.7%. The hardest script is still Modern, at 15.3%. On modern handwriting, three other systems we tested read better than Kindi HTR.
See the full benchmark tableHow it's built
In development, and work continues.
- A model for handwriting
- Handwriting has its own model and print has its own, so learning handwriting never weakens how print is read.
- An exam that doesn't move
- Every new version is scored on the same lines, so scores can be compared between versions.
- On our GPUs
- Training and serving run on GPUs we operate in Saudi Arabia.
Have manuscripts or handwritten records?
We'll read a sample and tell you the error rate on it before you commit to anything.