Find the passage that answers.
An Arabic retrieval model that picks, from up to twelve passages, the one that answers the question, before an assistant replies.
Arabic search
Before an assistant answers, it has to find the right passage. Give laya-ara-rag a question and up to 12 passages, and it picks the one that answers.
We trained it on MIRACL-ar alone. It had never seen the other four sets.
Illustration: a question about which script spread in everyday writing, and eight passages about Arabic scripts. The model's first pick is the Ruq‘ah passage, which is the right one.
Illustration: we wrote the question and passages, and the scores are for display only.
| Item | Base model | laya-ara-rag |
|---|---|---|
| MIRACL-ar (trained on) | 15.3% | 58.8% |
| Mr.TyDi-ar | 17.6% | 63.2% |
| SadeemQuestion | 16.8% | 87.1% |
| MLQA-ar | 13.3% | 58.2% |
| XPQA-ar | 21.3% | 66.5% |
The base is laya-multilingual before our training. Share of questions where the first pick was the right passage.
Five public sets
9,735 questions in all. Trained on MIRACL-ar only.
Use it
The model is published on Hugging Face under the Apache-2.0 license.
| Set | Questions | Before | After |
|---|---|---|---|
| MIRACL-ar · seen in training | 2,896 | 15.3% | 58.8% |
| Mr.TyDi-ar | 2,000 | 17.6% | 63.2% |
| SadeemQuestion | 2,089 | 16.8% | 87.1% |
| MLQA-ar | 2,000 | 13.3% | 58.2% |
| XPQA-ar | 750 | 21.3% | 66.5% |