Skip to content

Your model speaks your language. Does it get it right?

We test LLMs across datasets and languages, then publish the scores, prompts and answers. Including the wrong ones.

How the models did, language by language

How the models did, language by language — Mean accuracy, %
OpenAI: gpt-oss-120bHigh100.0 Baseline96.0-4.0pp vs English88.0-12.0pp vs English92.0-8.0pp vs English88.0-12.0pp vs English
OpenAI: gpt-oss-120bLow92.0 Baseline88.0-4.0pp vs English80.0-12.0pp vs English92.00.0pp vs English88.0-4.0pp vs English
inclusionAI: Ling 3.0 FlashHigh96.0 Baseline92.0-4.0pp vs English32.0-64.0pp vs English96.00.0pp vs English92.0-4.0pp vs English
inclusionAI: Ling 3.0 FlashLow96.0 Baseline92.0-4.0pp vs English36.0-60.0pp vs English88.0-8.0pp vs English92.0-4.0pp vs English
inclusionAI: Ling 3.0 FlashMedium96.0 Baseline88.0-8.0pp vs English—92.0-4.0pp vs English—
Inception: Mercury 2.5 PreviewHigh96.0 Baseline88.0-8.0pp vs English92.0-4.0pp vs English96.00.0pp vs English88.0-8.0pp vs English
Inception: Mercury 2.5 PreviewLow80.0 Baseline72.0-8.0pp vs English64.0-16.0pp vs English80.00.0pp vs English76.0-4.0pp vs English
Qwen: Qwen3.7 FlashHigh96.0 Baseline96.00.0pp vs English92.0-4.0pp vs English92.0-4.0pp vs English100.0+4.0pp vs English
Qwen: Qwen3.7 FlashLow96.0 Baseline96.00.0pp vs English84.0-12.0pp vs English92.0-4.0pp vs English100.0+4.0pp vs English
Upstage: Solar Pro 4High96.0 Baseline92.0-4.0pp vs English96.00.0pp vs English88.0-8.0pp vs English92.0-4.0pp vs English
Upstage: Solar Pro 4Low96.0 Baseline92.0-4.0pp vs English88.0-8.0pp vs English80.0-16.0pp vs English100.0+4.0pp vs English

Missing a model?Or a language?Run your own test.

We haven’t tested everything. Pick your models, then a dataset and the languages it supports. We’ll write the command. You run the experiment.

Build a runSet it up here. Run it in your terminal.