The best local LLM for Thai, Hindi, Arabic and other languages
Measured scores in seven languages, why quantization hurts some languages more, and how to test any model in your own language in a few minutes.
Scores per language
Accuracy (%) on Global-MMLU-Lite (the same questions translated) and INCLUDE (each country's own exams; ThaiExam for Thai). RX 9070 XT, llama.cpp Vulkan, zero-shot.
| Language | Qwen3.8-27B Q3 translated | local exams | gemma-4-26B QAT translated | local exams |
|---|---|---|---|---|
| English | 81.5 | - | 82.2 | - |
| Chinese | 76.2 | 74.7 | 73.5 | 66.5 |
| Spanish | 80.2 | 76.8 | 74.5 | 75.2 |
| Japanese | 73.5 | 87.6 | 74.5 | 81.9 |
| Arabic | 70.8 | 71.2 | 71.5 | 73.6 |
| Hindi | 69.0 | 74.3 | 69.5 | 71.0 |
| Thai | - | 67.1 | - | 65.7 |
"-" means not measured. Margins are about ±4 and ±5 points.
Lower-resource languages lose more to quantization
Going from ~3.5 bits to 2 bits cost English, Chinese and Spanish 8-9 points and Hindi, Arabic and Thai 13. KL divergence from the 8-bit model shows the same order. If you work in one of these languages, avoid 2-bit files unless nothing else fits.
Translation and math
On FLORES-101 (chrF++), gemma-4 beats Qwen3.8 in Hindi (+4.7), Arabic (+2.9), Japanese (+1.6) and Thai (+1.5); Qwen3.8 leads Chinese by 1.4. On MGSM math, gemma-4 scores 89.6 in Thai vs 87.2.
Measure your language
Global-MMLU-Lite covers 23 languages and INCLUDE 44 countries. Against any running OpenAI-compatible server:
localllm eval --langs en,ko # knowledge localllm eval --suites math --langs de,ja localllm eval --suites translate --langs sw,vi
Each language takes a few minutes. Share the result with localllm report and it appears on the
leaderboard.
Thai speakers: this site in Thai.