Japanese LLM Rankings 2026 — Benchmark Analysis Report (October 22 Edition)
The bottom line first: LLM rankings as of October 22, 2026
This is the October 22, 2026 edition of our Japanese-capable LLM rankings. We will save the preamble for later and show you the latest rankings first.
Latest frontier model rankings TOP 20 (Epoch Capabilities Index)
| Rank | Model | Developer | Access | Release date (per Epoch AI) | ECI |
|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Anthropic | API | 09/22 | 168.8 |
| 2 | GPT-6 Astra | OpenAI | API | 09/03 | 168.8 |
| 3 | GPT-6.1 Sol | OpenAI | API | 09/29 | 168.4 |
| 4 | Claude Fable 5.1 | Anthropic | API | 09/01 | 166.5 |
| 5 | Gemini 4 Argon | API | 09/30 | 166.4 | |
| 6 | Claude Sonnet 5.5 | Anthropic | API | 09/28 | 166.2 |
| 7 | Claude Opus 5 | Anthropic | API | 07/24 | 163.9 |
| 8 | GPT-6 Sol | OpenAI | API | 09/22 | 163.7 |
| 9 | Claude Fable 5 | Anthropic | API | 06/09 | 163.4 |
| 10 | GPT-5.5 Pro | OpenAI | API | 04/23 | 162.9 |
| 11 | GPT-5.6 Sol | OpenAI | API | 07/09 | 162.8 |
| 12 | Muse Wonder | Meta AI | API | 10/12 | 161.8 |
| 13 | GPT-5.6 Terra | OpenAI | API | 07/09 | 160.1 |
| 14 | GPT-5.5 | OpenAI | API | 04/23 | 159.7 |
| 15 | GPT-5.4 Pro | OpenAI | API | 03/05 | 159.6 |
| 16 | Claude Haiku 5.5 | Anthropic | API | 10/07 | 159.3 |
| 17 | Claude Opus 4.8 | Anthropic | API | 05/28 | 159.0 |
| 18 | Gemini 3.7 Flash | API | 08/13 | 158.0 | |
| 19 | Kimi K3 | Moonshot | Open | 07/16 | 157.9 |
| 20 | Gemini 3.8 Flash | API | 09/02 | 157.4 |
* An index of overall capability (Epoch AI, updated October 21). How to read it, and our analysis, are in Part 1.
Measured rankings on Japanese tasks TOP 10 (Nejumi Leaderboard 4)
| Rank | Model | Category | Total Score |
|---|---|---|---|
| 1 | anthropic/claude-opus-5: adaptive-thinking-max | api | 0.8720 |
| 2 | anthropic/claude-fable-5: adaptive-thinking-max with fallback to Opus 4.8 | api | 0.8699 |
| 3 | Qwen/Qwen3.8-2.4T-A95B: reasoning-xhigh | Large (30B+) | 0.8598 |
| 4 | anthropic/claude-opus-4.8: adaptive-thinking-xhigh | api | 0.8523 |
| 5 | anthropic/claude-opus-4.7: adaptive-thinking-xhigh | api | 0.8509 |
| 6 | openai/gpt-5.6-sol: max-effort | api | 0.8499 |
| 7 | openai/gpt-5.6-terra: max-effort | api | 0.8440 |
| 8 | moonshotai/kimi-k3: reasoning-max | api | 0.8432 |
| 9 | google/gemini-3.1-pro-preview | api | 0.8430 |
| 10 | moonshotai/Kimi-K3: reasoning-enabled | Large (30B+) | 0.8425 |
* Total score on Japanese tasks (per evaluation run). Rows in light blue are open models. Second-place Claude Fable 5 was evaluated in a configuration that includes fallback to Opus 4.8. The full TOP 50 table and analysis are in Part 2.
Here are this edition's three big topics.
- Frontier update: Claude Opus 5.5 and GPT-6 Astra tie at the top.
Both score 168.8 on Epoch AI's ECI. Claude Fable 5.1, Gemini 4 Argon, and Claude Sonnet 5.5 follow in the 166 range, and none of them has been evaluated on Nejumi yet - On Nejumi (Japanese tasks), the leader Claude Opus 5 (0.8720) holds firm. The top 13 are exactly the same as last edition, and 13 of the 14 newcomers are open models
- Meta and Japan's llm-jp-4.1 debut in the Japanese rankings.
Meta's open model Muse Spark Open ranks 14th overall (0.8379), and the 33B version of llm-jp-4.1 from the National Institute of Informatics becomes the top Japanese-made open model

(We update these LLM rankings regularly. Follow our X (formerly Twitter) account to get notified of updates.)
Why this edition has two parts, and where the three rankings fit
Up to the previous September 1, 2026 edition, this series relied solely on data from Nejumi Leaderboard 4 (Weights & Biases Japan). Starting with this edition, we also draw on new ranking sources that can track models right after release.
From this edition on, Part 1 draws on independent evaluation data that tracks models right after release, and Part 2 holds the detailed Nejumi analysis.
The three sources we use from now on are as follows. Because they measure different things, we do not place the three scores side by side for comparison.
| Ranking | Operator | What it measures | Coverage | Latest update |
|---|---|---|---|---|
| Epoch Capabilities Index (ECI) | Epoch AI (U.S. research organization) | An index of overall capability that integrates more than 50 benchmarks, combining Epoch's own evaluations, independent leaderboards, and developer-reported figures | Both commercial APIs and open models | October 21 |
| Arena, Japanese category | Arena (formerly LMArena) | Ratings computed with the Bradley-Terry method from votes by people comparing answers from two models. Only Japanese-language prompts are counted | Both commercial APIs and open models | October 20 |
| Nejumi Leaderboard 4 | Weights & Biases Japan | Measures real performance on Japanese tasks across 35 columns on two axes, General Language Performance (GLP) and Alignment (ALT) | Both commercial APIs and open models | Latest run October 19 (the last entry from the American commercial API camp was July 25) |
* Rows in light blue are used in Part 1, white rows in Part 2.
- Epoch Capabilities Index (ECI). An index from the U.S. research organization Epoch AI that combines results from more than 50 benchmarks, such as GPQA Diamond and SWE-bench Verified, into a single measure of overall capability. In addition to Epoch's own evaluations, it incorporates values from independent leaderboards and developer-reported figures, so new models appear within days of release. Higher is better, and each estimate comes with a confidence interval (Epoch AI's public data)
- Arena, Japanese category. Ratings computed from users submitting the same prompt to two models and voting for the better answer. It is not a benchmark but a tally of which model people prefer in Japanese, and rankings can shift for models with few votes (Arena's public data)
- Nejumi Leaderboard 4. Measures real Japanese-task performance across 35 columns on two axes, GLP and ALT, from translation, summarization, reasoning, and coding to toxicity, bias, and truthfulness. Its unique value, which the other two lack, is showing what each model is good at and where it is risky, down to the breakdown
In addition, the independent evaluators Artificial Analysis and Vals AI also list the newest models quickly. We do not reproduce their figures in this article and provide links only.
Part 1: Latest frontier model rankings
The latest frontier by overall capability (Epoch Capabilities Index)

Here again is the TOP 20 table shown at the top of this article.
| Rank | Model | Developer | Access | Release date (per Epoch AI) | ECI |
|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Anthropic | API | 09/22 | 168.8 |
| 2 | GPT-6 Astra | OpenAI | API | 09/03 | 168.8 |
| 3 | GPT-6.1 Sol | OpenAI | API | 09/29 | 168.4 |
| 4 | Claude Fable 5.1 | Anthropic | API | 09/01 | 166.5 |
| 5 | Gemini 4 Argon | API | 09/30 | 166.4 | |
| 6 | Claude Sonnet 5.5 | Anthropic | API | 09/28 | 166.2 |
| 7 | Claude Opus 5 | Anthropic | API | 07/24 | 163.9 |
| 8 | GPT-6 Sol | OpenAI | API | 09/22 | 163.7 |
| 9 | Claude Fable 5 | Anthropic | API | 06/09 | 163.4 |
| 10 | GPT-5.5 Pro | OpenAI | API | 04/23 | 162.9 |
| 11 | GPT-5.6 Sol | OpenAI | API | 07/09 | 162.8 |
| 12 | Muse Wonder | Meta AI | API | 10/12 | 161.8 |
| 13 | GPT-5.6 Terra | OpenAI | API | 07/09 | 160.1 |
| 14 | GPT-5.5 | OpenAI | API | 04/23 | 159.7 |
| 15 | GPT-5.4 Pro | OpenAI | API | 03/05 | 159.6 |
| 16 | Claude Haiku 5.5 | Anthropic | API | 10/07 | 159.3 |
| 17 | Claude Opus 4.8 | Anthropic | API | 05/28 | 159.0 |
| 18 | Gemini 3.7 Flash | API | 08/13 | 158.0 | |
| 19 | Kimi K3 | Moonshot | Open | 07/16 | 157.9 |
| 20 | Gemini 3.8 Flash | API | 09/02 | 157.4 |
* Release dates are as stated by Epoch AI.
The ECI leaders are Claude Opus 5.5 (168.8) and GPT-6 Astra (168.8), with point estimates only about 0.02 apart. The confidence intervals published by Epoch AI (165.3 to 174.0 for Opus 5.5 and 165.5 to 174.5 for Astra) overlap heavily, so this data does not support a clear verdict on which is stronger. Third-place GPT-6.1 Sol (168.4) sits at nearly the same level.
Below them, Claude Fable 5.1 (166.5), Gemini 4 Argon (166.4), and Claude Sonnet 5.5 (166.2) cluster in the 166 range. All six of these models are absent from Nejumi.
Claude Opus 5, the Nejumi leader, ranks 7th on ECI at 163.9. In other words, six unevaluated models sit above the Nejumi leader. ECI and Nejumi measure different things, so rankings do not transfer directly, but once these models are evaluated on Nejumi, the top of the Japanese rankings could very well move.
Three other points stand out.
- Meta enters the frontier. Muse Wonder (161.8), released on October 12, is the first commercial API model from Meta to reach the ECI top 12. It is a higher-end model, distinct from the open-weight Muse Spark Open (155.1) covered in Part 2
- Small models catch up with last generation's flagships. Claude Haiku 5.5 (159.3) is on par with Claude Opus 4.8 (159.0), the flagship released in May. Claude Sonnet 5.5 (166.2) even outscores the Nejumi leader Opus 5 (163.9)
- Kimi K3 is the top open model. The highest open-weight score is Kimi K3 at 157.9 (19th), followed by GLM-5.3 (156.0), DeepSeek V4.1 Flash (155.9), and Muse Spark Open (155.1). On ECI, the gap between the open camp and the top API models is about 11 points
Because ECI also incorporates developer-reported figures, the influence of vendor claims cannot be ruled out entirely. Even so, since it combines independent leaderboard values with Epoch's own evaluations, it is currently the most practical indicator for gauging where a just-released model stands.
Human evaluation in Japanese (Arena, Japanese category)

| Rank | Model (Arena naming) | Developer | Rating | Votes |
|---|---|---|---|---|
| 1 | Claude Fable 5.1 (Max) | anthropic | 1,562 | 282 |
| 2 | Claude Opus 5.5 (High) | anthropic | 1,534 | 179 |
| 3 | Claude Opus 5 (Max) | anthropic | 1,520 | 594 |
| 4 | Claude Opus 5 (High) | anthropic | 1,516 | 1,163 |
| 5 | Gemini 3.7 Flash (High) | 1,511 | 476 | |
| 6 | Gemini 3 Pro | 1,509 | 476 | |
| 7 | Gemini 3.8 Flash (High) | 1,507 | 762 | |
| 8 | Claude Fable 5 (High) | anthropic | 1,504 | 643 |
| 9 | GPT-5.5 (High) | openai | 1,499 | 907 |
| 10 | Qwen3.5-Max-Preview | alibaba | 1,495 | 209 |
| 11 | Kimi K3 (Max) | moonshot | 1,494 | 662 |
| 12 | Gemini 3.1 Pro Preview | 1,491 | 1,582 | |
| 13 | Gemini 3 Flash | 1,489 | 340 | |
| 14 | Claude Opus 4.6 (High) | anthropic | 1,488 | 950 |
| 15 | Gemini 3.5 Flash (High) | 1,486 | 700 |
* Parentheses show the reasoning settings that Arena distinguishes.
In Arena's Japanese category, which tallies only Japanese-language prompts, Claude Fable 5.1 (1562) leads and Claude Opus 5.5 (1534) is second. The Nejumi leader Claude Opus 5 holds 3rd and 4th, so the same picture as on ECI, with Fable 5.1 and Opus 5.5 above Opus 5, appears in Japanese human evaluation as well.
That said, Fable 5.1 has only 282 votes so far, and its rating may still shift. Opus 5 (High) has 1,163 votes and Gemini 3.1 Pro Preview 1,582, a larger body of comparisons, but the reliability of any ranking must be judged not only by vote counts but also by the estimation error in the ratings.
Another point worth noting is that GPT-6 Astra, tied for first on ECI, does not appear in the Japanese-category top 15 we retrieved. The same goes for Gemini 4 Argon and Muse Wonder. This table alone cannot tell us whether they are not listed in the Japanese category, have too few votes yet, or simply rank 16th or lower, so we withhold any relative assessment in Japanese for now.
Among open models, Kimi K3 (1494) came in 11th. It is also the second open model and 10th overall on Nejumi in Part 2, and it leads the open camp in Japanese human evaluation too.
Part 1 summary: who will move the top of Nejumi next?
- On both overall capability (ECI) and Japanese human evaluation (Arena), Claude Opus 5.5 and Claude Fable 5.1 rank above the Nejumi leader Claude Opus 5. They are strong candidates to reach the top of Nejumi as well, but a change of leader on Japanese tasks will have to wait for actual evaluation results
- GPT-6 Astra is tied for first in overall capability but is not yet visible near the top in Japanese human evaluation. Its Japanese-task performance awaits Nejumi's evaluation
- Claude Sonnet 5.5 and Claude Haiku 5.5 show lower-priced models matching last generation's flagship class, making them strong candidates for cost-sensitive use cases
With that, let's move to Part 2 and take a detailed look at Japanese-task performance using the Nejumi Leaderboard 4 data.
Part 2: Rankings of Japanese-capable models (Nejumi Leaderboard 4)
Part 2 is an analysis based on the Nejumi Leaderboard 4 benchmark data (October 22, 2026 snapshot). This time the top of the board, leader included, did not move at all, while all 14 newly added models are either open models or from outside the United States. A quiet edition, but one with the feel of tectonic plates shifting!

About open-source models
Models with open weights are sometimes called "open-source models" or "OSS models." Since not all of them release their training data and training code, this article uses the term "open models" throughout.
About this benchmark analysis
This report presents trends and characteristics read from benchmark data, as reference information for LLM selection. For actual deployment, we recommend validating candidate models in your own environment for your specific use case.
Also, this edition's tally is based on data retrieved from the evaluation runs currently listed on the leaderboard (excluding archived ones). Note that the same model may appear as separate entries when evaluated under different settings such as reasoning/thinking modes.
With that, let's look at the overall rankings of Japanese-capable LLMs as of October 22, 2026.
Overall Score Rankings: TOP 50

Here is the full TOP 50 table, including the TOP 10 shown at the top of this article.
| Rank | Model | Category | Total Score |
|---|---|---|---|
| 1 | anthropic/claude-opus-5: adaptive-thinking-max | api | 0.8720 |
| 2 | anthropic/claude-fable-5: adaptive-thinking-max with fallback to Opus 4.8 | api | 0.8699 |
| 3 | Qwen/Qwen3.8-2.4T-A95B: reasoning-xhigh | Large (30B+) | 0.8598 |
| 4 | anthropic/claude-opus-4.8: adaptive-thinking-xhigh | api | 0.8523 |
| 5 | anthropic/claude-opus-4.7: adaptive-thinking-xhigh | api | 0.8509 |
| 6 | openai/gpt-5.6-sol: max-effort | api | 0.8499 |
| 7 | openai/gpt-5.6-terra: max-effort | api | 0.8440 |
| 8 | moonshotai/kimi-k3: reasoning-max | api | 0.8432 |
| 9 | google/gemini-3.1-pro-preview | api | 0.8430 |
| 10 | moonshotai/Kimi-K3: reasoning-enabled | Large (30B+) | 0.8425 |
| 11 | openai/gpt-5.5-2026-04-23: xhigh-effort | api | 0.8411 |
| 12 | openai/gpt-5.4-2026-03-05: high-effort | api | 0.8397 |
| 13 | anthropic/claude-opus-4-6: extended-thinking | api | 0.8394 |
| 14 | meta-models/Muse-Spark-Open-816B-A42B: reasoning-xhigh | Large (30B+) | 0.8379 |
| 15 | tencent/Hy4-preview: reasoning-high | Large (30B+) | 0.8344 |
| 16 | google/gemini-3.6-flash | api | 0.8312 |
| 17 | qwen/qwen3.6-max-preview: openrouter-reasoning | api | 0.8295 |
| 18 | Qwen/Qwen3.8-Flash-Next: reasoning-xhigh | Large (30B+) | 0.8291 |
| 19 | zai-org/GLM-5.3: reasoning-max | Large (30B+) | 0.8287 |
| 20 | openai/gpt-5.4-2026-03-05: xhigh-effort | api | 0.8286 |
| 21 | openai/gpt-5.2-2025-12-11: xhigh-effort | api | 0.8285 |
| 22 | zai-org/GLM-5.3-Flash: reasoning-max | Large (30B+) | 0.8260 |
| 23 | google/gemini-3.5-flash | api | 0.8249 |
| 24 | thinkingmachines/Inkling: reasoning-max | Large (30B+) | 0.8244 |
| 25 | anthropic/claude-sonnet-4.6: extended-thinking | api | 0.8230 |
| 26 | thinkingmachines/Inkling-Small: reasoning-max | Large (30B+) | 0.8222 |
| 27 | tencent/Hy3: reasoning-high | Large (30B+) | 0.8200 |
| 28 | Qwen/Qwen3.5-397B-A17B: reasoning-enabled | Large (30B+) | 0.8191 |
| 29 | deepseek-ai/DeepSeek-V4-Flash-0731: reasoning-max | Large (30B+) | 0.8169 |
| 30 | ornith-ai/Ornith-1.5-397B: reasoning-enabled | Large (30B+) | 0.8161 |
| 31 | XiaomiMiMo/MiMo-V2.6-Pro-MOPD: reasoning-enabled | Large (30B+) | 0.8159 |
| 32 | deepseek-ai/DeepSeek-V4.1-Flash: reasoning-effort-100 | Large (30B+) | 0.8158 |
| 33 | zai-org/GLM-5.2: reasoning-max | Large (30B+) | 0.8156 |
| 34 | xai/grok-4.5: reasoning-high | api | 0.8156 |
| 35 | google/gemini-3-flash-preview | api | 0.8155 |
| 36 | openai/gpt-5.6-luna: max-effort | api | 0.8145 |
| 37 | google/gemini-3-pro-preview | api | 0.8134 |
| 38 | Qwen/Qwen3.5-122B-A10B: reasoning-enabled | Large (30B+) | 0.8094 |
| 39 | Qwen/Qwen3.8-27B: reasoning-xhigh | Medium (10B-30B) | 0.8091 |
| 40 | openai/gpt-5.1-2025-11-13: high-effort | api | 0.8085 |
| 41 | deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: reasoning-max | Large (30B+) | 0.8077 |
| 42 | google/gemma-4-31b-it | Large (30B+) | 0.8077 |
| 43 | deepseek-ai/DeepSeek-V4-Pro: reasoning-max | Large (30B+) | 0.8067 |
| 44 | anthropic/claude-opus-4.5-20251125: extended-thinking | api | 0.8064 |
| 45 | MiniMaxAI/MiniMax-M3: reasoning-enabled | Large (30B+) | 0.8061 |
| 46 | Qwen/Qwen3.5-27B: reasoning-enabled | Medium (10B-30B) | 0.8049 |
| 47 | z-ai/glm-5.2: openrouter-reasoning-xhigh | api | 0.8040 |
| 48 | anthropic/claude-opus-4-1-20250805: extended-thinking | api | 0.7992 |
| 49 | deepseek-ai/DeepSeek-V4-Pro-0813: reasoning-max | Large (30B+) | 0.7984 |
| 50 | Qwen/Qwen3.6-35B-A3B: reasoning-enabled | Large (30B+) | 0.7978 |
* Rows highlighted in light blue are open models.
Overall Score Trends and Analysis
The defining feature of this edition is, without question, that the top of the board did not move. From the leader Claude Opus 5 (0.8720), second-place Claude Fable 5 (0.8699), and third-place Qwen3.8-2.4T-A95B (0.8598) down to 13th-place Claude Opus 4.6 (0.8394), the top 13 are identical to last edition in membership, order, and score.
- Five models score 0.85 or higher, the same as last edition (Opus 5 / Fable 5 / Qwen3.8 / Opus 4.8 / Opus 4.7)
- 46 models score 0.80 or higher (up from 42 last edition. Both are per-model counts, with multiple evaluation runs of the same model consolidated into its best score. The TOP 50 table counts evaluation runs, so the numbers do not match)
- Four newcomers entered the TOP 50: Muse Spark Open (14th), MiMo-V2.6-Pro (31st), DeepSeek-V4.1-Flash (32nd), and DeepSeek-V4-Flash-Vision-Exp (41st), all of them open models
- Last edition's 47th to 50th (GPT-5, the API version of DeepSeek V4 Pro, Qwen3.6-27B, and Claude Sonnet 4.5) were pushed down to 51st through 55th. Their scores are unchanged
The first edition of 2026 in which the leader stays put
In this series, the top spot has changed hands with every update so far in 2026: Gemini 3.1 Pro in the March 6 edition, Claude Opus 4.8 on July 10, and Claude Opus 5 on September 1. This is the first edition this year in which the leader holds.
The reason is simple. Among the evaluation runs added to the leaderboard over these seven weeks, not a single one came from the American commercial API camp of Anthropic, OpenAI, Google, or xAI. Of the 15 new runs (14 models), the only commercial API was Solar Pro 4 from Korea's Upstage, and the remaining 13 models were all open models.
As we saw in Part 1, GPT-6 Astra, Claude Opus 5.5, Claude Fable 5.1, Gemini 4 Argon, and others were released during this period, but none has been evaluated on Nejumi yet. This edition's rankings are therefore rankings among evaluated models only, and we note up front that the top could move once these are evaluated.
Within Nejumi, these seven weeks left the frontier ceiling untouched and only deepened the field. Of the 46 models above 0.80, 23 are commercial APIs and 23 are open models, an exact split.
The 14 newcomers

The chart above lists the 14 models added to the leaderboard since the previous edition. Here are the newcomers worth highlighting.
- Muse Spark Open 816B-A42B (14th, 0.8379). Meta's open model debuts in the overall TOP 15 and third among open models (more in the open models section)
- MiMo-V2.6-Pro (31st, 0.8159). Xiaomi's MoE with 1.02T total and 42B active parameters. It adds roughly 0.035 over the previous MiMo-V2.5-Pro (0.7812), making it Xiaomi's first model above 0.80
- DeepSeek-V4.1-Flash (32nd, 0.8158). DeepSeek's new generation. It beats its predecessor on GLP (language performance) but falls just short of the older V4-Flash-0731 (0.8169) on the total score (more in the next section)
- Solar Pro 4 (52nd, 0.7960). A commercial API from Korea's Upstage and the only API newcomer this time. It officially lists Japanese alongside English and Korean as supported languages, and came within a hair of 0.80
- llm-jp-4.1 series (102nd, 117th, and 124th). Three domestic models from Japan's National Institute of Informatics (NII) arrive at once. The 33B version (0.7206) is now the top Japanese-made open model (more in the open models section)
The GLP x ALT landscape: this edition's lesson is that a new version does not always beat the old one

The scatter plot above maps the major models on two axes: GLP (language performance) and ALT (alignment). The overall top models all sit in the upper right. In other words, they combine "smarts" with safety and controllability.
This edition's case study is the newcomer DeepSeek-V4.1-Flash. Its GLP of 0.8135 beats the 0.8002 of the older V4-Flash-0731. Mathematical reasoning at 0.9683 and abstract reasoning at 0.81 show solid gains in the reasoning categories.
But its truthfulness score of 0.497 was the lowest in the TOP 50 (the next lowest being Gemini 3 Flash Preview at 0.518). Comparing the ALT breakdown with the previous version, controllability, ethics, toxicity, and robustness all improved, and bias slipped only marginally (0.870 to 0.864), but truthfulness alone plunged from 0.760 to 0.497. Driven mainly by that one category, ALT fell from 0.8603 to 0.8373, and the total of 0.8158 landed just below the old version's 0.8169. A rare reversal in which a new generation fails to overtake its predecessor.
Following Gemini 3.6 Flash (truthfulness 0.607) last edition, low truthfulness again stood out this time in a speed- and cost-oriented model. Two cases are not enough to generalize, but if you are adopting a fast, low-cost model, we recommend checking this axis individually.
Incidentally, the reversal we reported last time, GPT-5.4's high setting (0.8397) outscoring its xhigh setting (0.8286), remains unchanged in this edition's data.
How to interpret benchmark results
The scores in this report are, in the end, results from benchmark tests. Benchmarks are a useful tool for comparing LLM performance objectively, but please keep the following in mind.
- Benchmarks evaluate models on a specific task set, so models well suited to that task mix tend to score higher
- Real-world usability and usefulness for a specific purpose cannot be fully captured by benchmark scores alone
- Some models have unique strengths that benchmarks simply do not measure
Next, let's narrow the focus to open models. Almost everything that moved this time happened in this division.
Open Model Overall Score Rankings: TOP 20

| Rank | Model | Size Class | Total Score |
|---|---|---|---|
| 1 | Qwen/Qwen3.8-2.4T-A95B: reasoning-xhigh | Large (30B+) | 0.8598 |
| 2 | moonshotai/Kimi-K3: reasoning-enabled | Large (30B+) | 0.8425 |
| 3 | meta-models/Muse-Spark-Open-816B-A42B: reasoning-xhigh | Large (30B+) | 0.8379 |
| 4 | tencent/Hy4-preview: reasoning-high | Large (30B+) | 0.8344 |
| 5 | Qwen/Qwen3.8-Flash-Next: reasoning-xhigh | Large (30B+) | 0.8291 |
| 6 | zai-org/GLM-5.3: reasoning-max | Large (30B+) | 0.8287 |
| 7 | zai-org/GLM-5.3-Flash: reasoning-max | Large (30B+) | 0.8260 |
| 8 | thinkingmachines/Inkling: reasoning-max | Large (30B+) | 0.8244 |
| 9 | thinkingmachines/Inkling-Small: reasoning-max | Large (30B+) | 0.8222 |
| 10 | tencent/Hy3: reasoning-high | Large (30B+) | 0.8200 |
| 11 | Qwen/Qwen3.5-397B-A17B: reasoning-enabled | Large (30B+) | 0.8191 |
| 12 | deepseek-ai/DeepSeek-V4-Flash-0731: reasoning-max | Large (30B+) | 0.8169 |
| 13 | ornith-ai/Ornith-1.5-397B: reasoning-enabled | Large (30B+) | 0.8161 |
| 14 | XiaomiMiMo/MiMo-V2.6-Pro-MOPD: reasoning-enabled | Large (30B+) | 0.8159 |
| 15 | deepseek-ai/DeepSeek-V4.1-Flash: reasoning-effort-100 | Large (30B+) | 0.8158 |
| 16 | zai-org/GLM-5.2: reasoning-max | Large (30B+) | 0.8156 |
| 17 | Qwen/Qwen3.5-122B-A10B: reasoning-enabled | Large (30B+) | 0.8094 |
| 18 | Qwen/Qwen3.8-27B: reasoning-xhigh | Medium (10B-30B) | 0.8091 |
| 19 | deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: reasoning-max | Large (30B+) | 0.8077 |
| 20 | google/gemma-4-31b-it | Large (30B+) | 0.8077 |
* Rankings are per model (for models with multiple evaluation runs, only the best score is used).
Open Model Score Trends and Analysis
Meta enters at third among open models: Muse Spark Open
The biggest news in this edition's open division is the arrival of Meta's Muse Spark Open 816B-A42B (0.8379). It takes third place behind the open leader Qwen3.8-2.4T-A95B (0.8598) and second-place Kimi K3 (0.8425), overtaking Tencent Hy4 (0.8344).
Released by Meta Superintelligence Labs on October 15, it is an MoE with 816B total and 42B active parameters, using 8 of 256 experts, and it includes an encoder for image input. According to Meta, it is an open-weight release of the same base model as Muse Spark 1.3, which is available via API. Following the distilled Muse Glimmer 30B (0.7754) we covered last time, a full-scale Meta model has finally landed in the Japanese rankings.
Looking at the breakdown, mathematical reasoning at 0.9633 and logical reasoning at 0.97 are strong, and its ALT (alignment) of 0.8990 is among the best in the open camp. On the other hand, coding at 0.7121 and function calling at 0.5636 lag the top models, which is why its GLP of 0.8034 fell short of Qwen3.8 and Kimi K3.
One caveat. The license is not Apache 2.0 or MIT but Meta's own Muse Model License, which attaches conditions to commercial use. Also, while the training languages are described as more than 100, Japanese is not named specifically. We recommend checking the terms before commercial use.
In terms of depth, the number of open models above 0.80 grew from 19 last edition to 23. That makes open models exactly half of the 46 models above 0.80.
Chinese newcomers: Xiaomi, DeepSeek, Nex-AGI, and inclusionAI
The rush of new open models from China continues this edition.
- MiMo-V2.6-Pro (0.8159) / MiMo-V2.6-Flash (0.7600). Xiaomi's new generation. Pro is an MoE with 1.02T total and 42B active parameters, Flash roughly 310B total and 15B active, both under the MIT license. Xiaomi describes the evaluated MOPD versions as having been on-policy distilled from multiple teacher models to curb repetitive tool calling (vendor description)
- DeepSeek-V4.1-Flash (0.8158). The new generation released on September 10. Its configuration pairs a 552B main model with 196B of Engram conditional memory, with active parameters stated as 8B during prefill and 16B during decode (MIT license). The October 17 update "1017" (0.7541) is also listed, but its evaluation run name carries no reasoning setting, so it appears to have been evaluated without reasoning
- DeepSeek-V4-Flash-Vision-Exp (0.8077). An experimental version of V4-Flash with a vision module added and further trained. It was listed on the leaderboard on September 1, but its API offering ended on September 10 when it was replaced by V4.1-Flash
- Nex-N2.5 series (Pro 0.7836 / Max 0.7711 / mini 0.7038). Apache 2.0 models from Shanghai-based Nex-AGI. Pro has 397B total and 17B active parameters (based on Qwen3.5-397B-A17B), Max 1.6T total and 49B active (based on DeepSeek-V4-Pro-Base), and mini 35B total and 3B active (based on Qwen3.5-35B-A3B). Notably, the smaller Pro outscores the larger Max
- Ling-3.0-flash-VL (0.7767). inclusionAI's multimodal version with 124B total and 5.5B active parameters under the MIT license. It scored slightly below the text-only Ling-3.0-flash (0.7924)
A new leader among Japanese-made open models: llm-jp-4.1
From Japan, the llm-jp-4.1 series from the Research and Development Center for Large Language Models at the National Institute of Informatics (NII) arrived with three models at once. They are the 33B dense version (0.7206), the 8B dense version (0.6921), and the 32B-A3B MoE version (0.6840), all under Apache 2.0 and all explicitly supporting Japanese.
Compared with the previous generation, the 33B version adds about 0.045 over llm-jp-4-33b-thinking (0.6760), and the 32B-A3B version rose from 0.6679 to 0.6840. According to NII, 4.1 is an improved post-training (SFT and DPO) release. The 33B version overtakes GPT-OSS-Swallow-120B-RL-v0.1 (0.6914) from the Swallow team at Institute of Science Tokyo to become the top Japanese-made open model.
In the breakdown, the 33B version's mathematical reasoning of 0.93 and translation of 0.9029 hold up well. Toxicity at 0.5887, however, is far below the top models (0.78 to 0.85), and abstract reasoning was extremely low at 0.14. The ALT side lags the GLP gains, so for work where factuality and safety matter, we recommend validating this axis.
Nearby moves: Solar Pro 4 from Korea's Upstage
Closer to home, Solar Pro 4 (0.7960), a commercial API model from Korea's Upstage, arrived as the only API newcomer this edition. Its parameter count is undisclosed, but the official documentation explicitly lists English, Korean, and Japanese as supported input and output languages. The large gap from its no-reasoning evaluation run (0.7207) suggests it is meant to be used with reasoning enabled.
Together with SK Telecom's A.X-K2 (0.7041) and LG AI's K-EXAONE-236B-A23B (0.7186) from last edition, the range of Korean models that officially cover Japanese is widening.
Next, let's look at mid-size models in the 10B to 30B range. The nice thing about mid-size models is that they run on GPUs that individuals can obtain relatively easily.
Mid-size Model (10B-30B) Overall Score Rankings

| Rank | Model | Total Score |
|---|---|---|
| 1 | Qwen/Qwen3.8-27B: reasoning-xhigh | 0.8091 |
| 2 | Qwen/Qwen3.5-27B: reasoning-enabled | 0.8049 |
| 3 | Qwen/Qwen3.6-27B: reasoning-enabled | 0.7955 |
| 4 | google/gemma-4-26B-A4B-it | 0.7872 |
| 5 | meta-models/Muse-Glimmer-30B: reasoning-xhigh | 0.7754 |
| 6 | Qwen/Qwen3.5-9B: reasoning-enabled | 0.7485 |
| 7 | Qwen/Qwen3-14B: reasoning-enabled | 0.7233 |
* Because the tally is based on evaluation runs currently listed on the leaderboard, some models from the previous edition are excluded this time due to archiving. The mid-size / small classification (such as Qwen3.5-9B being treated as Medium) also follows the leaderboard's size categories.
Mid-size Model Score Trends and Analysis
The mid-size category saw no newcomers in these seven weeks, and all seven models hold the same positions as last edition.
The leader is still Qwen3.8-27B (0.8091). A 27B dense model with a multimodal configuration that also accepts image and video input, released under Apache 2.0. Together with second-place Qwen3.5-27B (0.8049), the picture of Qwen's two models as the top picks for 30B-class local LLMs is unchanged.
A model of this size, which with quantization can run on a single GPU, scoring above 0.80 overall tops GPT-5 (high setting, 0.7970), released in August 2025, on the same leaderboard (required VRAM varies with precision and context length). It remains a strong, practical choice for on-prem and sensitive-data use cases.
Fourth-place Gemma 4 26B-A4B (0.7872) and fifth-place Muse-Glimmer-30B (0.7754) are still going strong. In future editions we will also be watching how the 32B-A3B version of llm-jp-4.1 is treated in this category (this time the leaderboard classifies it as Large).
Finally, let's look at small models. These compact models run on relatively inexpensive consumer GPUs, making them the prime candidates for edge and on-prem use.
Small Model (under 10B) Overall Score Rankings
| Rank | Model | Total Score |
|---|---|---|
| 1 | Qwen/Qwen3.5-4B: reasoning-enabled | 0.7352 |
| 2 | nvidia/NVIDIA-Nemotron-Nano-9B-v2-Japanese: reasoning-enabled | 0.7111 |
| 3 | llm-jp/llm-jp-4.1-8b-thinking: reasoning-high | 0.6921 |
| 4 | ornith-ai/Ornith-1.5-9B: reasoning-enabled | 0.6909 |
| 5 | google/gemma-4-E4B-it | 0.6692 |
| 6 | google/gemma-4-E2B-it | 0.6564 |
| 7 | tokyotech-llm/Qwen3-Swallow-8B-RL-v0.2: reasoning-enabled | 0.6552 |
Small Model Score Trends and Analysis
Among small models, Qwen3.5-4B (0.7352) remains in first place. Scoring above 0.73 with just 4B parameters, it outperforms Qwen3-30B-A3B-Thinking-2507 (0.7331), released in July 2025, on the same leaderboard, a symbol of how far miniaturization has come.
Second-place NVIDIA Nemotron Nano 9B v2 Japanese (0.7111) is also still going strong. As a small model further trained for Japanese, it remains one of the leading options.
This edition's newcomer is third-place llm-jp-4.1-8b-thinking (0.6921). It is the 8B dense version of the llm-jp-4.1 series introduced in the open models section, under Apache 2.0. As a Japanese-made small model that explicitly supports Japanese, it entered just ahead of Ornith-1.5-9B (0.6909). Its mathematical reasoning of 0.925 is eye-catching for a small model, while its SWE-Bench-based score of 0.0625 and toxicity of 0.5515 are major weaknesses.
Gemma 4's on-device E4B (0.6692) and E2B (0.6564), along with Qwen3-Swallow-8B-RL-v0.2 (0.6552) from the Swallow team at Institute of Science Tokyo, remain small-model staples.
Summary: a guide toward full-scale deployment
This edition analyzed three data sources: Epoch AI, Arena, and Nejumi Leaderboard 4. Here are the key takeaways.
The frontier has moved, and the Japanese rankings have stopped just short of it
There are two main points this time. On the frontier, Claude Opus 5.5 and GPT-6 Astra are tied in overall capability, and Claude Fable 5.1 leads Arena's Japanese category, and on Nejumi, which measures Japanese tasks directly, these models remain unevaluated and the top 13, led by Claude Opus 5, are unchanged from last edition.
What moved within Nejumi was the open camp: 13 of the 14 newcomers are open models, and open models above 0.80 reached 23, half of the 46. The trend we called the dawn of the open 0.85 era last time continues, now with a broader base.
- Commercial APIs: In overall capability, Anthropic (Opus 5.5 / Fable 5.1 / Sonnet 5.5) and OpenAI (GPT-6 Astra / GPT-6.1 Sol) form one leading group, joined by Google (Gemini 4 Argon) and Meta (Muse Wonder). Their Japanese evaluation on Nejumi is still to come
- Open models: On Nejumi, Meta's Muse Spark Open breaks into third behind the two leaders Qwen3.8 and Kimi K3. Chinese newcomers from Xiaomi, DeepSeek, Nex-AGI, and others keep coming
- Japanese-made models: llm-jp-4.1 arrives with three models, and the 33B version takes the top spot among domestic open models. With the Swallow RL versions and the Japanese Nemotron, Japanese-specialized options keep expanding steadily
Characteristics by model size
| Category | Candidate Models | Notes |
|---|---|---|
| Maximum performance (commercial API) | Claude Opus 5.5 / GPT-6 Astra / Claude Fable 5.1 | Top-tier in overall capability (ECI). Not yet evaluated on Nejumi, so for Japanese-language work, also validate the Nejumi leader Claude Opus 5 |
| Balancing cost and performance (commercial API) | Claude Sonnet 5.5 / Claude Haiku 5.5 / Gemini 3.8 Flash | On ECI, Sonnet 5.5 outscores Opus 5 and Haiku 5.5 matches last generation's flagship Opus 4.8. Suited to high-volume, everyday work (not yet evaluated on Nejumi) |
| Best-in-class open models | Qwen3.8-2.4T-A95B / Kimi K3 | 0.84 to 0.86 total on Nejumi, on par with top API models; can run on your own infrastructure (check the custom license terms) |
| Single-GPU-class local deployment | Qwen3.8-27B / Qwen3.5-27B | 27B class above 0.80 total; single-GPU setups feasible with quantization. The practical choice for on-prem and sensitive data |
| Small / edge | Qwen3.5-4B / Gemma 4 E4B | Qwen3.5-4B scores above 0.73 with 4B parameters. Along with Gemma 4 E4B (0.6692), a candidate for on-device and low-resource environments |
| Japanese-specialized | NVIDIA Nemotron Nano 9B v2 Japanese / llm-jp-4.1-33b-thinking | Models trained on Japanese data; for domestic operation and Japanese-language work. llm-jp-4.1 is the new leader among Japanese-made open models |
Toward full-scale LLM adoption: choosing the right model for each use case
This edition has used total scores as the main lens for positioning each model, but real deployment decisions are not that simple.
Benchmarks are a valuable reference, but in practice you need a multifaceted assessment that goes beyond accuracy to inference cost, response speed, handling of sensitive data, and integration with existing systems. Especially in a situation like this one, where the choices keep multiplying, what matters on the ground is less about picking the single highest-scoring model and more about building a setup in which several models can be used appropriately for different purposes.
To support exactly this kind of multi-model operation, we offer Bestllam , an integrated AI platform that lets you use multiple LLMs on a single platform. We can support you from model selection through integration into business workflows and operational design built around AI agents.
Beyond providing tools, we also offer BPR consulting focused on bringing AI into your business operations, accompanying you from business analysis through BPR planning, success metric design, and deployment support.
Please feel free to reach out to us.
https://qualiteg.com/contact?inquiry=consulting