Japanese LLM Rankings 2026 — Benchmark Analysis Report (October 22 Edition)

Japanese LLM Rankings 2026 — Benchmark Analysis Report (October 22 Edition)

The bottom line first: LLM rankings as of October 22, 2026

This is the October 22, 2026 edition of our Japanese-capable LLM rankings. We will save the preamble for later and show you the latest rankings first.

Latest frontier model rankings TOP 20 (Epoch Capabilities Index)

Rank Model Developer Access Release date (per Epoch AI) ECI
1Claude Opus 5.5AnthropicAPI09/22168.8
2GPT-6 AstraOpenAIAPI09/03168.8
3GPT-6.1 SolOpenAIAPI09/29168.4
4Claude Fable 5.1AnthropicAPI09/01166.5
5Gemini 4 ArgonGoogleAPI09/30166.4
6Claude Sonnet 5.5AnthropicAPI09/28166.2
7Claude Opus 5AnthropicAPI07/24163.9
8GPT-6 SolOpenAIAPI09/22163.7
9Claude Fable 5AnthropicAPI06/09163.4
10GPT-5.5 ProOpenAIAPI04/23162.9
11GPT-5.6 SolOpenAIAPI07/09162.8
12Muse WonderMeta AIAPI10/12161.8
13GPT-5.6 TerraOpenAIAPI07/09160.1
14GPT-5.5OpenAIAPI04/23159.7
15GPT-5.4 ProOpenAIAPI03/05159.6
16Claude Haiku 5.5AnthropicAPI10/07159.3
17Claude Opus 4.8AnthropicAPI05/28159.0
18Gemini 3.7 FlashGoogleAPI08/13158.0
19Kimi K3MoonshotOpen07/16157.9
20Gemini 3.8 FlashGoogleAPI09/02157.4

* An index of overall capability (Epoch AI, updated October 21). How to read it, and our analysis, are in Part 1.

Measured rankings on Japanese tasks TOP 10 (Nejumi Leaderboard 4)

Rank Model Category Total Score
1anthropic/claude-opus-5: adaptive-thinking-maxapi0.8720
2anthropic/claude-fable-5: adaptive-thinking-max with fallback to Opus 4.8api0.8699
3Qwen/Qwen3.8-2.4T-A95B: reasoning-xhighLarge (30B+)0.8598
4anthropic/claude-opus-4.8: adaptive-thinking-xhighapi0.8523
5anthropic/claude-opus-4.7: adaptive-thinking-xhighapi0.8509
6openai/gpt-5.6-sol: max-effortapi0.8499
7openai/gpt-5.6-terra: max-effortapi0.8440
8moonshotai/kimi-k3: reasoning-maxapi0.8432
9google/gemini-3.1-pro-previewapi0.8430
10moonshotai/Kimi-K3: reasoning-enabledLarge (30B+)0.8425

* Total score on Japanese tasks (per evaluation run). Rows in light blue are open models. Second-place Claude Fable 5 was evaluated in a configuration that includes fallback to Opus 4.8. The full TOP 50 table and analysis are in Part 2.

Here are this edition's three big topics.

  • Frontier update: Claude Opus 5.5 and GPT-6 Astra tie at the top.
    Both score 168.8 on Epoch AI's ECI. Claude Fable 5.1, Gemini 4 Argon, and Claude Sonnet 5.5 follow in the 166 range, and none of them has been evaluated on Nejumi yet
  • On Nejumi (Japanese tasks), the leader Claude Opus 5 (0.8720) holds firm. The top 13 are exactly the same as last edition, and 13 of the 14 newcomers are open models
  • Meta and Japan's llm-jp-4.1 debut in the Japanese rankings.
    Meta's open model Muse Spark Open ranks 14th overall (0.8379), and the 33B version of llm-jp-4.1 from the National Institute of Informatics becomes the top Japanese-made open model
Summary image of this edition's three big topics
This edition's three big topics (chart by Qualiteg)

(We update these LLM rankings regularly. Follow our X (formerly Twitter) account to get notified of updates.)


Why this edition has two parts, and where the three rankings fit

Up to the previous September 1, 2026 edition, this series relied solely on data from Nejumi Leaderboard 4 (Weights & Biases Japan). Starting with this edition, we also draw on new ranking sources that can track models right after release.

From this edition on, Part 1 draws on independent evaluation data that tracks models right after release, and Part 2 holds the detailed Nejumi analysis.

The three sources we use from now on are as follows. Because they measure different things, we do not place the three scores side by side for comparison.

Ranking Operator What it measures Coverage Latest update
Epoch Capabilities Index (ECI)Epoch AI (U.S. research organization)An index of overall capability that integrates more than 50 benchmarks, combining Epoch's own evaluations, independent leaderboards, and developer-reported figuresBoth commercial APIs and open modelsOctober 21
Arena, Japanese categoryArena (formerly LMArena)Ratings computed with the Bradley-Terry method from votes by people comparing answers from two models. Only Japanese-language prompts are countedBoth commercial APIs and open modelsOctober 20
Nejumi Leaderboard 4Weights & Biases JapanMeasures real performance on Japanese tasks across 35 columns on two axes, General Language Performance (GLP) and Alignment (ALT)Both commercial APIs and open modelsLatest run October 19 (the last entry from the American commercial API camp was July 25)

* Rows in light blue are used in Part 1, white rows in Part 2.

  • Epoch Capabilities Index (ECI). An index from the U.S. research organization Epoch AI that combines results from more than 50 benchmarks, such as GPQA Diamond and SWE-bench Verified, into a single measure of overall capability. In addition to Epoch's own evaluations, it incorporates values from independent leaderboards and developer-reported figures, so new models appear within days of release. Higher is better, and each estimate comes with a confidence interval (Epoch AI's public data)
  • Arena, Japanese category. Ratings computed from users submitting the same prompt to two models and voting for the better answer. It is not a benchmark but a tally of which model people prefer in Japanese, and rankings can shift for models with few votes (Arena's public data)
  • Nejumi Leaderboard 4. Measures real Japanese-task performance across 35 columns on two axes, GLP and ALT, from translation, summarization, reasoning, and coding to toxicity, bias, and truthfulness. Its unique value, which the other two lack, is showing what each model is good at and where it is risky, down to the breakdown

In addition, the independent evaluators Artificial Analysis and Vals AI also list the newest models quickly. We do not reproduce their figures in this article and provide links only.


Part 1: Latest frontier model rankings

The latest frontier by overall capability (Epoch Capabilities Index)

Horizontal bar chart of the frontier model top 20 on the Epoch Capabilities Index
ECI TOP 20. Light blue = not yet on Nejumi, navy = listed on Nejumi. Horizontal lines are the confidence intervals published by Epoch AI

Here again is the TOP 20 table shown at the top of this article.

Rank Model Developer Access Release date (per Epoch AI) ECI
1Claude Opus 5.5AnthropicAPI09/22168.8
2GPT-6 AstraOpenAIAPI09/03168.8
3GPT-6.1 SolOpenAIAPI09/29168.4
4Claude Fable 5.1AnthropicAPI09/01166.5
5Gemini 4 ArgonGoogleAPI09/30166.4
6Claude Sonnet 5.5AnthropicAPI09/28166.2
7Claude Opus 5AnthropicAPI07/24163.9
8GPT-6 SolOpenAIAPI09/22163.7
9Claude Fable 5AnthropicAPI06/09163.4
10GPT-5.5 ProOpenAIAPI04/23162.9
11GPT-5.6 SolOpenAIAPI07/09162.8
12Muse WonderMeta AIAPI10/12161.8
13GPT-5.6 TerraOpenAIAPI07/09160.1
14GPT-5.5OpenAIAPI04/23159.7
15GPT-5.4 ProOpenAIAPI03/05159.6
16Claude Haiku 5.5AnthropicAPI10/07159.3
17Claude Opus 4.8AnthropicAPI05/28159.0
18Gemini 3.7 FlashGoogleAPI08/13158.0
19Kimi K3MoonshotOpen07/16157.9
20Gemini 3.8 FlashGoogleAPI09/02157.4

* Release dates are as stated by Epoch AI.

The ECI leaders are Claude Opus 5.5 (168.8) and GPT-6 Astra (168.8), with point estimates only about 0.02 apart. The confidence intervals published by Epoch AI (165.3 to 174.0 for Opus 5.5 and 165.5 to 174.5 for Astra) overlap heavily, so this data does not support a clear verdict on which is stronger. Third-place GPT-6.1 Sol (168.4) sits at nearly the same level.

Below them, Claude Fable 5.1 (166.5), Gemini 4 Argon (166.4), and Claude Sonnet 5.5 (166.2) cluster in the 166 range. All six of these models are absent from Nejumi.

Claude Opus 5, the Nejumi leader, ranks 7th on ECI at 163.9. In other words, six unevaluated models sit above the Nejumi leader. ECI and Nejumi measure different things, so rankings do not transfer directly, but once these models are evaluated on Nejumi, the top of the Japanese rankings could very well move.

Three other points stand out.

  • Meta enters the frontier. Muse Wonder (161.8), released on October 12, is the first commercial API model from Meta to reach the ECI top 12. It is a higher-end model, distinct from the open-weight Muse Spark Open (155.1) covered in Part 2
  • Small models catch up with last generation's flagships. Claude Haiku 5.5 (159.3) is on par with Claude Opus 4.8 (159.0), the flagship released in May. Claude Sonnet 5.5 (166.2) even outscores the Nejumi leader Opus 5 (163.9)
  • Kimi K3 is the top open model. The highest open-weight score is Kimi K3 at 157.9 (19th), followed by GLM-5.3 (156.0), DeepSeek V4.1 Flash (155.9), and Muse Spark Open (155.1). On ECI, the gap between the open camp and the top API models is about 11 points

Because ECI also incorporates developer-reported figures, the influence of vendor claims cannot be ruled out entirely. Even so, since it combines independent leaderboard values with Epoch's own evaluations, it is currently the most practical indicator for gauging where a just-released model stands.

Human evaluation in Japanese (Arena, Japanese category)

Horizontal bar chart of the Arena Japanese category top 15
Arena Japanese category TOP 15. Light blue = not yet on Nejumi, navy = listed on Nejumi. Numbers on the right are vote counts
Rank Model (Arena naming) Developer Rating Votes
1Claude Fable 5.1 (Max)anthropic1,562282
2Claude Opus 5.5 (High)anthropic1,534179
3Claude Opus 5 (Max)anthropic1,520594
4Claude Opus 5 (High)anthropic1,5161,163
5Gemini 3.7 Flash (High)google1,511476
6Gemini 3 Progoogle1,509476
7Gemini 3.8 Flash (High)google1,507762
8Claude Fable 5 (High)anthropic1,504643
9GPT-5.5 (High)openai1,499907
10Qwen3.5-Max-Previewalibaba1,495209
11Kimi K3 (Max)moonshot1,494662
12Gemini 3.1 Pro Previewgoogle1,4911,582
13Gemini 3 Flashgoogle1,489340
14Claude Opus 4.6 (High)anthropic1,488950
15Gemini 3.5 Flash (High)google1,486700

* Parentheses show the reasoning settings that Arena distinguishes.

In Arena's Japanese category, which tallies only Japanese-language prompts, Claude Fable 5.1 (1562) leads and Claude Opus 5.5 (1534) is second. The Nejumi leader Claude Opus 5 holds 3rd and 4th, so the same picture as on ECI, with Fable 5.1 and Opus 5.5 above Opus 5, appears in Japanese human evaluation as well.

That said, Fable 5.1 has only 282 votes so far, and its rating may still shift. Opus 5 (High) has 1,163 votes and Gemini 3.1 Pro Preview 1,582, a larger body of comparisons, but the reliability of any ranking must be judged not only by vote counts but also by the estimation error in the ratings.

Another point worth noting is that GPT-6 Astra, tied for first on ECI, does not appear in the Japanese-category top 15 we retrieved. The same goes for Gemini 4 Argon and Muse Wonder. This table alone cannot tell us whether they are not listed in the Japanese category, have too few votes yet, or simply rank 16th or lower, so we withhold any relative assessment in Japanese for now.

Among open models, Kimi K3 (1494) came in 11th. It is also the second open model and 10th overall on Nejumi in Part 2, and it leads the open camp in Japanese human evaluation too.

Part 1 summary: who will move the top of Nejumi next?

  • On both overall capability (ECI) and Japanese human evaluation (Arena), Claude Opus 5.5 and Claude Fable 5.1 rank above the Nejumi leader Claude Opus 5. They are strong candidates to reach the top of Nejumi as well, but a change of leader on Japanese tasks will have to wait for actual evaluation results
  • GPT-6 Astra is tied for first in overall capability but is not yet visible near the top in Japanese human evaluation. Its Japanese-task performance awaits Nejumi's evaluation
  • Claude Sonnet 5.5 and Claude Haiku 5.5 show lower-priced models matching last generation's flagship class, making them strong candidates for cost-sensitive use cases

With that, let's move to Part 2 and take a detailed look at Japanese-task performance using the Nejumi Leaderboard 4 data.


Part 2: Rankings of Japanese-capable models (Nejumi Leaderboard 4)

Part 2 is an analysis based on the Nejumi Leaderboard 4 benchmark data (October 22, 2026 snapshot). This time the top of the board, leader included, did not move at all, while all 14 newly added models are either open models or from outside the United States. A quiet edition, but one with the feel of tectonic plates shifting!

Trend of the top model's total score and the number of models scoring 0.80 or higher
The top score holds at 0.8720. The number of 0.80+ models (per model) rose from 42 to 46, and every addition is an open model

About open-source models

Models with open weights are sometimes called "open-source models" or "OSS models." Since not all of them release their training data and training code, this article uses the term "open models" throughout.

About this benchmark analysis

This report presents trends and characteristics read from benchmark data, as reference information for LLM selection. For actual deployment, we recommend validating candidate models in your own environment for your specific use case.

Also, this edition's tally is based on data retrieved from the evaluation runs currently listed on the leaderboard (excluding archived ones). Note that the same model may appear as separate entries when evaluated under different settings such as reasoning/thinking modes.

With that, let's look at the overall rankings of Japanese-capable LLMs as of October 22, 2026.


Overall Score Rankings: TOP 50

Horizontal bar chart of the top 15 Japanese LLM total scores
Overall TOP 15. The top 13 are the same faces as last edition, and Meta's Muse Spark Open debuts at 14th

Here is the full TOP 50 table, including the TOP 10 shown at the top of this article.

Rank Model Category Total Score
1anthropic/claude-opus-5: adaptive-thinking-maxapi0.8720
2anthropic/claude-fable-5: adaptive-thinking-max with fallback to Opus 4.8api0.8699
3Qwen/Qwen3.8-2.4T-A95B: reasoning-xhighLarge (30B+)0.8598
4anthropic/claude-opus-4.8: adaptive-thinking-xhighapi0.8523
5anthropic/claude-opus-4.7: adaptive-thinking-xhighapi0.8509
6openai/gpt-5.6-sol: max-effortapi0.8499
7openai/gpt-5.6-terra: max-effortapi0.8440
8moonshotai/kimi-k3: reasoning-maxapi0.8432
9google/gemini-3.1-pro-previewapi0.8430
10moonshotai/Kimi-K3: reasoning-enabledLarge (30B+)0.8425
11openai/gpt-5.5-2026-04-23: xhigh-effortapi0.8411
12openai/gpt-5.4-2026-03-05: high-effortapi0.8397
13anthropic/claude-opus-4-6: extended-thinkingapi0.8394
14meta-models/Muse-Spark-Open-816B-A42B: reasoning-xhighLarge (30B+)0.8379
15tencent/Hy4-preview: reasoning-highLarge (30B+)0.8344
16google/gemini-3.6-flashapi0.8312
17qwen/qwen3.6-max-preview: openrouter-reasoningapi0.8295
18Qwen/Qwen3.8-Flash-Next: reasoning-xhighLarge (30B+)0.8291
19zai-org/GLM-5.3: reasoning-maxLarge (30B+)0.8287
20openai/gpt-5.4-2026-03-05: xhigh-effortapi0.8286
21openai/gpt-5.2-2025-12-11: xhigh-effortapi0.8285
22zai-org/GLM-5.3-Flash: reasoning-maxLarge (30B+)0.8260
23google/gemini-3.5-flashapi0.8249
24thinkingmachines/Inkling: reasoning-maxLarge (30B+)0.8244
25anthropic/claude-sonnet-4.6: extended-thinkingapi0.8230
26thinkingmachines/Inkling-Small: reasoning-maxLarge (30B+)0.8222
27tencent/Hy3: reasoning-highLarge (30B+)0.8200
28Qwen/Qwen3.5-397B-A17B: reasoning-enabledLarge (30B+)0.8191
29deepseek-ai/DeepSeek-V4-Flash-0731: reasoning-maxLarge (30B+)0.8169
30ornith-ai/Ornith-1.5-397B: reasoning-enabledLarge (30B+)0.8161
31XiaomiMiMo/MiMo-V2.6-Pro-MOPD: reasoning-enabledLarge (30B+)0.8159
32deepseek-ai/DeepSeek-V4.1-Flash: reasoning-effort-100Large (30B+)0.8158
33zai-org/GLM-5.2: reasoning-maxLarge (30B+)0.8156
34xai/grok-4.5: reasoning-highapi0.8156
35google/gemini-3-flash-previewapi0.8155
36openai/gpt-5.6-luna: max-effortapi0.8145
37google/gemini-3-pro-previewapi0.8134
38Qwen/Qwen3.5-122B-A10B: reasoning-enabledLarge (30B+)0.8094
39Qwen/Qwen3.8-27B: reasoning-xhighMedium (10B-30B)0.8091
40openai/gpt-5.1-2025-11-13: high-effortapi0.8085
41deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: reasoning-maxLarge (30B+)0.8077
42google/gemma-4-31b-itLarge (30B+)0.8077
43deepseek-ai/DeepSeek-V4-Pro: reasoning-maxLarge (30B+)0.8067
44anthropic/claude-opus-4.5-20251125: extended-thinkingapi0.8064
45MiniMaxAI/MiniMax-M3: reasoning-enabledLarge (30B+)0.8061
46Qwen/Qwen3.5-27B: reasoning-enabledMedium (10B-30B)0.8049
47z-ai/glm-5.2: openrouter-reasoning-xhighapi0.8040
48anthropic/claude-opus-4-1-20250805: extended-thinkingapi0.7992
49deepseek-ai/DeepSeek-V4-Pro-0813: reasoning-maxLarge (30B+)0.7984
50Qwen/Qwen3.6-35B-A3B: reasoning-enabledLarge (30B+)0.7978

* Rows highlighted in light blue are open models.

The defining feature of this edition is, without question, that the top of the board did not move. From the leader Claude Opus 5 (0.8720), second-place Claude Fable 5 (0.8699), and third-place Qwen3.8-2.4T-A95B (0.8598) down to 13th-place Claude Opus 4.6 (0.8394), the top 13 are identical to last edition in membership, order, and score.

  • Five models score 0.85 or higher, the same as last edition (Opus 5 / Fable 5 / Qwen3.8 / Opus 4.8 / Opus 4.7)
  • 46 models score 0.80 or higher (up from 42 last edition. Both are per-model counts, with multiple evaluation runs of the same model consolidated into its best score. The TOP 50 table counts evaluation runs, so the numbers do not match)
  • Four newcomers entered the TOP 50: Muse Spark Open (14th), MiMo-V2.6-Pro (31st), DeepSeek-V4.1-Flash (32nd), and DeepSeek-V4-Flash-Vision-Exp (41st), all of them open models
  • Last edition's 47th to 50th (GPT-5, the API version of DeepSeek V4 Pro, Qwen3.6-27B, and Claude Sonnet 4.5) were pushed down to 51st through 55th. Their scores are unchanged

The first edition of 2026 in which the leader stays put

In this series, the top spot has changed hands with every update so far in 2026: Gemini 3.1 Pro in the March 6 edition, Claude Opus 4.8 on July 10, and Claude Opus 5 on September 1. This is the first edition this year in which the leader holds.

The reason is simple. Among the evaluation runs added to the leaderboard over these seven weeks, not a single one came from the American commercial API camp of Anthropic, OpenAI, Google, or xAI. Of the 15 new runs (14 models), the only commercial API was Solar Pro 4 from Korea's Upstage, and the remaining 13 models were all open models.

As we saw in Part 1, GPT-6 Astra, Claude Opus 5.5, Claude Fable 5.1, Gemini 4 Argon, and others were released during this period, but none has been evaluated on Nejumi yet. This edition's rankings are therefore rankings among evaluated models only, and we note up front that the top could move once these are evaluated.

Within Nejumi, these seven weeks left the frontier ceiling untouched and only deepened the field. Of the 46 models above 0.80, 23 are commercial APIs and 23 are open models, an exact split.

The 14 newcomers

Horizontal bar chart of the total scores of the 14 models that debuted since the previous edition
The 14 models added since the previous (September 1) edition. Solar Pro 4 is the only commercial API among them

The chart above lists the 14 models added to the leaderboard since the previous edition. Here are the newcomers worth highlighting.

  • Muse Spark Open 816B-A42B (14th, 0.8379). Meta's open model debuts in the overall TOP 15 and third among open models (more in the open models section)
  • MiMo-V2.6-Pro (31st, 0.8159). Xiaomi's MoE with 1.02T total and 42B active parameters. It adds roughly 0.035 over the previous MiMo-V2.5-Pro (0.7812), making it Xiaomi's first model above 0.80
  • DeepSeek-V4.1-Flash (32nd, 0.8158). DeepSeek's new generation. It beats its predecessor on GLP (language performance) but falls just short of the older V4-Flash-0731 (0.8169) on the total score (more in the next section)
  • Solar Pro 4 (52nd, 0.7960). A commercial API from Korea's Upstage and the only API newcomer this time. It officially lists Japanese alongside English and Korean as supported languages, and came within a hair of 0.80
  • llm-jp-4.1 series (102nd, 117th, and 124th). Three domestic models from Japan's National Institute of Informatics (NII) arrive at once. The 33B version (0.7206) is now the top Japanese-made open model (more in the open models section)

The GLP x ALT landscape: this edition's lesson is that a new version does not always beat the old one

Scatter plot of language performance (GLP) versus alignment (ALT)
Upper right is better. DeepSeek-V4.1-Flash beats its predecessor on GLP but sinks far on ALT

The scatter plot above maps the major models on two axes: GLP (language performance) and ALT (alignment). The overall top models all sit in the upper right. In other words, they combine "smarts" with safety and controllability.

This edition's case study is the newcomer DeepSeek-V4.1-Flash. Its GLP of 0.8135 beats the 0.8002 of the older V4-Flash-0731. Mathematical reasoning at 0.9683 and abstract reasoning at 0.81 show solid gains in the reasoning categories.

But its truthfulness score of 0.497 was the lowest in the TOP 50 (the next lowest being Gemini 3 Flash Preview at 0.518). Comparing the ALT breakdown with the previous version, controllability, ethics, toxicity, and robustness all improved, and bias slipped only marginally (0.870 to 0.864), but truthfulness alone plunged from 0.760 to 0.497. Driven mainly by that one category, ALT fell from 0.8603 to 0.8373, and the total of 0.8158 landed just below the old version's 0.8169. A rare reversal in which a new generation fails to overtake its predecessor.

Following Gemini 3.6 Flash (truthfulness 0.607) last edition, low truthfulness again stood out this time in a speed- and cost-oriented model. Two cases are not enough to generalize, but if you are adopting a fast, low-cost model, we recommend checking this axis individually.

Incidentally, the reversal we reported last time, GPT-5.4's high setting (0.8397) outscoring its xhigh setting (0.8286), remains unchanged in this edition's data.

How to interpret benchmark results

The scores in this report are, in the end, results from benchmark tests. Benchmarks are a useful tool for comparing LLM performance objectively, but please keep the following in mind.

  • Benchmarks evaluate models on a specific task set, so models well suited to that task mix tend to score higher
  • Real-world usability and usefulness for a specific purpose cannot be fully captured by benchmark scores alone
  • Some models have unique strengths that benchmarks simply do not measure

Next, let's narrow the focus to open models. Almost everything that moved this time happened in this division.


Open Model Overall Score Rankings: TOP 20

Horizontal bar chart of the top 15 open model total scores
Open TOP 15. Text inside the bars shows the parameter configuration (as stated on the HF model cards. GLM-5.3 and Ornith disclose total parameters only. DeepSeek-V4.1-Flash's active count is 8B for prefill / 16B for decode). The dashed line marks the gap to the API leader
Rank Model Size Class Total Score
1Qwen/Qwen3.8-2.4T-A95B: reasoning-xhighLarge (30B+)0.8598
2moonshotai/Kimi-K3: reasoning-enabledLarge (30B+)0.8425
3meta-models/Muse-Spark-Open-816B-A42B: reasoning-xhighLarge (30B+)0.8379
4tencent/Hy4-preview: reasoning-highLarge (30B+)0.8344
5Qwen/Qwen3.8-Flash-Next: reasoning-xhighLarge (30B+)0.8291
6zai-org/GLM-5.3: reasoning-maxLarge (30B+)0.8287
7zai-org/GLM-5.3-Flash: reasoning-maxLarge (30B+)0.8260
8thinkingmachines/Inkling: reasoning-maxLarge (30B+)0.8244
9thinkingmachines/Inkling-Small: reasoning-maxLarge (30B+)0.8222
10tencent/Hy3: reasoning-highLarge (30B+)0.8200
11Qwen/Qwen3.5-397B-A17B: reasoning-enabledLarge (30B+)0.8191
12deepseek-ai/DeepSeek-V4-Flash-0731: reasoning-maxLarge (30B+)0.8169
13ornith-ai/Ornith-1.5-397B: reasoning-enabledLarge (30B+)0.8161
14XiaomiMiMo/MiMo-V2.6-Pro-MOPD: reasoning-enabledLarge (30B+)0.8159
15deepseek-ai/DeepSeek-V4.1-Flash: reasoning-effort-100Large (30B+)0.8158
16zai-org/GLM-5.2: reasoning-maxLarge (30B+)0.8156
17Qwen/Qwen3.5-122B-A10B: reasoning-enabledLarge (30B+)0.8094
18Qwen/Qwen3.8-27B: reasoning-xhighMedium (10B-30B)0.8091
19deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: reasoning-maxLarge (30B+)0.8077
20google/gemma-4-31b-itLarge (30B+)0.8077

* Rankings are per model (for models with multiple evaluation runs, only the best score is used).

Meta enters at third among open models: Muse Spark Open

The biggest news in this edition's open division is the arrival of Meta's Muse Spark Open 816B-A42B (0.8379). It takes third place behind the open leader Qwen3.8-2.4T-A95B (0.8598) and second-place Kimi K3 (0.8425), overtaking Tencent Hy4 (0.8344).

Released by Meta Superintelligence Labs on October 15, it is an MoE with 816B total and 42B active parameters, using 8 of 256 experts, and it includes an encoder for image input. According to Meta, it is an open-weight release of the same base model as Muse Spark 1.3, which is available via API. Following the distilled Muse Glimmer 30B (0.7754) we covered last time, a full-scale Meta model has finally landed in the Japanese rankings.

Looking at the breakdown, mathematical reasoning at 0.9633 and logical reasoning at 0.97 are strong, and its ALT (alignment) of 0.8990 is among the best in the open camp. On the other hand, coding at 0.7121 and function calling at 0.5636 lag the top models, which is why its GLP of 0.8034 fell short of Qwen3.8 and Kimi K3.

One caveat. The license is not Apache 2.0 or MIT but Meta's own Muse Model License, which attaches conditions to commercial use. Also, while the training languages are described as more than 100, Japanese is not named specifically. We recommend checking the terms before commercial use.

In terms of depth, the number of open models above 0.80 grew from 19 last edition to 23. That makes open models exactly half of the 46 models above 0.80.

Chinese newcomers: Xiaomi, DeepSeek, Nex-AGI, and inclusionAI

The rush of new open models from China continues this edition.

  • MiMo-V2.6-Pro (0.8159) / MiMo-V2.6-Flash (0.7600). Xiaomi's new generation. Pro is an MoE with 1.02T total and 42B active parameters, Flash roughly 310B total and 15B active, both under the MIT license. Xiaomi describes the evaluated MOPD versions as having been on-policy distilled from multiple teacher models to curb repetitive tool calling (vendor description)
  • DeepSeek-V4.1-Flash (0.8158). The new generation released on September 10. Its configuration pairs a 552B main model with 196B of Engram conditional memory, with active parameters stated as 8B during prefill and 16B during decode (MIT license). The October 17 update "1017" (0.7541) is also listed, but its evaluation run name carries no reasoning setting, so it appears to have been evaluated without reasoning
  • DeepSeek-V4-Flash-Vision-Exp (0.8077). An experimental version of V4-Flash with a vision module added and further trained. It was listed on the leaderboard on September 1, but its API offering ended on September 10 when it was replaced by V4.1-Flash
  • Nex-N2.5 series (Pro 0.7836 / Max 0.7711 / mini 0.7038). Apache 2.0 models from Shanghai-based Nex-AGI. Pro has 397B total and 17B active parameters (based on Qwen3.5-397B-A17B), Max 1.6T total and 49B active (based on DeepSeek-V4-Pro-Base), and mini 35B total and 3B active (based on Qwen3.5-35B-A3B). Notably, the smaller Pro outscores the larger Max
  • Ling-3.0-flash-VL (0.7767). inclusionAI's multimodal version with 124B total and 5.5B active parameters under the MIT license. It scored slightly below the text-only Ling-3.0-flash (0.7924)

A new leader among Japanese-made open models: llm-jp-4.1

From Japan, the llm-jp-4.1 series from the Research and Development Center for Large Language Models at the National Institute of Informatics (NII) arrived with three models at once. They are the 33B dense version (0.7206), the 8B dense version (0.6921), and the 32B-A3B MoE version (0.6840), all under Apache 2.0 and all explicitly supporting Japanese.

Compared with the previous generation, the 33B version adds about 0.045 over llm-jp-4-33b-thinking (0.6760), and the 32B-A3B version rose from 0.6679 to 0.6840. According to NII, 4.1 is an improved post-training (SFT and DPO) release. The 33B version overtakes GPT-OSS-Swallow-120B-RL-v0.1 (0.6914) from the Swallow team at Institute of Science Tokyo to become the top Japanese-made open model.

In the breakdown, the 33B version's mathematical reasoning of 0.93 and translation of 0.9029 hold up well. Toxicity at 0.5887, however, is far below the top models (0.78 to 0.85), and abstract reasoning was extremely low at 0.14. The ALT side lags the GLP gains, so for work where factuality and safety matter, we recommend validating this axis.

Nearby moves: Solar Pro 4 from Korea's Upstage

Closer to home, Solar Pro 4 (0.7960), a commercial API model from Korea's Upstage, arrived as the only API newcomer this edition. Its parameter count is undisclosed, but the official documentation explicitly lists English, Korean, and Japanese as supported input and output languages. The large gap from its no-reasoning evaluation run (0.7207) suggests it is meant to be used with reasoning enabled.

Together with SK Telecom's A.X-K2 (0.7041) and LG AI's K-EXAONE-236B-A23B (0.7186) from last edition, the range of Korean models that officially cover Japanese is widening.

Next, let's look at mid-size models in the 10B to 30B range. The nice thing about mid-size models is that they run on GPUs that individuals can obtain relatively easily.


Mid-size Model (10B-30B) Overall Score Rankings

Horizontal bar charts of the mid-size and small model rankings
The local LLM battleground. Top mid-size (left) and small (right) models. The dashed line marks 0.80
Rank Model Total Score
1Qwen/Qwen3.8-27B: reasoning-xhigh0.8091
2Qwen/Qwen3.5-27B: reasoning-enabled0.8049
3Qwen/Qwen3.6-27B: reasoning-enabled0.7955
4google/gemma-4-26B-A4B-it0.7872
5meta-models/Muse-Glimmer-30B: reasoning-xhigh0.7754
6Qwen/Qwen3.5-9B: reasoning-enabled0.7485
7Qwen/Qwen3-14B: reasoning-enabled0.7233

* Because the tally is based on evaluation runs currently listed on the leaderboard, some models from the previous edition are excluded this time due to archiving. The mid-size / small classification (such as Qwen3.5-9B being treated as Medium) also follows the leaderboard's size categories.

The mid-size category saw no newcomers in these seven weeks, and all seven models hold the same positions as last edition.

The leader is still Qwen3.8-27B (0.8091). A 27B dense model with a multimodal configuration that also accepts image and video input, released under Apache 2.0. Together with second-place Qwen3.5-27B (0.8049), the picture of Qwen's two models as the top picks for 30B-class local LLMs is unchanged.

A model of this size, which with quantization can run on a single GPU, scoring above 0.80 overall tops GPT-5 (high setting, 0.7970), released in August 2025, on the same leaderboard (required VRAM varies with precision and context length). It remains a strong, practical choice for on-prem and sensitive-data use cases.

Fourth-place Gemma 4 26B-A4B (0.7872) and fifth-place Muse-Glimmer-30B (0.7754) are still going strong. In future editions we will also be watching how the 32B-A3B version of llm-jp-4.1 is treated in this category (this time the leaderboard classifies it as Large).

Finally, let's look at small models. These compact models run on relatively inexpensive consumer GPUs, making them the prime candidates for edge and on-prem use.


Small Model (under 10B) Overall Score Rankings

Rank Model Total Score
1Qwen/Qwen3.5-4B: reasoning-enabled0.7352
2nvidia/NVIDIA-Nemotron-Nano-9B-v2-Japanese: reasoning-enabled0.7111
3llm-jp/llm-jp-4.1-8b-thinking: reasoning-high0.6921
4ornith-ai/Ornith-1.5-9B: reasoning-enabled0.6909
5google/gemma-4-E4B-it0.6692
6google/gemma-4-E2B-it0.6564
7tokyotech-llm/Qwen3-Swallow-8B-RL-v0.2: reasoning-enabled0.6552

Among small models, Qwen3.5-4B (0.7352) remains in first place. Scoring above 0.73 with just 4B parameters, it outperforms Qwen3-30B-A3B-Thinking-2507 (0.7331), released in July 2025, on the same leaderboard, a symbol of how far miniaturization has come.

Second-place NVIDIA Nemotron Nano 9B v2 Japanese (0.7111) is also still going strong. As a small model further trained for Japanese, it remains one of the leading options.

This edition's newcomer is third-place llm-jp-4.1-8b-thinking (0.6921). It is the 8B dense version of the llm-jp-4.1 series introduced in the open models section, under Apache 2.0. As a Japanese-made small model that explicitly supports Japanese, it entered just ahead of Ornith-1.5-9B (0.6909). Its mathematical reasoning of 0.925 is eye-catching for a small model, while its SWE-Bench-based score of 0.0625 and toxicity of 0.5515 are major weaknesses.

Gemma 4's on-device E4B (0.6692) and E2B (0.6564), along with Qwen3-Swallow-8B-RL-v0.2 (0.6552) from the Swallow team at Institute of Science Tokyo, remain small-model staples.


Summary: a guide toward full-scale deployment

This edition analyzed three data sources: Epoch AI, Arena, and Nejumi Leaderboard 4. Here are the key takeaways.

The frontier has moved, and the Japanese rankings have stopped just short of it

There are two main points this time. On the frontier, Claude Opus 5.5 and GPT-6 Astra are tied in overall capability, and Claude Fable 5.1 leads Arena's Japanese category, and on Nejumi, which measures Japanese tasks directly, these models remain unevaluated and the top 13, led by Claude Opus 5, are unchanged from last edition.

What moved within Nejumi was the open camp: 13 of the 14 newcomers are open models, and open models above 0.80 reached 23, half of the 46. The trend we called the dawn of the open 0.85 era last time continues, now with a broader base.

  • Commercial APIs: In overall capability, Anthropic (Opus 5.5 / Fable 5.1 / Sonnet 5.5) and OpenAI (GPT-6 Astra / GPT-6.1 Sol) form one leading group, joined by Google (Gemini 4 Argon) and Meta (Muse Wonder). Their Japanese evaluation on Nejumi is still to come
  • Open models: On Nejumi, Meta's Muse Spark Open breaks into third behind the two leaders Qwen3.8 and Kimi K3. Chinese newcomers from Xiaomi, DeepSeek, Nex-AGI, and others keep coming
  • Japanese-made models: llm-jp-4.1 arrives with three models, and the 33B version takes the top spot among domestic open models. With the Swallow RL versions and the Japanese Nemotron, Japanese-specialized options keep expanding steadily

Characteristics by model size

Category Candidate Models Notes
Maximum performance (commercial API)Claude Opus 5.5 / GPT-6 Astra / Claude Fable 5.1Top-tier in overall capability (ECI). Not yet evaluated on Nejumi, so for Japanese-language work, also validate the Nejumi leader Claude Opus 5
Balancing cost and performance (commercial API)Claude Sonnet 5.5 / Claude Haiku 5.5 / Gemini 3.8 FlashOn ECI, Sonnet 5.5 outscores Opus 5 and Haiku 5.5 matches last generation's flagship Opus 4.8. Suited to high-volume, everyday work (not yet evaluated on Nejumi)
Best-in-class open modelsQwen3.8-2.4T-A95B / Kimi K30.84 to 0.86 total on Nejumi, on par with top API models; can run on your own infrastructure (check the custom license terms)
Single-GPU-class local deploymentQwen3.8-27B / Qwen3.5-27B27B class above 0.80 total; single-GPU setups feasible with quantization. The practical choice for on-prem and sensitive data
Small / edgeQwen3.5-4B / Gemma 4 E4BQwen3.5-4B scores above 0.73 with 4B parameters. Along with Gemma 4 E4B (0.6692), a candidate for on-device and low-resource environments
Japanese-specializedNVIDIA Nemotron Nano 9B v2 Japanese / llm-jp-4.1-33b-thinkingModels trained on Japanese data; for domestic operation and Japanese-language work. llm-jp-4.1 is the new leader among Japanese-made open models

Toward full-scale LLM adoption: choosing the right model for each use case

This edition has used total scores as the main lens for positioning each model, but real deployment decisions are not that simple.

Benchmarks are a valuable reference, but in practice you need a multifaceted assessment that goes beyond accuracy to inference cost, response speed, handling of sensitive data, and integration with existing systems. Especially in a situation like this one, where the choices keep multiplying, what matters on the ground is less about picking the single highest-scoring model and more about building a setup in which several models can be used appropriately for different purposes.

To support exactly this kind of multi-model operation, we offer Bestllam , an integrated AI platform that lets you use multiple LLMs on a single platform. We can support you from model selection through integration into business workflows and operational design built around AI agents.

Beyond providing tools, we also offer BPR consulting focused on bringing AI into your business operations, accompanying you from business analysis through BPR planning, success metric design, and deployment support.

Please feel free to reach out to us.
https://qualiteg.com/contact?inquiry=consulting

Read more