Hello!
This is the Qualiteg Product Development Team!

In the previous article, we set up a monitor-less Raspberry Pi 5 using only a Windows PC, going from inserting the SD card to an SSH connection in about 3 minutes.

Eventually, this Raspberry Pi will host a small web server that we want to reach from the internet.

But before that, there is something we need to do first.

When we actually measured it, a freshly set up Raspberry Pi OS turned out to be far more exposed than expected

.

Password authentication was still enabled with the simple throwaway password from our previous test setup, there was no firewall, 172 upgradable packages had piled up, and the device had even acquired a global IPv6 address we never intended to have.

In this article, we will

switch SSH to key-based authentication, set up UFW, fail2ban, and automatic updates, disable IPv6 — all with real commands — and then actually reboot the machine to verify that every setting survives

Full disclosure: we stumbled once during this verification.

The IPv6 we had disabled via sysctl came back after a reboot.

The culprit was the NetworkManager connection profile.

We cover that whole story in detail in the second half.

This article targets Raspberry Pi OS (64-bit, based on Debian 13 "trixie").

All work is done from a Windows PC over SSH; the Raspberry Pi never gets a monitor attached. All numbers in this article are actual measurements taken on August 14, 2026.

We measured the initial state — and it was this exposed

Before touching anything, we measured the current state. Instead of "it's probably unsafe," we confirm exactly what is open and how, and then close it.

The IPv6 part may surprise you. Even though we intended LAN-only operation, the device had received router advertisements (RA) and automatically acquired a global IPv6 address starting with 2409:.

Global IPv6 addresses do not assume NAT, so the exposure model is fundamentally different from IPv4.

Since this article's policy is to keep IPv6 out of operation, we explicitly disable it so that no unused path is left behind.

Policy: open no ports — publish through a tunnel

Here is the hardening policy we settled on.

SSH becomes key-authentication only, restricted to connections from our home LAN. As for every other inbound port, including 8080 for the web server, we open none at all。

"We want to use it from outside but won't open any ports" may sound contradictory, but publishing will happen next time through a WireCanal tunnel. With a tunnel, the Raspberry Pi only needs outbound connections — no inbound ports to open, no router port forwarding required.

In other words, this round of hardening means "close everything」、
and next time's publishing means "reach the outside while staying closed." That is the division of labor.

Step 1: Install the SSH key (the do-not-lock-yourself-out order is everything)

First, we install the key. There is one iron rule here.Verify that key-based login actually succeeds before turning off password authentication. Get the order wrong and you lock yourself out of SSH. On a monitor-less Raspberry Pi, recovering while staying headless is quite painful, so this order must never be broken.

If you do not have an SSH key yet, first run ssh-keygen -t ed25519 on the Windows side to create one.

We recommend setting the passphrase you are asked for along the way (it is your insurance if the key file ever leaks). You can get the public key with %USERPROFILE%\.ssh\id_ed25519.pub.

First, place the public key on the Raspberry Pi.

# On the Raspberry Pi (run as the pi user)
D=/home/pi/.ssh
mkdir -p "$D"; chmod 700 "$D"
cat > "$D/authorized_keys" <<'EOF'
ssh-ed25519 AAAA...(your public key)
EOF
chmod 600 "$D/authorized_keys"; chown -R pi:pi "$D"

Note that cat > overwrites the existing authorized_keys entirely. We write it this way because this is a brand-new build; on a machine that already has registered keys, append instead (>>).

If you use GitHub, https://github.com/<your-account>.keys returns the list of your public keys, so copying from there is the quickest way.

Once the key is in place, confirm from the Windows side that you can log in with the key alone.

ssh -o PasswordAuthentication=no pi@192.168.12.17

Only proceed once this goes through.

Step 2: Change the password, disable IPv6, and make SSH key-only

From here on we work with root privileges (either enter with sudo -i or prefix each command with sudo). First, change the password — from the simple test password of last time to a proper one. Even after going key-only, sudo still uses the password, so we sort it out here.

passwd pi

Next, disable IPv6 on the OS side.

cat > /etc/sysctl.d/99-disable-ipv6.conf <<'EOF'
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
EOF
sysctl --system

Then we harden SSH. Rather than editing the main config file, we add a drop-in.

cat > /etc/ssh/sshd_config.d/00-hardening.conf <<'EOF'
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
AddressFamily inet
EOF
sshd -t && systemctl restart ssh

AddressFamily inet tells sshd to listen on IPv4 only. On Ubuntu, sshd is sometimes activated via a socket (ssh.socket), which needs its own drop-in as well — but on our machine ssh was started directly as a service (ssh.socket was disabled), so the sshd_config setting alone made [::]:22 disappear (measured).

At this point, try password authentication from a separate terminal. If it gets rejected, you succeeded. Do not forget to also confirm that your existing key login still works.

Step 3: Apply the 172 pending updates, then fail2ban and automatic updates

export DEBIAN_FRONTEND=noninteractive
apt-get update -q
apt-get -y -q full-upgrade          # apply all 172 pending updates
apt-get -y -q install ufw fail2ban unattended-upgrades

fail2ban is the gatekeeper that counts failed SSH attempts and bans the offenders automatically. One thing worth knowing: our Raspberry Pi OS trixie machine has no rsyslog installed and /var/log/auth.log does not exist either.

That means sshd logs are read from journald. Debian's fail2ban package ships a setting that reads journald for sshd (sshd_backend = systemd), but many older guides assume auth.log, so we also declared the backend explicitly in our jail config.

cat > /etc/fail2ban/jail.d/sshd.local <<'EOF'
[sshd]
enabled = true
backend = systemd
maxretry = 5
bantime = 1h
EOF
systemctl enable --now fail2ban

We also set up automatic updates (unattended-upgrades).

We do not want surprise reboots in the middle of the night, so we turn off automatic reboots only. Note that the actual periodic execution is driven by APT-side timers (apt-daily.timer / apt-daily-upgrade.timer), so our verification includes confirming that those timers are enabled.

cat > /etc/apt/apt.conf.d/20auto-upgrades <<'EOF'
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
EOF
cat > /etc/apt/apt.conf.d/52unattended-upgrades-local <<'EOF'
Unattended-Upgrade::Automatic-Reboot "false";
EOF
systemctl enable --now unattended-upgrades

One interesting discovery came as a side effect of full-upgrade.

Our machine initially had /etc/sudoers.d/010_pi-nopasswd, which let sudo run without a password — but

after full-upgrade this file was gone and sudo started requiring a password
(this is what we measured in our environment, not something we verified as an official behavioral change). Since it moves in the right direction for hardening, we simply kept it.

Step 4: With UFW, add the allow rules first, then enable

The scary part of a firewall is locking yourself out. UFW is disabled right after installation, so setting the default policies does not start blocking anything by itself.

The dangerous move is running ufw enable without first adding an allow rule for SSH.

That is why we add the allow rules first and enable afterwards.

ufw default deny incoming
ufw default allow outgoing
ufw allow from 192.168.12.0/24 to any port 22 proto tcp   # match your home LAN segment
ufw --force enable

Inbound traffic is denied by default, and the only user-defined allow rule is SSH (22) from the home LAN. We deliberately do not allow port 8080 of the test web server either (next time, that is exactly what we will publish through the tunnel). After enabling, keep your current SSH session open and confirm that a brand-new SSH connection from another terminal still works.

Verification: every item, actually measured

With the configuration written, it is time to verify. Not "the config file says so, so it must be fine" — we check the actual behavior from the outside.

So far so good. The problem came next.

After a reboot, IPv6 was back

You can only prove persistence by actually rebooting. In the first reboot verification, SSH came back about 21 seconds after the reboot, and both UFW and fail2ban started automatically. And yet:

ip -6 addr show scope global
# → 2409:xx:xxxx:... is still there

The global IPv6 address we had removed was back.

The cause was NetworkManager. IPv6 was still enabled in the connection profile, so even with the sysctl disable_ipv6 applied,

the per-interface disable_ipv6 was being written back to 0 the moment the Wi-Fi connection came up (as observed in our environment).

Right after applying the setting, ip -6 addr comes back empty, so it looks like a success. But once you reboot and Wi-Fi reconnects, the setting gets rolled back. This behavior only surfaced during the first reboot verification.

To make it permanent, disable IPv6 in the NetworkManager connection profile itself.

# Check the profile name (on our machine it was called "preconfigured")
nmcli connection show --active

# Disable IPv6 on the profile side
nmcli connection modify "<the connection name from above>" ipv6.method "disabled"

With this in place we ran a second reboot verification, and this time the machine kept zero global addresses after the reboot.

The real lesson here is not about how to disable IPv6 — it is this:

"I applied the setting and checked it on the spot" guarantees nothing about persistence. Only when verification includes a reboot is the work actually done.

Our rule is that if even one verification item fails, the whole task is treated as incomplete — fix it and verify again. Thanks to that discipline, we caught the "setting I thought I disabled was silently restored after reboot" state before going public.

Differences from older guides, confirmed on this trixie machine

With the move to a Debian 13 trixie base, several assumptions from long-standing guides no longer held.

Here is a summary of what we could confirm on our machine.

Summary

Right after setup, our Raspberry Pi 5 had password authentication enabled with a simple test password, no firewall, 172 upgradable packages, and an unintended IPv6 address.

After this session's work, SSH is key-only and LAN-restricted, UFW denies inbound traffic by default with SSH from the home LAN as the only explicit allow rule, and updates covered by unattended-upgrades are applied automatically.

Everything was done over SSH; a monitor was never connected even once.

The biggest takeaway: we caught the phenomenon of NetworkManager writing back over sysctl's IPv6 disablement — through an actual reboot verification. If we had closed the task after only checking right after applying the setting, we would not have noticed the rollback until after going live.

Next time, we will finally take the web server on this Raspberry Pi and make it reachable from the internet without opening a single inbound port, using WireCanal.

See you in the next article!

Sources & References

• Raspberry Pi Documentation | Configuration (official documentation)
• fail2ban official repository (GitHub)
• UFW - Community Help Wiki (official Ubuntu wiki)
• NetworkManager official documentation (freedesktop.org)

Related articles

• Setting up a Raspberry Pi 5 with no monitor: from OS flashing to SSH using only a Windows PC
• Use your home PC's local LLM from your phone on the go: Ollama + Open WebUI + WireCanal
• ngrok vs. WireCanal: the developers compare pricing, operations, and MCP design

Hello!

This is the Qualiteg Product Development Team!

This article kicks off a series on how to control a Raspberry Pi over the internet.

As shown in the diagram below, the series goes like this:①After the headless setup (this installment), we'll harden the Raspberry Pi as a security measure in preparation for exposing it to the internet (part 2), and then make a web server running inside the Raspberry Pi directly accessible from the internet (part 3).

In this first installment, we'll cover everything from getting your hands on a Raspberry Pi to the initial setup procedure.

We'll walk through the steps we used to go from writing the OS to connecting over SSH using only a Windows 11 PC, with actual measured timings. From inserting the SD card and powering on to logging in via SSH took about 3 minutes.

What You'll Need

• Raspberry Pi 5 board — we use the 4GB model in this article.
• microSD card — 64GB this time. It gets automatically expanded to full capacity on first boot
• Heatsink for the Raspberry Pi 5 — for thermal management
• USB card reader — for writing the Raspberry Pi image to the SD card
• A Windows 11 PC
• USB-C power supply (the Raspberry Pi 5 recommends 5V/5A power, such as the official Raspberry Pi 27W USB-C Power Supply. A 5V/3A-class charger may boot depending on your configuration, but you may run into power throttling or unstable behavior)

Setting Up the Hardware

Assembly

Alright, let's get the Raspberry Pi assembled!

Here's the Raspberry Pi.

First, let's attach the heatsink for thermal management.

Before attaching the heatsink, stick the included silicone-rubber-like material onto the heat-generating parts such as the CPU chip. These are what's known as thermal pads.

OK, with all the thermal pads in place, let's attach the heatsink itself.

Fasten the heatsink from the back side with the insulated plastic screws like this, and you're done.

The heatsink is now attached.

There's a fan power connector on the upper part of the Raspberry Pi board, so plug the power cable coming from the heatsink into it.

By the way, don't insert the SD card yet. There's nothing on it, so inserting it wouldn't do anything.

Everything up to this point should only take a few minutes.

Next, let's move on to the software.

Setting Up the Raspberry Pi Without a Monitor

Let's get right into the software setup.
This time we'll set everything up using only the CLI — in other words, a headless setup.

The environment we verified this on:

- Imaging software: Raspberry Pi Imager 2.0.8
- OS written: Raspberry Pi OS 64-bit (based on Debian 13 trixie, kernel 6.18)
- Work PC: Windows 11 Pro.

Step 1: Identify the Physical Disk Number to Write To

We'll start by preparing the SD card to insert into the Raspberry Pi.

For that, we use the dedicated imaging tool, Raspberry Pi Imager 2.0.8.

Writing to an SD card happens at the physical-disk level, so first confirm the disk number.

Raspberry Pi Imager 2.0.8 has a safety mechanism that refuses to write to devices that shouldn't be targets, such as internal drives. Even so, if you mix up another USB storage device you'll wipe its data, so always verify the disk number, capacity, and BusType.

Connect the SD card to your PC with the card reader and run the following in PowerShell.

Get-Disk

Number FriendlyName           BusType SizeGB
------ ------------           ------- ------
     0 CT4000P3PSSD8          NVMe      3726
     1 Generic- SD/MMC/MS PRO USB       59.5

In our environment, the internal SSD showed up as disk 0 and the SD card via the USB card reader as disk 1. From the capacity (59.5GB) and the BusType being USB, we can tell it's the SD card.

In the steps that follow we specify the write target as \\.\PhysicalDrive1. Be sure to substitute the disk number for your own environment.

Step 2: Get Raspberry Pi Imager and the OS Image

Raspberry Pi Imager can be installed with winget.

winget install --id RaspberryPiFoundation.RaspberryPiImager --silent --accept-package-agreements --accept-source-agreements

It installs to C:\Program Files\Raspberry Pi Ltd\Imager\. The same executable used by the GUI has a CLI mode, and a scripting wrapper, rpi-imager-cli.cmd, is bundled as well.

Get the OS image from the official download site.
The URL raspios_arm64_latest always redirects to the latest 64-bit version (with desktop), which is handy to remember.

curl.exe -L --fail -o C:\qualiteg_examples\raspi\raspios.img.xz https://downloads.raspberrypi.com/raspios_arm64_latest

The size was about 1.3GB. It can be written while still xz-compressed, so there's no need to extract it yourself.

Step 3: Create the First-Boot Script firstrun.sh

This is the heart of the headless setup.

We call the helpers imager_custom and userconf that ship with Raspberry Pi OS to configure the hostname, SSH, user, Wi-Fi, keyboard layout, and timezone all at once. Name the file firstrun.sh; it can live anywhere (you'll pass its path when writing the image in the next step).

#!/bin/bash
# set +e keeps the script running to the final cleanup even if a command fails along the way.
# If a setting doesn't take effect, check the success/failure of each command individually
set +e

/usr/lib/raspberrypi-sys-mods/imager_custom set_hostname raspberrypi
/usr/lib/raspberrypi-sys-mods/imager_custom enable_ssh
/usr/lib/userconf-pi/userconf 'pi' '$6$… (hash generated with openssl passwd -6)'
/usr/lib/raspberrypi-sys-mods/imager_custom set_wlan 'your-ssid' 'your-wifi-password' 'JP'
/usr/lib/raspberrypi-sys-mods/imager_custom set_keymap 'jp'
/usr/lib/raspberrypi-sys-mods/imager_custom set_timezone 'Asia/Tokyo'

rm -f /boot/firstrun.sh /boot/firmware/firstrun.sh
sed -i 's| systemd.run.*||g' /boot/cmdline.txt /boot/firmware/cmdline.txt 2>/dev/null
exit 0

Replace the SSID and Wi-Fi password ('your-ssid' and 'your-wifi-password' in the script above) with your own.

The user's password is passed as a SHA-512 hash, not in plain text. You can generate the hash in Git Bash or similar like this.

openssl passwd -6 'your-desired-password'

The last two lines are cleanup: the script deletes itself and restores normal boot. They remove the entry that Imager plants in the boot configuration (cmdline.txt).

There is one important caveat.

The line endings of this file must be LF.

Files created in a Windows editor tend to end up as CRLF, and with CRLF the script fails on first boot. SSH never gets enabled, Wi-Fi never connects, and with no monitor you can't even see what went wrong — a rather painful situation.

One security note as well: firstrun.sh contains your Wi-Fi password in plain text. Avoid committing it to Git or placing it in shared folders, and delete any working copies left on the Windows side once they're no longer needed.

Step 4: Write to the SD Card from the CLI

Writing to a physical disk requires administrator privileges, so run it elevated from PowerShell. Approve the UAC dialog when it appears.

$imgr = 'C:\Program Files\Raspberry Pi Ltd\Imager\rpi-imager.exe'
$opts = @('--cli', '--debug', '--disable-eject',
          '--first-run-script', 'C:\qualiteg_examples\raspi\firstrun.sh',
          '--log-file',         'C:\qualiteg_examples\raspi\flash-log.txt',
          'C:\qualiteg_examples\raspi\raspios.img.xz',
          '\\.\PhysicalDrive1')
Start-Process -FilePath $imgr -ArgumentList $opts -Verb RunAs -Wait

Here's what the options we're using mean.

Read-back verification after writing is enabled by default. In our measurements, extracting the 1.3GB xz image took 53 seconds, and writing plus verification finished in about 5 minutes. With a slow SD card the log will show write-latency warnings, but Imager automatically adjusts the buffer size and keeps going, so just wait it out.

A note on progress display: with our setup, launching elevated via Start-Process -Verb RunAs, no progress appeared in the original PowerShell window. That's why we pass --log-file and check progress and results in the log file. If the log ends with succeeded, the write was successful.

Step 5: Power On, Wait 3 Minutes, and Connect via SSH

Insert the freshly written SD card into the Raspberry Pi 5.

Then simply connect the USB-C power supply and it boots.

The Raspberry Pi 5 does have a power button, but no button press is needed for a normal first boot. On first boot it runs filesystem auto-expansion, firstrun.sh, and a reboot in sequence, so wait 2–3 minutes. If the green LED on the board is blinking, the boot process is underway.

Once you've waited, connect via SSH from a Windows terminal.

ssh pi@raspberrypi.local

The name raspberrypi.local is resolved by a mechanism called mDNS. Windows 11 supports it out of the box, so no extra software is needed. In our measurements, the SSH port opened about 3 minutes after inserting the SD card and powering on, and we could log in with the user we configured.

We also checked the state after logging in. Here are excerpts from the output of uname -a, vcgencmd measure_temp, and df -h.

Linux raspberrypi 6.18.34+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.18.34-1+rpt1 (2026-06-09) aarch64 GNU/Linux
temp=34.0'C
/dev/mmcblk0p2   58G  6.6G   49G  12% /

The entire 64GB card had been expanded into the root filesystem, the CPU temperature was 34.0°C, and an IPv4 address had been obtained over Wi-Fi.

From here on, you can use it like any ordinary Linux machine.

Pitfalls and Workarounds

Here's a table of the traps we actually hit during this work.

That last plink trap is a bonus note for anyone who wants to script the whole flow up to the SSH connection. You won't hit it when typing the ssh command by hand.

Summary

A monitor-less Raspberry Pi 5 setup can be completed safely if you keep these five points in mind.

• Confirm the physical disk number with Get-Disk before writing (most important)
• Get the OS image from raspios_arm64_latest and write it while still xz-compressed
• Put your Wi-Fi, SSH, and user settings into firstrun.sh. Line endings must be LF
• Run the write with administrator privileges and check the results in the --log-file log
• Wait 3 minutes after first boot, then ssh pi@raspberrypi.local

Once you can log in, we recommend changing the password and updating packages as your first moves.

passwd
sudo apt update && sudo apt full-upgrade -y

If you find yourself wanting a desktop, enabling VNC with sudo raspi-config lets you operate it from Windows like a remote desktop.

We plan to cover that in a separate article.

See you next time!

Reference Links

• Raspberry Pi Imager (official download page)
• Raspberry Pi OS image list (official download site)
• Cloud-init on Raspberry Pi OS (official blog, on the migration from the firstrun.sh approach to cloud-init)
• PuTTY official site (plink download)

Hello!

The first wall you hit when building a LINE Bot is not the code — it is the webhook.

LINE's Messaging API delivers messages that users send to your bot as HTTPS POST requests from LINE's servers to your server.

For this receiving endpoint (the Webhook URL), you can only specifyan HTTPS URL reachable from the internet, so you cannot simply enter the http://localhost:5000 you use during development.

The standard workaround is to issue a temporary URL with a tunneling tool, but that brings its own problem. Depending on the tunneling method, a random URL is issued every time you start it. In that case, every time you resume development you have to open the LINE Developers console and rewrite the Webhook URL. Every time you fix your code, every time you pick things up the next day — you start by pasting in a new URL. It wears on you.

To state the conclusion up front:

give your local development server a single public URL that never changes, and configuring the Webhook URL becomes a one-time task.

This article covers the entire journey: creating a LINE Official Account, configuring the Messaging API, implementing a webhook receiver in Python, exposing it on a fixed URL, and confirming the bot replies on an actual smartphone.

Every step, command, and screen shown here was actually run and verified by the author
(verified on August 11, 2026; LINE's screens may change over time).

The bot itself is a minimal echo bot that returns whatever message it receives. The goal of this article is to build out the full path: a webhook from LINE reaches Python running locally, and a reply appears on your phone.

Step 1: Create a LINE Official Account (Current Procedure as of August 2026)

The first thing to know is that

Messaging API channels can no longer be created directly from the LINE Developers console

This screen currently displays the guidance
"After creating a LINE Official Account, enable the Messaging API in LINE Official Account Manager"
.

The current flow is as follows.

1. Create a LINE Business ID (LY Corporation Business ID). It can be created with just an email address; no LINE app account is required
2. Register as a developer on the LINE Developers console and create a provider
3. Create a LINE Official Account. During creation you will be asked to connect to a Business Manager organization; if you do not have one, create it on the spot (if you leave the organization name blank, it defaults to the account name)
4. Enable the Messaging API in LINE Official Account Manager

A provider represents the individual, company, or organization offering the bot. When combined with features such as LINE Login, it is also shown on the consent screen, so it is safest to use a name that lets users identify the source, such as your company or service name
(the author created one named "Qualiteg").

The Official Account creation form asks for the account name (the bot name shown in the friends list and chat screen), company/business name, industry, and so on. One thing to note here:creating a new Official Account requires phone number verification (SMS or voice call). According to the on-screen notice, the phone number used for verification is never disclosed to your friends.

Leaving the account type as an "unverified account" is fine. A verified account (the green verification badge) has benefits such as appearing in LINE's in-app search, but a development bot does not need it.

Step 2: Enable the Messaging API and Obtain the Three Keys

Once the Official Account exists, open LINE Official Account Manager and, under Settings, choose "Messaging API" and click "Use Messaging API." You will be asked to select a provider here, andonce a provider is linked, it cannot be changed later. Select the provider you created in Step 1.

Once enabled, this screen displays the Channel ID and Channel secret.

Three "keys" are needed for bot development.

Only the channel access token requires an issuing step in the Developers console. Pass the issued token and the Channel secret via environment variables rather than hard-coding them (we will use them in the next step).

Step 3: Write a Minimal Bot That Receives the Webhook

We will write the server in Python, using the official SDK (line-bot-sdk) and Flask.

pip install line-bot-sdk flask

In the author's environment, this installed line-bot-sdk 3.25.0 and flask 3.1.3 (Python 3.11).

app.py(full source)

import os

from flask import Flask, request, abort
from linebot.v3 import WebhookHandler
from linebot.v3.exceptions import InvalidSignatureError
from linebot.v3.messaging import (
    ApiClient,
    Configuration,
    MessagingApi,
    ReplyMessageRequest,
    TextMessage,
)
from linebot.v3.webhooks import MessageEvent, TextMessageContent

app = Flask(__name__)

configuration = Configuration(access_token=os.environ["LINE_CHANNEL_ACCESS_TOKEN"])
handler = WebhookHandler(os.environ["LINE_CHANNEL_SECRET"])


@app.route("/health", methods=["GET"])
def health():
    return "OK"


@app.route("/callback", methods=["POST"])
def callback():
    signature = request.headers.get("X-Line-Signature", "")
    body = request.get_data(as_text=True)
    try:
        handler.handle(body, signature)
    except InvalidSignatureError:
        abort(400)
    return "OK"


@handler.add(MessageEvent, message=TextMessageContent)
def handle_message(event):
    reply = f"Received your message: \"{event.message.text}\""
    with ApiClient(configuration) as api_client:
        MessagingApi(api_client).reply_message(
            ReplyMessageRequest(
                reply_token=event.reply_token,
                messages=[TextMessage(text=reply)],
            )
        )


if __name__ == "__main__":
    app.run(host="127.0.0.1", port=5000)

The code does only three things./callback receives the POST from LINE, X-Line-Signature is verified against the Channel secret (this verification is built into the SDK, so there is no need to implement it yourself), and when a text message arrives, the bot replies with "Received your message: ...".

Note that this code is a minimal example for local development. For production, LINE's official documentation recommends returning the webhook response (200) promptly and processing events asynchronously.

Put the keys from Step 2 into environment variables and start the server.

$env:LINE_CHANNEL_SECRET = '(Channel secret)'
$env:LINE_CHANNEL_ACCESS_TOKEN = '(channel access token)'
python app.py

http://localhost:5000/health returns OK, the server is up. At this point everything is still confined to localhost, and nothing from LINE can reach it yet.

Step 4: Give localhost a Public URL That Never Changes

This is the heart of the matter: attaching an HTTPS URL to localhost:5000 that survives restarts.

In this article we use WireCanal, a service we develop and operate ourselves (so please read this part with that in mind).

That said, using a fixed URL is not specific to WireCanal — the same approach works with the fixed-domain features of other tunneling services.

Create a canal (the tunnel's public endpoint) in the dashboard. Choose HTTP as the type, specify localhost:5000 as the forwarding target, and a single public URL is issued.

This URL is issued with a random string, but it does not change each time you start up. Reboot your PC, stop the agent and reconnect the next day — you keep receiving on the same URL. To be precise about the plan differences: on the free plan the public URL remains valid "as long as it is in use" (it expires 72 hours after the agent's last connection), so if you develop daily, the URL stays the same. If you want a persistent URL that survives even a pause of three days or more, you need the Lite plan or above (the author tested on the Lite plan; the "Reserved (persistent)" label in the screenshot reflects that).

Next, install and start the agent on your PC. As described in the setup guide (wirecanal.com), it is a single line in PowerShell.

irm https://download.wirecanal.com/install.ps1 | iex

Download the connection settings file (wirecanal.json) from the dashboard, place it in the same folder as the agent, and start it.

.\wirecanal.exe -config wirecanal.json

When the dashboard shows "Connected!", access to the public URL is reaching localhost:5000.

Let us verify from the outside.

> curl https://7pqvyoih.ja100.wirecanal.com/health
OK

The Flask server from Step 3 responded across the internet. No inbound port forwarding and no VPN are involved (the agent communicates using outbound connections only).

Step 5: Configure and Verify the Webhook URL

Return to the LINE Developers console and set the Webhook URL on the channel's "Messaging API settings" tab. It is your public URL with the path handled in app.py, /callback, appended.

https://(your-public-URL)/callback

After setting it, be sure toturn on "Use webhook" just below. Entering the URL alone is not enough for webhooks to be delivered.

Pressing the "Verify" button sends an actual test connection from LINE's servers to this URL. If it reports "Success," the path from LINE through the tunnel to your local Flask server is working.

A Stumbling Block: Duplicate Replies

At this point everything should work — yet when the author actually sent a message from a phone, something odd happened. Before the bot's reply, an unfamiliar canned message arrived.

"Thank you for your message! Unfortunately, this account cannot respond to individual inquiries."

The culprit is theresponse messages (auto-reply) featurebuilt into Official Accounts from the start. On the author's newly created account, a blanket auto-reply named "Default" was enabled. It operates independently of the webhook, so users were receiving both the bot's reply and the canned message.

If you see duplicate replies, check the state of response messages in Official Account Manager. There are two ways to stop them: disable the individual message in the response message list, orturn off "Response messages" for the whole feature under "Settings" → "Response settings". You can confirm whether they are currently being sent via "Current status" at the top of the response message list (if it says the feature is in use and response messages are being sent, they will still go out). The author failed to notice this status was still "in use" and wasted one round of testing.

Trying It from a Smartphone

Scan the friend-add QR code (found under "Add friends guide" in Official Account Manager) with LINE on your phone, add the bot as a friend, and send it a message.

Send "こんにちは" (hello), and the reply comes back immediately: Received your message: "こんにちは". The server-side log also records the POST from LINE.

127.0.0.1 - - [11/Aug/2026 00:24:32] "POST /callback HTTP/1.1" 200 -

With this, the entire path is connected: LINE on your phone → the LINE platform → the fixed public URL → Flask on your local PC → the reply.

Summary: No More URL Rewriting

The biggest benefit of this setup is that resuming development the next day comes down to starting the agent and app.py. Because the public URL never changes, there is simply no step where you reopen the LINE Developers console and paste in a new Webhook URL. After a code change, restarting app.py is all it takes to test on the same URL.

Finally, here are the key points of this setup.

See you next time.

Sources and References

• Get started with the Messaging API (LINE Developers official documentation)
• Messaging API overview (LINE Developers official documentation)
• line-bot-sdk-python (official LINE SDK, GitHub)
• WireCanal pricing plans
• WireCanal setup guide (Windows)

Related Articles

• What Sets ngrok and WireCanal Apart? The Developers Compare Pricing, Operations, and MCP Design
• Use a Local LLM on Your Home PC from Your Phone While Out: Ollama + Open WebUI + WireCanal

Hello!

If you run a local LLM on a GPU-equipped PC at home, sooner or later a moment arrives when you are out and think, "I'd like to ask my own model about this."

Your home PC carries models you chose or tuned yourself, and your conversation history can stay on your own machine.

Having built that environment, it is only natural to want to use it from your phone as well.

At the same time, opening a port on your home router and exposing it to the outside is an uncomfortable prospect.

To state the conclusion up front:

with a single Windows PC, you can get to a state where you chat in Japanese with your home LLM from your phone while out

. No Docker, no WSL, and no port forwarding involved.

This article walks through the entire procedure: running a local LLM with Ollama and gemma4, adding a chat interface with Open WebUI, exposing it through a tunnel, and using it from an actual smartphone.

Every step, command, and screen shown here was actually run and verified by the author. The test machine is Windows 11 Pro with a GeForce RTX 3090 Ti (24GB), but with an appropriate model choice the same setup works on a 12GB-class GPU.

Note that this article covers using your home PC's LLM from outside, by yourself, for yourself.

Step 1: Install Ollama and Run gemma4 on the GPU

First, install Ollama, the runtime that runs the LLM.

From the official Ollama site, simply download and run OllamaSetup.exe. Walking through the wizard completes the install (the author used the silent install OllamaSetup.exe /VERYSILENT; the version verified was v0.32.6).

The GPU is detected automatically, so no manual CUDA setup is required.

One thing worth deciding in advance is where models are stored. By default, models accumulate in your user folder on the C drive. A single model can easily consume around 10GB, so if you want to move them to a larger drive, specify the location with a system environment variable.
The author used the D drive as the storage location.

OLLAMA_MODELS = D:\qualiteg_examples\ollama-models

For the model, we chose gemma4 (a 9.6GB download) from the top of Ollama's library rankings — its Japanese is natural and it fits within our GPU.

ollama pull gemma4

Once the download completes, verify it works. Here we ask, in Japanese, "What is the capital of Japan?" and it answers "Tokyo."

> ollama run gemma4 "日本の首都は"
東京（とうきょう）です。

Whether the model is loaded on the GPU can be checked with ollama ps.

NAME             ID              SIZE      PROCESSOR    CONTEXT
gemma4:latest    c6eb396dbd59    3.4 GB    100% GPU     32768

If PROCESSOR shows 100% GPU, you are in good shape. If it has fallen back to the CPU, responses become orders of magnitude slower, so be sure to check this. Incidentally, while the download is 9.6GB, on the author's Ollama 0.32.6 the SIZE column in ps showed 3.4GB (reported here as measured, without adjustment).

Step 2: Eliminate the "No Reply the First Time" Problem Up Front

This was the most alarming moment during the build.

Ask a question after leaving the machine idle for a while, and the screen stays silent for tens of seconds. Suspecting something was broken, we checked the logs — the inference itself had finished in 2.1 seconds.

The culprit is Ollama's default behavior.After 5 minutes of inactivity, it unloads the model from the GPU.The moment the next question arrives it reloads all 9.6GB, and during that time the system appears unresponsive.

When using it from a phone while out, the typical pattern is "open it occasionally and ask one question," which means hitting this reload every time. Keep the model resident via a system environment variable.

OLLAMA_KEEP_ALIVE = -1

After setting it, restart Ollama and check ollama ps — the UNTIL column will show Forever.

NAME             ID              SIZE      PROCESSOR    CONTEXT    UNTIL
gemma4:latest    c6eb396dbd59    3.4 GB    100% GPU     32768      Forever

Now the very first question gets an immediate answer no matter how long the machine sits idle. If the PC's GPU is also used for gaming or training, look for a compromise with a value like 24h instead of -1.

Step 3: Open WebUI Installs with pip (No Docker Needed)

For the chat interface we usedOpen WebUI.

It is a well-established OSS project with a ChatGPT-like UI that discovers Ollama automatically.

Open WebUIofficially supports installation via pip.

Open WebUI supports Python 3.11 and 3.12; 3.13 is not supported
(the official recommendation is 3.11; the author created a 3.11.15 environment with conda, though venv works just as well).

pip install open-webui

With its many dependencies, this takes a few minutes. Once installed, start it with two environment variables (the version verified was v0.11.0).

$env:OLLAMA_BASE_URL = 'http://localhost:11434'
$env:DATA_DIR = 'D:\qualiteg_examples\open-webui-data'
open-webui serve --port 3000

OLLAMA_BASE_URL points to the Ollama instance from Step 1, and DATA_DIR is where user data and chat history are stored.

In the author's environment, about 35 seconds after startup http://localhost:3000/health returned {"status":true}.

Open http://localhost:3000 in a browser and you will be prompted to create the first account.The first account created here becomes the administrator.Publishing comes later, but make it a firm habit: complete the administrator registration before exposing anything. If you publish first, any third party who happens to hit the URL could become the administrator.

Step 4: Expose It Externally Without Opening an Inbound Port

Here is the heart of the matter: making localhost:3000 reachable from your phone while you are out.

There are several ways to expose it.

This time we use the tunnel-based approach.

All communication happens over outbound connections established from the home side to the tunnel provider, sonot a single inbound port at home is opened.

In this article we use WireCanal, a service we develop and operate ourselves. For the use case in this article, the free plan is more than sufficient, and no card registration is required.

There are only three steps.

First, inthe WireCanal dashboard, create a canal (the tunnel's public endpoint). Choose HTTP as the type, specify localhost:3000 as the forwarding target, and a single public URL is issued.

Next, install the Agent on your home PC.As described in the setup guide (wirecanal.com), it is a single line in PowerShell (the Agent version verified was v0.18.0).

irm https://download.wirecanal.com/install.ps1 | iex

Download wirecanal.json (the connection settings) from the dashboard and place it in the same folder as the Agent, and you are ready.

Finally, register the Agent as a Windows service. It works when started manually as well, but as a service it recovers on its own after a PC reboot.

.\wirecanal.exe service install -config wirecanal.json

wirecanal service install: service "wirecanal-agent" installed (auto start; restarts 5s after a failure)
wirecanal service install: service started.

Now, accessing the public URL brings up the login screen of the Open WebUI running at home. Run through the full flow once in a PC browser (log in → ask a question → get a response) to confirm it works before taking it outside.

Step 5: Use It from Your Phone

All that remains is to open the public URL on your phone while out.

As a convenience, you can display a QR code for the canal's URL on the canal detail screen beforehand and scan it with your phone, saving you from typing the URL

Log in with the account created in Step 3, and the same chat screen you saw on your home PC appears as is.

Over a mobile connection, gemma4 running on the RTX 3090 Ti at home answers in Japanese.

The model itself and Open WebUI's conversation history are stored on your home PC (the communication with your phone itself passes through the tunnel's relay server; the public URL is HTTPS).

With that, you can access your own dedicated LLM from wherever you are.

How to Think About Safety When Exposing to the Internet

In this setup, Open WebUI's own login acts as the gatekeeper — as noted in Step 3, complete the administrator registration before publishing. WireCanal also provides access protections such as IP, country, and time-of-day restrictions as standard, so combine them as your use case requires.

A canal can be paused and resumed at any time from a phone browser, so it is also easy to open the line only when you use it and keep the canal stopped otherwise.

Please read this article as one example of a setup for using your machine from outside without opening ports.

Making It Persistent So It Is Always Available

Finally, we make the setup survive PC reboots and re-logons. The Agent was already registered as a service in Step 4, so what remains is Ollama and Open WebUI. Register them as "at startup" tasks in Task Scheduler (run PowerShell as administrator).

$action   = New-ScheduledTaskAction -Execute 'powershell.exe' `
  -Argument '-NoProfile -ExecutionPolicy Bypass -File D:\qualiteg_examples\open-webui\run-open-webui.ps1'
$trigger  = New-ScheduledTaskTrigger -AtStartup
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit ([TimeSpan]::Zero) `
  -RestartCount 5 -RestartInterval (New-TimeSpan -Minutes 1)
Register-ScheduledTask -TaskName 'openwebui-serve' -Action $action -Trigger $trigger `
  -User 'SYSTEM' -RunLevel Highest -Settings $settings

The contents of run-open-webui.ps1 are almost the same as the startup command in Step 3, with one important difference. A task running as SYSTEM does not go through conda activate, soopen-webui must be specified as the absolute path to the executable inside the environment.

$env:OLLAMA_BASE_URL = 'http://localhost:11434'
$env:DATA_DIR = 'D:\qualiteg_examples\open-webui-data'
& 'C:\tools\Anaconda3\envs\openwebui\Scripts\open-webui.exe' serve --port 3000

The key points are disabling the execution time limit (the default cuts tasks off at 72 hours) and running as SYSTEM, decoupled from logons. Create a task of the same shape for Ollama, ollama serve. Specify ollama.exe by absolute path here as well (in the author's environment, C:\Users\<your-username>\AppData\Local\Programs\Ollama\ollama.exe). The regular installer version also sets up auto-start at logon, so disable that auto-start to avoid running two instances.

One more caveat: if a PowerShell script launched from a SYSTEM task contains Japanese comments, it may fail to run due to an encoding mismatch (UTF-8 without BOM being read as ANSI). Write scripts in ASCII only, or save them as UTF-8 with BOM. This cost us another 30 minutes.

Summary: Your Home GPU, Available from Your Pocket

Listing what we did: one installer, one pip install, a tunnel configuration, and a few environment variables. Without any elaborate machinery, your home GPU machine becomes your own dedicated LLM server.

Finally, here are the key points of this setup.

See you next time.

Sources and References

• Ollama official site (download)
• Ollama FAQ (keep_alive behavior)
• Open WebUI official documentation (pip installation and Python requirements)
• WireCanal setup guide (Windows)

Related Articles

• How to Build an MCP Server — Writing Your Own in Python with FastMCP, Until AI Answers Database Questions in Japanese (Part 1)
• How to Build an MCP Server — Using Your Own MCP Server from Web-Based ChatGPT and Claude (Part 2)

Hello! This is the Qualiteg Product Development Team!

In Part 1, we built our own MCP server with Python and FastMCP, getting to the point where we could query a sales database in Japanese from Claude Code on our local machine. In this Part 2, we continue as promised.

We will make this MCP server, running locally on localhost, usable from the web versions of ChatGPT and Claude.

To state the conclusion first: without changing a single line of server code, the question "show me the top 3 occupations by sales" now works from both browser-based ChatGPT and Claude. The numbers returned match the measured values from Part 1 exactly. The three walls — reachability, HTTPS, and OAuth authentication — are crossed with our own WireCanal. As disclosed at the end of Part 1, it is our own service, but every step in this article uses only features available on the free plan. No credit card registration is required.

If you have not read Part 1, you can find it here.

How to Build an MCP Server (Part 1) | Connecting a Sales DB to AI with Python and FastMCP (this blog)

Why the Web Versions Cannot Use It While It Stays on localhost

When the server from Part 1 is started in HTTP mode, http://127.0.0.1:9904/mcp becomes the MCP endpoint. The CLI version of Claude Code lives on the same PC, so this URL is reachable for it.

The web versions are an entirely different story. For both ChatGPT and Claude, the connection requests to a connector are sent bytheir servers — not by your browser. That gives us three walls.

The first is reachability. On a typical home or office network, your PC sits behind NAT and a firewall and cannot be reached from outside as is. The second is HTTPS. URLs registered with a connector are expected to be https://. The third is authentication. Making the server reachable from outside means it is reachable by anyone in the world, so running a server connected to an internal database without authentication is simply not an option.

Of these, the third is the most formidable.

Both ChatGPT and Claude can, in some configurations, connect to a remote MCP server without authentication. But since exposing a server that reaches an internal database unauthenticated is not an option, we protect it with OAuth in line with the MCP Authorization specification. That means returning 401 to unauthenticated access, publishing authorization server information, handling client information, and providing the authorization screen, token issuance, and per-request verification. Implementing all of this yourself is a far larger job than the tool itself (the 200 lines we wrote in Part 1). This is where people tend to run out of steam and settle for "forget the web versions, we'll just use the CLI" — such was the reality of remote MCP.

WireCanal Takes On All Three at Once

So in Part 2 we use WireCanal, the secure tunnel service we provide.

The mechanism is shown in Figure 3. Run a small program called the Agent on your PC, and the Agent connectsoutbound to WireCanal's servers to establish a tunnel. No inbound port opening and no VPN are required. On the outside, a public URL of the form https://<name>.wirecanal.com is created, and requests arriving there flow through the tunnel to your local 127.0.0.1:9904.

The same URL also handles OAuth. Unauthenticated access receives a 401, and connections from ChatGPT and Claude go through only if the owner has approved them on WireCanal's authorization screen.

There is one more mechanism specific to MCP.You choose which tools may be shown to outside AI in a local configuration file (wirecanal.json). The default denies all tools, and only tools listed in the allowlist are exposed. The authoritative copy of this allowlist lives in the local file. Changes made from the dashboard are treated as proposals, and no additional tools become public unless approved and applied on the local side.

With that, we start from the state where the Part 1 server is running at 127.0.0.1:9904.

Step 1: Create a canal in the Dashboard

Register an account on the WireCanal dashboard (app.wirecanal.com) and you can create a public route called a "canal." The creation wizard has five steps.

First, the type. Choose "MCP — expose an internal MCP server to AI services."

Next, "Which AI will use it?" This time we checked both Claude and ChatGPT. Based on the AI selected here, the OAuth connection settings are added to the canal automatically.

For the public address, proceed with the auto-assigned subdomain. For the forwarding target, enter 127.0.0.1:9904, where the Part 1 server is listening.

The public URL is finalized on the confirmation screen. In the author's environment, https://mh1sjzat.ja100.wirecanal.com was issued.

Press "Create" and you move to the canal detail screen. That completes the server-side configuration. No static IP, no certificate acquisition — nothing of the sort.

Step 2: List the Tools You Are Willing to Expose in wirecanal.json

The "Setup" tab of the canal detail screen shows the connection file (wirecanal.json). You can download and use it as is, but for MCP there is one edit to make.tools.allow, write the names of the tools that outside AI may see.

wirecanal.json(tools.allow with the two tools added)

{
  "access_key": "ck_(connection key issued per canal)",
  "forward_target": "127.0.0.1:9904",
  "mode": "mcp",
  "tools": {
    "default": "deny",
    "allow": ["execute_sql_query", "get_database_stats"]
  },
  "lang": "ja"
}

The default is "default": "deny" with an empty allow — in other words,immediately after connecting, all tools are denied. The Part 1 server has only two tools so we listed both, but even if, say, an internal MCP server had 20 tools, none beyond the two listed here would be visible to outside AI. They do not appear in the listing (tools/list) either.

This fail-safe default follows the same philosophy as the "don't show the AI everything" design from Part 1. Since the authoritative ledger deciding the exposure scope lives locally, dashboard operations alone cannot expand the set of public tools. Applying changes requires local approval.

Step 3: Start the Agent

Following the guidance on the Setup tab, install and start the Agent on the machine where the server you want to expose is running. On Windows, it is two commands.

irm https://download.wirecanal.com/install.ps1 | iex
.\wirecanal.exe -config wirecanal.json

The first line downloads the Agent (wirecanal.exe is placed in the current folder); the second starts it. If your organization's security policy prohibits direct execution of remote scripts, save install.ps1 to a file first, review its contents, and then run it. Here is the startup log from the author's environment.

Within a few seconds the dashboard also shows "Connected!", and at this point the tunnel is up.

Confirm That Unauthenticated Access Is Stopped with a 401

Before connecting, let us confirm that the public URL is not in a "callable by anyone" state. As we wrote in Part 1, nothing is more dangerous than a defense that is not actually in effect — the reliable way to check is to actually hit it.

curl -i -X POST https://mh1sjzat.ja100.wirecanal.com/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","method":"initialize","id":1}'

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer resource_metadata="https://mh1sjzat.ja100.wirecanal.com/.well-known/oauth-protected-resource"

The request without an authentication token was stopped with a 401 before it ever reached the local server. In addition, the WWW-Authenticate header carries resource_metadata. This is the URL of the "protected resource metadata document" defined in RFC 9728 — not the URL of the authorization server itself. ChatGPT and Claude first fetch this document, follow the authorization_servers written in it to find the authorization server, and start the OAuth connection procedure automatically.

Connecting from Claude

From the settings of web-based Claude, open the connectors screen and choose "Add custom connector." All you enter is a name and the public URL from earlier with /mcp appended. The client ID and secret fields can be left empty (the official WireCanal guide listed at the end describes entering a client ID and secret manually; in the author's environment as of August 6, 2026, entering only the URL was enough to connect).

After "Add" and then "Connect," the screen switches to WireCanal's authorization page. It shows which canal the connection is being requested for; review the details and approve.

Once connected, the connector details show the tool list. Note this carefully:

the only tools listed are the two written in the allow list of wirecanal.json.

From here, just ask in chat — in Japanese, exactly as with Claude Code in Part 1. By default, you are asked for approval before a tool is used.

After approval, Claude issued SQL against the sales database and returned the answer.

Company executives: ¥25,621,641. Researchers: ¥24,525,975. Civil servants: ¥23,685,112. These match the aggregates Claude Code returned in Part 1 — exactly the measured values of the fixed-seed dummy data. From Claude in the browser all the way to SQLite running on the local PC, a single unbroken path is in place.

Connecting from ChatGPT

On the ChatGPT side, first enable developer mode in order to connect an unverified MCP server. In the author's environment, we turned on "Developer mode" under "Security and login" in Settings (there is also a link to the same place at the bottom of the "Plugins" screen in Settings). This feature lets you add connectors that could damage data, at your own risk, so read the risk explanation carefully before proceeding. The screen layout may vary by plan and rollout timing.

Next, from "Plugins" in the sidebar, choose "Create an app." Enter a name and the server URL (https://<name>.wirecanal.com/mcp), and the OAuth settings in the authentication section are detected automatically.

After "Create," proceed to "Sign in," and the same WireCanal authorization screen as with Claude appears; approve it.

When using it, there is one trick to know. From the "+" in the chat input field, select the app you just created and attach it to the conversation before asking your question. In the author's environment, asking in plain prose to "use the sales database" without attaching the app led ChatGPT to start a web search instead of using the connector. Selecting the app explicitly via "+" reliably invokes the MCP server.

The table returned matches Claude's numbers. The local server log also recorded the request from the ChatGPT side as POST /mcp.

Pricing

The steps in this article (creating an MCP canal, OAuth authentication, auto-assigned subdomain) are, per the comparison table on the WireCanal pricing page (wirecanal.com), available on the free plan. No credit card registration is required either.

In other words, everything done in this article — from publishing a self-built MCP server to connecting it with web-based ChatGPT and Claude — can be done entirely for free.

The public hostname issued on the free plan does not disappear as long as it is in use. While the Agent's connection is alive, its expiration is extended automatically.

Conversely, if you sever the connection, for example by stopping the Agent, it expires 72 hours after the last connection (you can recreate it after expiration, but the URL will change).

If you use the MCP server regularly, the Agent will presumably be running continuously, so in practice you can keep using the same URL on the free plan.

If you want to reserve a subdomain with a name of your choosing, the Lite plan and above — which offer persistent hostnames that survive disconnections — are the candidates. Other plan differences include TCP exposure (RDP and SSH) from the Pro plan up, and bringing your own custom domain on Premium.

Note that some screenshots in this article were taken on a Lite-plan account, but every step can be performed with free-plan features alone.

One more point, on organizational use. This time the canal's owner personally approved the OAuth connection, but on Lite and above you can integrate with your company's identity platform (an OIDC IdP such as Google Workspace) so thatmembers of the organization can receive connection approval with their own company accounts. Having the whole team use the MCP server is something we will actually do in a later installment of this series.

OAuth for Connecting to ChatGPT Is Solved on the Route Side

The point of this installment is thatnot a single line of the Part 1 server was touched. The MCP server written for localhost worked as is from both web-based ChatGPT and Claude. The three walls — reachability, HTTPS, and OAuth — are problems of the route, not of the server code, so solving them on the route side is the sound approach. That is this installment's answer.

The safety of the public route is layered in two. At the entrance, OAuth stops unauthenticated access with a 401; beyond that, the tool allowlist hides everything except the two tools we chose to show. The authoritative allowlist lives in the local wirecanal.json and cannot be changed by dashboard operations alone — applying changes requires local approval. The three safety valves built inside the server in Part 1 (read-only access, SELECT-only inspection, and an execution time limit) remain in place as the last line of defense.

One thing that tripped us up was that ChatGPT did not select the app automatically. If you do not know this beforehand, it looks as though the connector is "connected but never used." Since many readers are likely to stall at the same spot, we kept it in the article.

We should also note what was not verified. What we connected were the author's ChatGPT Plus account, on which developer mode was visible as of August 6, 2026, and a paid Claude account. The availability and screen layout of ChatGPT's developer mode and custom MCP apps may vary by plan and account rollout, and the same items may not appear on every account. We also have not confirmed how far this works on the AI services' free plans. Furthermore, when connecting to a real internal database, there is work to do beyond this article, as described in Part 1 — splitting into task-specific tools, auditing access logs, and so on.

See you next time.

Sample Code

We use the Part 1 code as is. After cloning, the commands up to creating the DB and starting HTTP mode are as follows.

qualiteg/mcp-server-tutorial | Complete sample code and README (GitHub)

git clone https://github.com/qualiteg/mcp-server-tutorial.git
cd mcp-server-tutorial
python -m pip install -r requirements.txt
python db_setup.py
python mcp_server_sales.py --http --port 9904

References

• MCP Authorization (official Model Context Protocol specification) * The draft is subject to updates. This article was verified against the content as of August 6, 2026
• OAuth 2.0 Protected Resource Metadata (RFC 9728)
• WireCanal official site | MCP-ready secure tunnels
• How to connect Claude to an MCP canal (guide on wirecanal.com)
• How to connect ChatGPT to an MCP canal (guide on wirecanal.com)

Related Articles

How to Build an MCP Server — Let AI Answer Questions About Your Database with Python and FastMCP (Part 1)

A hands-on guide to building an MCP server, with working code from start to finish. Using Python and FastMCP, we wrap a SQLite sales database as MCP tools so an AI can take a plain-language question, write the SQL itself, and return aggregated results.

Qualiteg BlogQualiteg プロダクト開発部

What Sets ngrok and WireCanal Apart? The Developers Compare Pricing, Operations, and MCP Design

An honest comparison from the developers of WireCanal, a Japan-based service built on the same reverse tunnel approach as ngrok. The differences come down to three: pricing philosophy, where operations run, and where permissions live when connecting to AI.

Qualiteg BlogQualiteg プロダクト開発部

Hello!
This is the Qualiteg Product Development Team!

"Is there a Japan-based tunnel service that can serve as an alternative to ngrok?"

Many of you have likely arrived at this article after searching for exactly that.

This article answers that question head-on, from the developer's side.

Let us be clear from the start: the service introduced here, WireCanal , is developed and operated by our company (Qualiteg Inc.), a Japanese corporation, with its relay infrastructure running on servers in Japan — atunnel service.

Since this is an article in which we compare our own product, listing only the points that flatter us would earn no one's trust. Where the two are the same, we say so; where ngrok is the better fit, we say that too.

To state the conclusion up front:

the basic tunneling mechanism is the same as ngrok's。

There are three differences.

The pricing philosophy (a USD-denominated structure with fixed allotments and usage-based elements, versus a flat JPY rate with no overage billing) · the shape of operations and contracting (domestic relay infrastructure and a Japanese-language, JPY-denominated contract with a Japanese corporation) · and where permissions live when connecting to AI (the authoritative permission ledger stays on the company's side)

If none of these three matter for your use case, there is no need to force a switch. Let us look, in order, at what is the same and what is different.

The Mechanism Is the Same, So There Is Almost Nothing to Relearn

The reverse tunnel approach that ngrok made widely known works like this: place a small program (WireCanal calls it the Agent) on your PC or an internal server, and it connects outward, from the inside, to a relay server to establish a tunnel. A public URL is created on the outside, and access arriving there flows through the tunnel back to your machine.

The advantage of this approach is that no inbound port needs to be opened on the firewall.

WireCanal is built on the same idea (Figure 1). Run a command to install the Agent, create a public route called a canal, and specify a local port as the forwarding target. If you have used ngrok, there is essentially nothing conceptual to relearn.

Note that today's ngrok has broadened its product scope with cloud-side endpoints and more, but this article compares the basic configuration: publishing local and internal services using an Agent.

So what this article should address is not "which one is superior," but how the design beyond that point differs.

Difference 1: Pricing Is a Flat Rate in JPY, Tax Included

ngrok offers plans such as Free, Hobbyist, and Pay-as-you-go (as of August 2026, confirmed on the official site). First, a comparison of the structures.

Pay-as-you-go, intended for production use, includes a $20 monthly allotment, with additional charges beyond it based on transfer volume, request counts, endpoint uptime, and so on. Paying for what you use is a rational design, but the bill for a month when usage grows is hard to predict in advance, and being USD-denominated, it also moves with exchange rates.

WireCanal made every plan a flat monthly rate in JPY, tax included. The differences between plans are as follows.

With annual billing, the monthly equivalents become Lite ¥1,280, Pro ¥2,980, and Premium ¥9,980.

There is no additional billing based on transfer volume.

The Free plan has a guideline of 10GB per month, and speeds may be adjusted when it is exceeded. Paid plans have no monthly transfer cap for normal use, but to keep usage fair and maintain the network, extreme high-volume traffic may be subject to speed limiting or similar measures.

To be candid, a flat rate is not always the better deal. Depending on your scale and use case, ngrok's Free, Hobbyist, or Pay-as-you-go plans may work out cheaper. The value of flat-rate pricing lies in knowing the bill in advance, and it matters more the more your organization relies on approval workflows and budget planning.

Difference 2: Runs in a Domestic Region, with Contracts, Billing, and Support Completed in Japanese and in JPY

WireCanal's relay infrastructure is deployed in a domestic (Japan) region. Contracts and support are provided in Japanese by our company, a Japanese business, and billing is in JPY. Here too, a table to organize the comparison.

Note that ngrok also has a Tokyo point of presence (PoP) and a mechanism for region-pinned routing. So this is not a simple "has a domestic route or not" comparison. What sets WireCanal apart is that operating the relay infrastructure on servers in Japan is an explicit part of the service design, packaged together with a contract with a Japanese corporation, Japanese-language support, and JPY billing.

For organizations where "requests to use overseas SaaS rarely get approved internally" or "accounting is a headache without a JPY invoice," this is not a matter of feature superiority — it is the deciding factor in whether adoption is possible at all.

To be honest here as well: a domestic region does not mean it is always faster than overseas services. Actual latency varies with your location, destination, and route. Conversely, if your primary goal is to serve users around the world, or to use the service the same way from overseas offices, ngrok or Cloudflare — with their global points of presence — are the better fit.

Difference 3: The "Permission Ledger" for What AI May See Lives on the Company's Side

This is the part we thought through most carefully.

MCP (Model Context Protocol) is spreading as the way to let generative AI operate internal systems. In practice, the question then becomes: which of your internal system's capabilities do you let the AI touch? An internal MCP server typically hosts everything from read-only tools to tools that update and delete.

The idea of controlling MCP at the gateway when connecting to external AI is not ours alone. Here is what can be confirmed in each vendor's official documentation.

What makes WireCanal different is that, in addition to per-tool permissions themselves, the authoritative copy of that permission ledger lives ina configuration file on the company's side (Figure 2).

wirecanal.json(the tool permission section, excerpt)

{
  "tools": {
    "default": "deny",
    "allow": ["execute_sql_query", "get_database_stats"]
  }
}

The default denies everything, and only the tools written in this allow list are visible to the AI. Tools that are not permitted do not even appear in the tool listing (tools/list). The cloud console can go only as far as proposing changes; applying them requires an approval action on the company's side. In other words, ordinary console operations alone cannot widen the exposure scope without in-house approval.

One more thing: the Agent connects only to the forwarding targets registered in its configuration file, and does not accept instructions from the cloud side to change targets dynamically. The Agent itself is restricted so that the tunnel's exit cannot become a general-purpose relay toward arbitrary internal destinations.

The procedure for actually connecting a self-built MCP server to web-based ChatGPT and Claude is published, with screenshots, in a series on this blog. Links are at the end of this article.

When Staying with ngrok Is the Right Call

As you can tell from reading this far, WireCanal suits those who want to operate domestically, with predictable pricing, while governing what is exposed to AI. Organized by use case, it looks like this.

Choose your tools by use case. If there is no compelling reason to switch, staying put is itself a sound decision.

How Far You Can Go for Free

WireCanal's free plan hasno time limit、and requires no credit card registration。

With one canal, everything is available: HTTPS publishing, MCP publishing (OAuth integration with ChatGPT, Claude, and others), and all 8 types of access protection including IP restrictions, BASIC authentication, Bearer tokens, and time-window publishing.

The public hostname does not disappear as long as it is in use. While the Agent's connection is alive, its expiration is extended automatically, and it expires 72 hours after the last connection (it can be recreated after expiration). When you find yourself wanting to choose the name yourself, or to keep the same name even while disconnected, that is the time to consider Lite or above.

We recommend starting by publishing one of your own web apps or an MCP server and seeing how it feels to use.WireCanal is free to start (wirecanal.com)。

Summary: Same Approach, Different Placement

ngrok and WireCanal stand on the same foundation: the reverse tunnel. The differences come down to three points — how you pay (a USD-denominated structure with fixed allotments and usage-based elements, versus a flat JPY rate with no overage billing), the shape of operations and contracting (domestic relay infrastructure with a Japanese-language, JPY-denominated contract with a Japanese corporation, versus using a global service), and keeping the authoritative AI tool permission ledger on the company's side.

A tunnel service, once woven into your operations, is a tool you live with for a long time. Choose not just by the checkmarks on a feature table, but by whether these three placements fit your organization — and you will not regret it later.

See you next time.

References (Primary Sources)

• ngrok pricing plans (official site)　* Confirmed against the content as of August 2026
• ngrok's official MCP gateway guide (ngrok Docs)
• ngrok points of presence (ngrok Docs)
• Cloudflare MCP Server Portals (Cloudflare Docs)
• WireCanal plan and feature comparison table (wirecanal.com)
• WireCanal's guide to MCP and AI integration (wirecanal.com)

Related Articles

• How to Build an MCP Server (Part 1) | Connecting a Sales DB to AI with Python and FastMCP (this blog)

Hello! This is the Qualiteg Product Development Team!

You want to make your local database queryable by an AI. What you need for that is an MCP server.

The goal sounds simple, but once you start researching, you can get as far as "just build an MCP server" and still get stuck on how exactly to turn your own database into MCP.

This article walks you through everything up to

building your own MCP server with Python and FastMCP and querying a sales database in natural language from the CLI version of Claude Code

, with working code from start to finish. The code is about 200 lines. When you ask "Show me the top 3 occupations by sales" (in Japanese, in our demo), Claude Code assembles the SQL itself and comes back with the aggregated answer.

All the code is on GitHub. Clone it, install the dependencies, create the DB, and it runs.The repository README covers every step of running it, so if you'd rather get your hands moving without reading the article, start there.

qualiteg/mcp-server-tutorial | Complete sample code and README (GitHub)

This is a two-part series. Part 1 (this article) gets you to the point where it works from the CLI version of Claude Code. In Part 2, we make the same server usable from the web versions of ChatGPT and Claude. A CLI version running locally, and a web version running in the browser. Between those two lies a wall that trips up a lot of people.

What MCP is, in three minutes

MCP (Model Context Protocol) is a common standard for connecting AI apps to external data and tools. Anthropic released it in late 2024, and today it is supported not just by Claude but by ChatGPT, Gemini, and a range of coding agents.

Before MCP, if you wanted an AI to touch your database, you had to build one-off integrations for each AI app's proprietary plugin or function-calling scheme. MCP standardizes how the "server" (the side exposing data and tools — what we build here) and the "client" (the AI app side: Claude, ChatGPT, and so on) talk to each other. Build an MCP server once, and multiple MCP-capable AI apps can use it without app-specific plugins. That said, the available transports, auth methods, plans, and admin settings differ per client.

An MCP server can expose three kinds of things — tools, resources, and prompts — but the one you will use first in practice is tools: functions the AI can call. This article covers tools only.

One more piece of background that pays off in Part 2: the MCP spec defines two standard transports.

In this article, Part 1 uses a local stdio server from the CLI version of Claude Code, and Part 2 uses a remote Streamable HTTP server from the web versions. That said, it is not a strict one-to-one mapping of "CLI means stdio, web means HTTP". Claude Code, for example, can connect to both stdio servers and remote HTTP servers. In Part 1 we will try both.

What we are building

A sales database (SQLite) modeled on an online store for PC parts, plus an MCP server that queries it.

There are three tables: a customer master (customers) with 500 people, a product master (products) with 72 products across 8 PC-part categories, and sales line items (sales_transactions) with 4,000 records spanning 2024 through 2026. Customers carry attributes like age, occupation, and income bracket, so you can run analyses like "broken down by occupation."

We are going to be bold and give Claude Code just two tools: get_database_stats, which returns the table structure and statistics, and execute_sql_query, which executes SELECT statements.

You could also design this as a lineup of per-task functions — a "sales aggregation tool," a "customer analysis tool," and so on. But it is Claude Code that can write SQL, so handing over one general-purpose SQL execution tool goes further. The trade-off is that the safety work becomes the server's responsibility — more on that later.

Setup

Implementing the MCP server itself requires Python 3.10 or later and FastMCP. If you want to try it from Claude Code the way this article does, you will also need a working Claude Code environment.

python -m pip install fastmcp

pip is not the one to use here; go with python -m pip instead.pip can land the package in a different environment from the Python that will later launch your MCP server.

FastMCP is a high-level framework that lets you define MCP tools by adding a single decorator to a Python function. It also bundles client functionality and an HTTP server. The version I verified against is standalone FastMCP 3.4.5.

First, a working MCP server in 10 lines

Before diving into the sales DB, let's get the full picture with a minimal build.

# hello_mcp.py
from fastmcp import FastMCP

mcp = FastMCP(name="hello-server")

@mcp.tool(description="Adds two numbers")
def add(a: int, b: int) -> int:
    return a + b

if __name__ == "__main__":
    mcp.run()  # stdio transport by default

That's it — a complete server. The tool definition handed to the AI is generated automatically from the function's type hints and description.

Note that just running python hello_mcp.py prints nothing. It simply sits waiting for input as a stdio server. To see it work, call it from FastMCP's in-memory client.

import asyncio

from fastmcp import Client
from hello_mcp import mcp


async def main():
    async with Client(mcp) as c:
        print([t.name for t in await c.list_tools()])   # ['add']
        result = await c.call_tool("add", {"a": 120, "b": 5})
        print(result.data)                              # 125


if __name__ == "__main__":
    asyncio.run(main())

Registering it with Claude Code is a one-liner (we will actually register and use it later in this article).

claude mcp add hello-server -- python hello_mcp.py

If it fails to connect, specify absolute paths for both Python and the script. If you are using a virtual environment, you need to point at the Python inside it.

What I want you to notice here is that the only material the AI has for deciding "this needs a calculation," "there is a tool called add," and "let's call it" is the tool name, the description, and the inputSchema generated from the argument names and types. Among these, the description is the central place where you can explain in natural language when the tool should be used — and it turns out to matter more than anything else in MCP server design.

Step 1: Prepare the sales database

Create three tables in SQLite.

db_setup.py(Full code for the table definitions and dummy-data generation (GitHub))

cur.execute("""
CREATE TABLE sales_transactions (
    transaction_id   INTEGER PRIMARY KEY,
    date             TEXT NOT NULL,     -- 'YYYY-MM-DD'
    customer_id      TEXT NOT NULL REFERENCES customers(customer_id),
    product_id       TEXT NOT NULL REFERENCES products(product_id),
    product_name     TEXT NOT NULL,
    product_category TEXT NOT NULL,
    quantity         INTEGER NOT NULL,
    unit_price       INTEGER NOT NULL,
    total_price      INTEGER NOT NULL
)""")

cur.execute("CREATE INDEX idx_trans_date     ON sales_transactions(date)")
cur.execute("CREATE INDEX idx_trans_customer ON sales_transactions(customer_id)")
cur.execute("CREATE INDEX idx_trans_category ON sales_transactions(product_category)")

Do create the indexes. The AI will not hesitate to fire full-period aggregation queries at you.

The random seed is fixed, so everyone who runs it gets identical data. Here is what it looked like in my environment.

Every number in this article is a measured value from this data. You should see the same numbers on your machine.

Step 2: Write the entire schema into the tool description

This is the single most important point in this article.

execute_sql_query's description, we write in the entire database schema plus example queries。

mcp_server_sales.py(Full code for the description (GitHub))

@mcp.tool(description="""Executes a SQL query against the sales data. Only SELECT statements are supported.

[Database schema]

■ customers (customer master)
  - customer_id (TEXT): customer ID (e.g. 'C0001')
  - age (INTEGER): age
  - gender (TEXT): gender ('男性' = male, '女性' = female)
  - prefecture (TEXT): prefecture
  - occupation (TEXT): occupation ('ITエンジニア' = IT engineer, '会社員' = office worker, '学生' = student, '自営業' = self-employed, etc.)
  - annual_income (INTEGER): annual income (in units of 10,000 yen)

■ products (product master)
  - product_id (TEXT): product ID (e.g. 'CPU_001')
  - product_category (TEXT): CPU, GPU, Memory, SSD, HDD, Motherboard, PowerSupply, PCCase
  - tier (TEXT): 'high' / 'mid' / 'entry'
  (and so on — list every column in this style)

■ sales_transactions (sales line items)
  - date (TEXT): sale date (YYYY-MM-DD format)
  - customer_id (TEXT): customer ID → customers.customer_id
  - product_id (TEXT): product ID → products.product_id
  - quantity / unit_price / total_price (INTEGER)

[Data period] 2024-01-01 to 2026-12-31 (4,000 rows)

[Example queries]
- Sales by occupation:
  SELECT c.occupation, SUM(t.total_price) AS sales
  FROM sales_transactions t JOIN customers c ON t.customer_id = c.customer_id
  GROUP BY c.occupation ORDER BY sales DESC

- High-end GPU sales:
  SELECT p.product_name, SUM(t.total_price) AS sales
  FROM sales_transactions t JOIN products p ON t.product_id = p.product_id
  WHERE p.product_category = 'GPU' AND p.tier = 'high'
  GROUP BY p.product_name ORDER BY sales DESC
""")
async def execute_sql_query(sql: str, ctx: Context) -> str:
    ...

That may look absurdly long. But whether the AI writes correct SQL depends less on how smart the model is and more on how accurately you hand it the schema. For a small, fixed schema like this one, putting the necessary columns and JOIN relationships in the description proved effective.

The AI's material for choosing tools is the tool name, the description, and the inputSchema generated from argument names and types. Of these, the description is where you can say in natural language when the tool should be called.

Including two or three example queries is another good trick. With worked examples of how to write the JOINs and use the columns, the accuracy of the AI's SQL stabilizes visibly.

Step 3: Give the SQL execution tool three safety valves

If you are letting an AI execute SQL, you have to guard against writes and against heavy queries hogging your resources. We set up three safety valves with distinct roles.

Layer 1: open the database read-only. With SQLite, connecting via a URI and specifying mode=ro makes the database refuse writes at the source. Even if something slips past the later checks, it stops here.

mcp_server_sales.py(Code for the read-only connection function (GitHub))

conn = sqlite3.connect(f"file:{DB_PATH.as_posix()}?mode=ro", uri=True)

Layer 2: reject anything that is not a SELECT. This is a string check, not a SQL parser, so treat it as a conservative helper filter that knocks out obvious write commands early. Final write prevention is layer 1's job.

mcp_server_sales.py(Code for the SELECT-only filter (GitHub))

normalized = sql.strip().upper()
if not normalized.startswith("SELECT"):
    return "Error: only SELECT statements are supported"
for keyword in ("DROP", "DELETE", "INSERT", "UPDATE", "ALTER", "CREATE", "TRUNCATE", "ATTACH", "PRAGMA"):
    if keyword in normalized:
        return f"Error: {keyword} is not allowed"

Layer 3: put a hard limit on execution time. To keep heavy aggregations from hanging things, we cut them off with SQLite's progress handler.

mcp_server_sales.py(Code for the execution-time limit (GitHub))

import time

def install_query_timeout(conn, timeout_sec):
    deadline = time.monotonic() + timeout_sec

    def guard():
        return 1 if time.monotonic() >= deadline else 0

    # Call guard every time the SQLite virtual machine executes 1,000 instructions
    conn.set_progress_handler(guard, 1000)

This is a spot where I got it wrong once.I was counting the number of progress-handler invocations and using that as the limit, but the second argument of set_progress_handler is a count of SQLite virtual-machine instructions, not seconds. The invocation count is not wall-clock time, so the wait varied with machine speed and query content.time.monotonic() and comparing against real elapsed time was the correct answer.

Note that this third layer is not a measure for enforcing read-only access. It is protection against a runaway query monopolizing your resources. Each layer guards something different.

Whether all three layers actually work is something we will verify for real later on.

Step 4: Let the server decide how much goes back to the AI

This is the part that formats query results. Look at where we cap the number of rows returned.

mcp_server_sales.py(Code for formatting query results (GitHub))

    if not rows:
        return "No data matched your query."

    lines = [" | ".join(columns), "-" * 40]
    for row in rows[:MAX_ROWS]:                      # MAX_ROWS = 50
        lines.append(" | ".join("" if v is None else str(v) for v in row))
    if len(rows) > MAX_ROWS:
        lines.append(f"... plus {len(rows) - MAX_ROWS} more rows (aggregate or narrow your conditions to see the whole picture)")

    return f"Query result ({len(rows)} rows):\n\n" + "\n".join(lines)

The AI will happily throw SELECT * FROM sales_transactions at you. Returning all 4,000 rows would just overflow the context window. What worked in practice was reporting the true row count honestly and nudging it to "aggregate or narrow your conditions."

Note, though, that what this implementation limits is only the number of rows returned to the AI — it does not limit how many rows are read from SQLite. Facing millions of rows of real data, you will need additional measures: switch to fetchmany(MAX_ROWS + 1), add a LIMIT on the SQL side, look at query cost, and so on.

Step 5: Add a tool that shows the big picture first

The second tool simply returns a summary of the table structures and statistics.

mcp_server_sales.py(Full code for the statistics tool (GitHub))

@mcp.tool(description="""Returns the database's table structures, row counts, data period, and per-category sales statistics.
Before writing any SQL, always call this tool first to check the structure.""")
async def get_database_stats(ctx: Context) -> str:
    ...

The last sentence of the description does the real work.

"Before writing any SQL, always call this tool first"

In my environment, adding this line made the model check the statistics first and then build its SQL.

Keep in mind, though, that this is a request to the model, not a control enforced by the server. Depending on the model, the client, and the flow of conversation, it may fire off SQL without ever looking at the statistics. The important thing is not to let your safety story rest on this one sentence. A healthy mental model: a tool description works not only as feature documentation but as the place to tell the model the conditions and procedure for using the tool.

Trying it out

Register it with Claude Code.

claude mcp add sales-db -- python C:\qualiteg_examples\mcp_server_sales.py

Confirm the registration.

claude mcp list

sales-db: python C:\qualiteg_examples\mcp_server_sales.py - ✔ Connected

✔ Connected shows up, the AI can see this server. After that, you just ask in natural language — plain Japanese, in our demo.

claude -p "職業別の売上トップ3を教えて"  # "Show me the top 3 occupations by sales"

This is what came back.

The question was a single sentence in natural language. Without me writing a single line of SQL, the AI checked the table structure, issued a query with a JOIN and GROUP BY, and returned the aggregation neatly formatted as a table.

The amounts match the measured values in the dummy data. On top of that, it also aggregated transaction counts — which the question never asked for — and added its own reading: researchers have the most transactions but a low average ticket, while executives buy less often but spend more per purchase. It is not just generating SQL; it interprets the aggregated results.

Multi-step conditions like "high-end GPUs released in 2025, ordered by sales to customers in their 20s" go through in the same easy way. People who have never written SQL can now talk to the sales database.

That said, being able to ask in natural language and the aggregation being correct for business purposes are two different things. If the results feed important decisions, make sure you can review the executed SQL, the aggregation definitions, the time period, and any exclusion conditions. "Sales," "customer count," and "average ticket" can all change depending on your in-house definitions.

Verifying the guardrails actually work

Nothing is scarier than a defense you wrote but that is not actually working. In fact, while writing this article I discovered the timeout was not functioning.

The three layers each protect something different, so they have to be verified in different ways as well. The keyword check that runs through the tool is verified with FastMCP's in-memory client; the read-only mode and the timeout are verified by touching the DB connection directly.

verify.py(Full code for the verification script (GitHub))

# Layer 1: does the connection itself refuse writes?
conn = connect_readonly()
try:
    conn.execute("DELETE FROM customers")
    raise AssertionError("This was supposed to be read-only, but the write went through")
except sqlite3.OperationalError as e:
    assert "readonly" in str(e).lower()

# Layer 3: is a heavy query interrupted based on real time?
conn = connect_readonly()
install_query_timeout(conn, QUERY_TIMEOUT_SEC)
started = time.monotonic()
try:
    conn.execute(HEAVY_SQL).fetchone()
    raise AssertionError("The query was not interrupted")
except sqlite3.OperationalError as e:
    assert "interrupted" in str(e).lower()
    print(f"interrupted after {time.monotonic() - started:.1f} s")

Here are the results in my environment.

SELECT at the start does not save a query that smuggles in a DROP — that gets stopped too. The read-only connection rejects writes with "attempt to write a readonly database," and the heavy query was interrupted at 10.0 seconds of real time. The row cap also works: throwing SELECT * FROM sales_transactions returns 50 rows followed by "... plus 3,950 more rows."

Note that this verification used the in-memory client and direct DB connections; it does not cover behavior through the stdio or HTTP transports.

Only after checking this far should you start letting people in your company use it.

Running it as a resident HTTP server

Everything so far used stdio — the AI app launches the server locally. To run it as an independent, long-lived server used over the network, switch to Streamable HTTP. Only one line of the startup code changes.

mcp_server_sales.py(Code for the startup section (GitHub))

mcp.run(transport="http", host="127.0.0.1", port=9904)

With this, http://127.0.0.1:9904/mcp becomes your MCP endpoint.

Bind to 127.0.0.1. This server has no authentication.0.0.0.0 means anyone who can reach you over that network can execute SQL. The sample code defaults to 127.0.0.1 and prints a warning when you specify an external bind address. Once you move on to sharing it inside the company or using it from outside, the baseline becomes HTTPS, authentication and authorization, source restrictions, Origin validation, and audit logging in front of it. The server in this article is the minimal build that comes before all of that — for understanding the mechanism.

Adding a health-check endpoint also makes operations easier.

mcp_server_sales.py(Health-check code (GitHub))

@mcp.custom_route("/health", methods=["GET"])
async def health_check(request):
    from starlette.responses import JSONResponse
    return JSONResponse({"status": "ok", "service": "sales-database-server"})

In the sample code, the --http option lets you switch between them.

python mcp_server_sales.py --http --port 9904

Once it is up, hit the health check to confirm it is alive.

curl http://127.0.0.1:9904/health

{"status":"ok","service":"sales-database-server"}

It returns status only. There is no reason to leak information monitoring does not need (like the DB's absolute path or exception internals).

What I learned building this

What helped most this time was limiting the server to two tools. Rather than lining up per-task functions, handing over one general-purpose SQL execution tool lets the AI think for itself and produce answers. That said, this holds for a small dummy DB. With real data, per-task tools that restrict which tables, columns, and aggregation granularity can be accessed are safer and easier to audit.

In exchange, the way you write the description changes the results dramatically. Writing in the full schema and example queries made the AI's SQL stable. A description works not only as feature documentation but as the place to tell the model the conditions and procedure for using the tool. Remember, though, that procedural instructions are requests — nothing guarantees they are followed.

The safety work came in three layers: open read-only, reject non-SELECT, cap execution time. And we did not stop at writing them — we actually exercised each of the three. Skip that step and you will ship defenses to your company that you merely believe are working. In fact, my first timeout implementation never measured real time and did nothing at all.

Do not forget the cap on returned rows, either. The AI will happily request full table dumps. Cutting off at 50 rows while honestly reporting the total count was enough, in my environment, to make it switch to aggregation queries.

If you are going to run it as a resident server, switch to Streamable HTTP and add the health check — it makes operations easier. But remember there is no authentication built in, so do not put it on the office LAN or the internet as-is.

One more thing: this sample is a minimal build for understanding the mechanism. For multiple concurrent users you will separately need measures like moving the synchronous DB work onto threads, capping concurrency, and setting DB-side timeouts.

Coming up in Part 2

What Part 2 does is clear-cut.

We will connect this MCP server, built on localhost, to the web versions of ChatGPT and Claude — and get over the OAuth wall.

The CLI version of Claude Code is for developers. But the web versions, running in a browser, can be handed to people who never touch a command line. A sales-planning teammate can ask "What was our best-selling GPU last month?" right from their usual ChatGPT screen. Same MCP server — a dramatically wider audience.

And this is where it gets genuinely hard. To connect an MCP server to the web versions' connectors, you need two things.

One is an HTTPS URL reachable from the internet. Nobody outside can reach a server running on your local PC — and you cannot simply expose a server that can reach your internal database, either.

The other is authentication and authorization to identify users and control access.

Depending on configuration, both ChatGPT and Claude can connect to remote MCP servers without authentication. But publishing an unauthenticated MCP server that can reach an internal database is simply not an option. So in Part 2 we use OAuth, following the MCP Authorization spec.

Implementing that yourself means authorization metadata, client registration, an authorization endpoint, and token issuance and validation — an entire implementation completely separate from the tools themselves. The tool code took 200 lines, yet you burn out before reaching the main event. This is where a lot of people stall.

In Part 2, we will walk through how to get over these two walls, with screenshots, all the way to connecting the web versions of ChatGPT and Claude to this sales DB.And with a way you can try for free.

Full disclosure up front: Part 2 uses WireCanal, a service we provide. It has a free tier, so you can follow along hands-on (terms and the free tier are as of the time of writing).

See you next time!

Sample code

All the code from this article is on GitHub.

qualiteg/mcp-server-tutorial | Complete sample code and README (GitHub)

git clone https://github.com/qualiteg/mcp-server-tutorial.git
cd mcp-server-tutorial
python -m pip install -r requirements.txt
python db_setup.py

Then register it with Claude Code.

claude mcp add sales-db -- python /path/to/mcp-server-tutorial/mcp_server_sales.py
claude mcp list
claude -p "職業別の売上トップ3を教えて"  # "Show me the top 3 occupations by sales"

When using stdio, you do not need to launch python mcp_server_sales.py manually in a separate terminal. Claude Code starts the registered command as a child process.

Everything described here is also in the repository's README. It covers the quick start, how to register with Claude Code, the tools provided, and running as an HTTP server — so when working hands-on, the README alone is enough.

References

• Model Context Protocol official site
• FastMCP
• Anthropic: Introducing the Model Context Protocol

Related articles

Qualiteg Inc. (Headquarters: Chiyoda-ku, Tokyo; CEO: Tomonori Misawa) announced the general availability of WireCanal on August 4, 2026. WireCanal is a service that securely connects cloud AI services such as Claude and ChatGPT to databases and MCP servers inside corporate networks. Users are authenticated through the company's own identity provider, and only organizational accounts belonging to administrator-approved domains can access internal systems through AI. Beyond AI integration, WireCanal can also be used as a general-purpose secure tunnel for HTTPS and TCP (RDP, SSH, databases, and more).

A canal is an artificial waterway that admits only vessels with an approved destination. In the same way, WireCanal builds a dedicated passage — a secure tunnel — between cloud AI and internal systems through which only permitted traffic can travel. In WireCanal, each of these individual communication paths is called a "canal." A canal is opened by the WireCanal Agent, a relay program installed inside the corporate network that connects outward, so there is no need to open inbound ports or build a new VPN environment.

WireCanal is available starting with a free plan (USD 0) that requires no credit card registration. To mark the start of general availability, Qualiteg is also running a launch campaign: customers who sign up for an annual Light Plan contract (one canal) by August 31, 2026 can use the service for 6,000 yen (tax included; equivalent to 500 yen per month) for the first year. After the campaign ends (from September 1, 2026), the price for new annual contracts is 15,360 yen per year (tax included).

You want AI to work with live internal data — but internal systems cannot be exposed

"I want to ask Claude about our internal sales data." "I want to check inventory and production status in plain language, without learning SQL or system operations." As business adoption of cloud AI advances, the databases and core systems that hold the answers — and the MCP servers that make them usable from AI — remain on local PCs and inside corporate networks. Cloud-hosted AI cannot reach them as-is.

MCP (Model Context Protocol) is a common specification that lets AI discover and execute external tools, and it also defines mechanisms for authentication and authorization. Supporting MCP alone, however, does not automatically provide a secure path from cloud AI into the corporate network, nor account management tied to the company's IdP.

The enterprise operational foundation — a communication path from cloud AI into the corporate network, user management based on company identities, and access revocation linked to company account suspension when an employee leaves — must be built separately.

At the same time, publishing an MCP server to the internet without proper authentication and access control means exposing a wide-open entrance to internal systems. Trying to solve this with inbound port openings or VPN construction instead places a heavy build-and-operate burden on the IT department. "We want to use it, but there is no safe way to expose it." This is where many companies come to a halt.

WireCanal's answer: connect outward from inside, and guard the entrance with company identities

Place the WireCanal Agent — a connection app that runs on Windows and Linux — inside the corporate network, and the Agent establishes an outbound encrypted tunnel. Requests from Claude or ChatGPT reach only the internal MCP servers designated in advance.

And the entrance is guarded by company identities. When a user accesses a canal, the login screen of the company's identity provider — such as Google Workspace — appears, and only organizational accounts from administrator-approved domains can connect. There is no need to issue separate WireCanal accounts to each user. When the company suspends an account, for example upon an employee's departure, that user's access is automatically revoked within one hour. Delegating the management of "who can use it" entirely to the company's existing identity infrastructure — that is WireCanal's answer.

What sets WireCanal apart: the company chooses which tools the AI can see

WireCanal also works as a general-purpose secure tunnel, offering HTTPS publishing of web systems along with eight kinds of access protection — including IP address restrictions, BASIC authentication, and token authentication — on every plan. TCP relaying for RDP, SSH, databases, and more is available on the Pro Plan and above.

On top of that comes its distinctive feature: per-tool publication control for MCP. Even if an internal MCP server exposes many tools, an allowlist ledger determines which ones the AI can see. The default is deny-all. The authoritative copy of the ledger is a configuration file kept inside the corporate network; the cloud-side console can only "propose" changes, and applying them requires an approval action performed internally. If a tunnel is a plain pipe, WireCanal is a gatekeeper that understands the contents and chooses what to let through.

Defense in depth for organizational AI use

The boundaries WireCanal protects are threefold: "who can connect" and "what the AI can see," as described above, plus "where inside the network traffic can reach." The connection app connects only to its configured forwarding destinations, which prevents the tunnel from being used as a passage relaying traffic to arbitrary internal destinations. WireCanal's relay infrastructure is operated on servers in Japan, and inbound access is logged per canal, so you can review afterwards when and from where connections occurred. WireCanal adopts a fail-closed design: any traffic whose authorization cannot be confirmed by its access control is rejected.

These boundaries are supported by the following five layers of defense.

1. Fixed forwarding destinations — The Agent connects only to its configured forwarding destinations. It re-validates the destination on every connection and does not follow redirects
2. Restricted entrances — The public entrance can be narrowed to the IP addresses of legitimate callers. Agent connections are verified against issued keys
3. Access protection — Eight kinds are available on every plan: IP restrictions, country restrictions, BASIC authentication, token authentication, time-window publishing, path restrictions, automatic blocking, and stealth mode
4. Publication allowlists — The MCP tools and URL paths exposed externally are managed in an allowlist ledger kept on the internal side. The default is deny-all
5. Standards-compliant MCP authentication — Built-in authentication and authorization conforming to the MCP Authorization specification and related RFCs (RFC 9728, RFC 8414, RFC 8707, and others); credentials verified at the entrance are never passed on to the forwarding destination

In addition, WireCanal provides isolation via a dedicated subdomain and unique ID per canal, per-canal suspend and resume, immediate shutdown from the internal side, and access logs scaled by plan (the most recent 1,000 to 10,000 entries, with CSV download). Your internal databases and MCP servers themselves stay inside the corporate network. You can start with a single user, and when rolling out to a department, management carries over under the same structure of IdP settings, canals, and allowed tools.

How it works: connecting Claude to an internal sales database in three steps

After preparing an internal MCP server and the WireCanal Agent, the connection can be configured in three steps: create an MCP canal, link it to the company's identity provider, and register the connection URL with Claude.

1. Create an MCP canal

In the WireCanal console, specify the AI service to use and the internal MCP server as the forwarding destination to create an MCP canal; a connection URL for registering with Claude is then issued. The WireCanal Agent inside the corporate network connects outward, and the communication path is open from this point on.

2. Protect the MCP canal with the company's identity provider

Assign an organizational IdP such as Google Workspace to the MCP canal and configure the domains permitted to connect. From then on, the entrance to this canal is the company's login screen.

3. Register the connection URL with Claude

Register the issued connection URL as a custom connector in Claude and sign in with your company account — setup is complete. WireCanal responds automatically to the OAuth configuration required for the connection, so there is no need to build a separate authorization server for connecting to AI services (the initial organizational IdP setup does require registering a client on your IdP's side). From there, simply talk to Claude to ask questions and run analyses — sales by category, annual sales trends, product-level sales movement — based on the latest internal data. No SQL required.

Launch campaign: the Light Plan, with one canal and a permanent subdomain, for 6,000 yen for the first year (tax included; equivalent to 500 yen per month)

To mark the start of general availability, Qualiteg is running a launch campaign for the Light Plan (one canal). Customers who sign up for an annual contract by Monday, August 31, 2026 can use the service for 6,000 yen (tax included; equivalent to 500 yen per month) for the first year. After the campaign ends (from September 1, 2026), the price for new annual contracts is 15,360 yen per year (tax included).

* The promotion applies only to the initial annual contract. Renewals from the second year onward are billed at the regular price in effect at the time of renewal. The campaign's content and duration are subject to change or termination without notice.

About WireCanal

WireCanal is a secure tunnel and MCP policy gateway that safely connects cloud AI and external services to internal systems, with its relay infrastructure operated on servers in Japan. It provides user verification through the company's identity provider and per-tool MCP permission control, making internal systems available from Claude, ChatGPT, and other AI services without opening inbound ports or building a new VPN. In addition to MCP, it supports HTTPS and TCP (RDP, SSH, databases, and more), and the WireCanal Agent connection app runs on Windows and Linux. Service starts with a free plan that requires no credit card registration.

Product, pricing, and security details:https://wirecanal.com/

About Qualiteg Inc.

Qualiteg Inc. is an AI company founded in July 2023. Under its corporate philosophy of "deepening the creativity of people around the world," Qualiteg works to amplify human creativity through cutting-edge AI technology and to realize a world that produces new forms of expression and value once beyond imagination. Rather than leaving AI as the privilege of a few specialists, the company pursues a society free of divides caused by technology gaps — one where anyone can use AI intuitively to deepen their own creativity — through the research, development, and consulting of generative AI products.

Media inquiries

Qualiteg Inc.

WireCanal PR

Contact: wirecanal-pr@qualiteg.com

* Organizational IdP integration is available on the Light Plan and above.

* When using cloud AI, the requests required to invoke MCP tools and their execution results are also transmitted to the AI service in use. Data handling is subject to each AI service's terms and settings.

* Supported AI services, available features, setup procedures, and required administrator privileges vary by each AI service's plan and workspace settings. Information is current as of August 4, 2026.

* The number of access log entries viewable in the dashboard varies by plan.

* Company names, product names, and service names mentioned are trademarks or registered trademarks of their respective owners. Third-party service names are given as connection examples and do not indicate any partnership, sponsorship, or endorsement.

Hello!

In the previous article, "TensorRT 10 × Blackwell Migration Guide (Part 1): Inference Assets Won't Run on RTX 50 — The Basics and the First Walls," we covered why existing inference assets stop working in the move to the Blackwell generation, and how to get a minimal TensorRT 10 conversion through. The problems that appeared in Part 1 actually have one thing in common.

Every one of them had the decency to stop with an error.

What is truly frightening lies beyond that. In a TensorRT migration pipeline, there exists a way to fail where:

• the build passes
• execution passes too
• the speed is properly there
• the output's shape and values look plausible at a glance
• and yet the contents are wrong

.

In this article — covering not only TensorRT's own behavior but also export mistakes, precision settings, and engine deployment mistakes — we take failures that produce wrong output without stopping on an error and, for convenience, call them "silent degradation".

It is exceptionally troublesome, because it comes to light only after the tests pass, the benchmarks post good numbers, and you have already reported "we made it N times faster."

In this article we share five instances of silent degradation we actually stepped on, in the order of symptom → why it happens → how to find it → workaround.

At the end, we consolidate a verification design for detecting these systematically. First, in the interest of honesty, here is how far each case has been substantiated.

Part 1: Mixing up input/output dtypes

Symptom

We built an engine with FP16 tactics allowed and ran it by passing PyTorch half tensors directly
(the naive implementation, from before we added the "two dtype-matching lines" included in Part 1's TRTRunner).

No errors. Good speed.But the output was completely different from PyTorch's.

How different? On a model whose output range is a bit under 0.5, the maximum absolute error was about 13.8. That is not on the level of an fp16 precision issue. It is a different animal entirely.

Why it happens

config.set_flag(trt.BuilderFlag.FP16) means "allow FP16 implementation candidates," so FP16 tactics become selectable for internal computation (some layers may still get fp32). However, the network's input and output tensor types remain as defined in the ONNX. If you exported the ONNX in fp32, the engine's inputs and outputs stay fp32.

Now, what happens when you hand it the address of a PyTorch half tensor (data_ptr())? TensorRT treats that region as an input buffer laid out with fp32 elements. Since fp16 is 2 bytes per element and fp32 is 4, not only does the interpretation of the values change — TensorRT may also read past the region PyTorch allocated. The behavior is undefined: beyond corrupted output, some environments may see CUDA errors or crashes.

And the worst part is that in our measurements, even in this state `execute_async_v3` returned `True` (success), produced no NaN, and threw no exception.

Workaround

Query the dtype the engine requires and explicitly convert before passing. The two lines we put into Part 1's TRTRunner are exactly this.

want = _TRT2TORCH[self.engine.get_tensor_dtype(name)]   # query the dtype the engine requires
t = t.to(device=self.device, dtype=want).contiguous()   # match it, then pass
if not self.ctx.set_tensor_address(name, t.data_ptr()):
    raise RuntimeError(f"set_tensor_address failed: {name}")

"I built with fp16, so the input must be fp16" does not hold.Always ask the engine.

Part 2: mode='linear' was silently being executed as nearest

Of the five, this was the hardest to find and the most surprising trap.

Symptom

With the 5D grid_sample covered in Part 1, TensorRT refuses to build (which is, in its way, kind). So what about 4D `grid_sample`? There, both the build and the execution succeed without a problem。

But compare the output against PyTorch and it is uniformly off. It is off by the same amount in fp16 and fp32. So this is not a precision issue — something is structurally different. And yet, at a glance, the output image looks perfectly plausible.

First, pin down what is happening

The minimal reproduction is just this (build_engine / TRTRunner are used unchanged from Part 1).

class PlanarWarp(nn.Module):
    """Minimal module containing a 4D grid_sample"""
    def forward(self, im, grid):
        return F.grid_sample(im, grid, align_corners=False)

torch.manual_seed(0)
im = torch.randn(1, 8, 128, 128, device="cuda")
grid = torch.rand(1, 128, 128, 2, device="cuda") * 2 - 1
model = PlanarWarp().eval().cuda()

torch.onnx.export(model, (im, grid), "planar_warp.onnx",
                  input_names=["im", "grid"], output_names=["y"],
                  opset_version=20, dynamo=True)
build_engine("planar_warp.onnx", "planar_warp.plan", fp16=False)
runner = TRTRunner("planar_warp.plan")
trt_out = runner.run({"im": im, "grid": grid})["y"]

To identify the cause, we cross-checked TensorRT's output exhaustively against every interpretation of PyTorch's `F.grid_sample`。mode (bilinear / nearest) × align_corners (True / False) × padding_mode (zeros / border / reflection) — all 12 combinations.

for mode, align, pad in itertools.product(
        ["bilinear", "nearest"], [False, True], ["zeros", "border", "reflection"]):
    ref = F.grid_sample(im, grid, mode=mode, align_corners=align, padding_mode=pad)
    print(mode, align, pad,
          (trt_out - ref).abs().max().item(),   # maximum absolute error
          torch.equal(trt_out, ref))            # bitwise equality check

Here are the results.

TensorRT's output matched the result of `mode='nearest', align_corners=False` with `torch.equal` returning True — a bitwise exact match. Meanwhile, the difference from the intended bilinear is more than 3.1. When bilinear interpolation is swapped for nearest-neighbor, the output changes only to the extent of "looking somewhat jaggy." Buried inside a warping pipeline, you will almost never catch it by eye.

Why it happens — isolating with opset 16 versus 20

Inspecting the node attributes in the ONNX file, this is what we found.

ONNX GridSample attributes: {'align_corners': 0, 'mode': 'linear', 'padding_mode': 'zeros'}

mode is, per the opset 20 specification, correctly written as linear. And here an important fact comes in.In ONNX's GridSample, the interpolation mode name was `bilinear` in opset 16 and was renamed to `linear` in opset 20. And according to NVIDIA's official documentation, linear interpolation for GridSample is supported by TensorRT. So it is not that "TensorRT does not support linear." What is suspect is how the parser interprets the new opset 20 attribute name.

So we exported the same model with the same inputs at both opset 16 and opset 20 and compared.

Same TensorRT, same model: opset 16 is correct, opset 20 comes out nearest. From this measurement we can infer with high confidence that the ONNX ingestion path of TensorRT 10.16 in our environment fails to map the mode='linear' introduced in opset 20 to linear interpolation and falls back to nearest (in both the ONNX specification and TensorRT's documentation, linear is supported and is the default-side value, so this is not a case of "nearest is the default"). We have not inspected the post-parse layer attributes or the parser implementation, so we are not asserting the exact location, and a future version may fix it.

Workarounds — three, in order of ease

Workaround 1: export at opset 16 (easiest; verified by measurement)

As the table shows, exporting at opset 16 yields correct bilinear behavior in our environment. Unless you need 5D (which requires opset 20), this is sufficient.

Workaround 2: decompose `grid_sample` into primitive operations (verified through TensorRT execution)

If you have reasons to use opset 20, or would rather not entrust the operation to TensorRT, another option is to rebuild the operation as a mathematically equivalent composition of primitive operations with proven numerics in TensorRT. Bilinear interpolation is, at heart, just "fetch the 4 neighbors and take a weighted average," so it decomposes into Floor / Clamp / Gather and basic arithmetic.

Workaround 3: a custom plugin(covered in the final installment)

import torch
import torch.nn.functional as F

def bilinear_grid_sample(im, grid, align_corners=False):
    """Reimplements the forward inference of F.grid_sample(im, grid, mode='bilinear', padding_mode='zeros').

    Swap this in only at export time. Assumes finite-valued grids;
    NaN/Inf, backward, border/reflection, and 5D inputs are out of scope.
      im:   (N, C, H, W)
      grid: (N, Hg, Wg, 2)  range [-1, 1] (x, y order)
    """
    n, c, h, w = im.shape
    gx, gy = grid[..., 0], grid[..., 1]

    # normalized coordinates [-1, 1] → pixel coordinates
    if align_corners:
        x = (gx + 1) / 2 * (w - 1)
        y = (gy + 1) / 2 * (h - 1)
    else:
        x = ((gx + 1) * w - 1) / 2
        y = ((gy + 1) * h - 1) / 2

    x0, y0 = torch.floor(x), torch.floor(y)
    x1, y1 = x0 + 1, y0 + 1

    # weights for the 4 neighbors
    wa = (x1 - x) * (y1 - y)
    wb = (x1 - x) * (y - y0)
    wc = (x - x0) * (y1 - y)
    wd = (x - x0) * (y - y0)

    # Reproducing padding_mode='zeros':
    # add 1px of zero padding, clamp indices to [-1, size], then add 1.
    # However far out of range a coordinate goes, it ends up referencing a zero-valued padding pixel.
    im_p = F.pad(im, (1, 1, 1, 1), mode="constant", value=0.0)
    x0i = (x0.clamp(-1.0, float(w)) + 1).long()
    x1i = (x1.clamp(-1.0, float(w)) + 1).long()
    y0i = (y0.clamp(-1.0, float(h)) + 1).long()
    y1i = (y1.clamp(-1.0, float(h)) + 1).long()

    wp = w + 2
    im_flat = im_p.reshape(n, c, -1)          # flatten to (N, C, (H+2)*(W+2))

    def gather(yi, xi):
        # shape it as GatherElements(dim=2) to guarantee the ONNX / TensorRT correspondence
        idx = (yi * wp + xi).reshape(n, 1, -1).expand(-1, c, -1)
        return torch.gather(im_flat, 2, idx)

    va, vb, vc, vd = gather(y0i, x0i), gather(y1i, x0i), gather(y0i, x1i), gather(y1i, x1i)

    hg, wg = grid.shape[1], grid.shape[2]
    out = (va * wa.reshape(n, 1, -1) + vb * wb.reshape(n, 1, -1)
           + vc * wc.reshape(n, 1, -1) + vd * wd.reshape(n, 1, -1))
    return out.reshape(n, c, hg, wg)

This decomposed version was verified not only with self-tests in PyTorch but all the way through ONNX export → TensorRT build → execution. The three-way comparison measured as follows.

We have confirmed that it stays correctly bilinear when running on TensorRT. It is also worth writing a self-test that includes out-of-range grids alongside it.

torch.manual_seed(0)
dev = "cuda:0"
src = torch.randn(4, 3, 256, 256, device=dev)

cases = {
    "uniform_pm1":   torch.rand(4, 256, 256, 2, device=dev) * 2 - 1,
    "outside_pm1.5": torch.rand(4, 256, 256, 2, device=dev) * 3 - 1.5,   # includes out-of-range values
}
ys, xs = torch.meshgrid(torch.linspace(-1, 1, 256, device=dev),
                        torch.linspace(-1, 1, 256, device=dev), indexing="ij")
ident = torch.stack([xs, ys], dim=-1).expand(4, -1, -1, -1)
cases["identity_plus_flow"] = ident + torch.randn(4, 256, 256, 2, device=dev) * 0.05

for name, grid in cases.items():
    ref = F.grid_sample(src, grid)          # bilinear / zeros / align_corners=False
    got = bilinear_grid_sample(src, grid)
    ma = float((ref - got).abs().max())
    print(f"[selftest] {name}: max_abs={ma:.3e} -> {'PASS' if ma < 1e-5 else 'FAIL'}")

The point is to perform this swap only inside the export wrapper. Your production model code stays completely untouched.

Part 3: Even within the same generation, engines can break across GPU models

Symptom

We took an engine built on one GPU and ran it on a different model of the same Blackwell generation (sm_120). It runs. It is fast.

But the output was corrupted.

TensorRT's log said this (quoting the latter sentence verbatim).

[TRT] [WARNING] Using an engine plan file across different models of devices is not
supported and is likely to affect performance or even cause errors or deadlock.

Before it, one more warning appears, to the effect of "this engine requires more SMs (Streaming Multiprocessors) than this device has; a deadlock is likely". Even so, `deserialize` succeeds, `execute_async_v3` returns `True`, and processing does not stop.

Why it happens

As described in Part 1, TensorRT benchmarks kernels on the actual machine at build time and picks the fastest implementations. A default-configuration engine bases its execution plan not only on Compute Capability but on multiple hardware characteristics of the GPU it was built on (SM count, shared memory, L2 cache, and so on). A different model differs in these, which can lead not only to performance loss but to execution problems — and TensorRT itself detects the mismatch and warns.

"Same Compute Capability means compatible" is wrong. Even between two sm_120 GPUs, different models must be treated as different targets.

The genuinely nasty part

We also ran an experiment reproducing this situation: build the engine on a model with more SMs, then load and run it on one with fewer. The result was this.

Both warnings appeared. Yet the output matched the result on the build machine (the measured maximum absolute error was exactly 0.0, though we did not verify as far as `torch.equal`).

In other words, with a small, simple model, ignoring the warnings still "works". That is the worst possible outcome. Once you have seen it "work" during development, the warning starts to look like mere noise. Then, when you switch to the complex production model, it bares its teeth — spewing CUDA errors while continuing to stream corrupted output. What we actually experienced was the latter.

The official documentation does note that an engine may function on a different GPU of the same architecture with only a small performance loss. Even so, for production we treat this warning as a fail-fast condition (stop when it appears). As long as the breaking conditions remain unidentified, our judgment is that one should not bet on "it works now, so it is fine."

Workarounds

1. Manage and build engines per GPU model, not per Compute Capability — splitting the storage directory by sm_120/ alone is not enough; separate by model name as well
2. At build time, save "GPU product name, Compute Capability, SM count, TensorRT / CUDA versions" as metadata (a sidecar JSON, for example); at startup, check it against the runtime environment and refuse to deserialize on mismatch — this is the effective seawall. Note that checking the return value of `execute_async_v3` cannot detect this trap — in our experiment, True was returned even under the model mismatch
3. Treat TensorRT warnings as errors in production — do not swallow the logger's WARNING; combine it with the check above and fail fast
4. If distribution to multiple models is truly unavoidable, build with hardware compatibility mode (`SAME_COMPUTE_CAPABILITY`, etc.) — the official mechanism for use across GPUs with the same Compute Capability. It constrains performance and features, however, so verify both speed and numerics separately before adopting it

Part 4: Building in fp32 still lets TF32 into the candidate pool

Symptom

A module whose accuracy fell short under FP16-allowed settings was rebuilt with an FP32 configuration that disallows FP16 tactics. Yet its agreement with PyTorch did not improve as expected. What is more, the agreement fluctuated slightly with every rebuild — sometimes clearing the acceptance bar, sometimes not.

Why it happens

For convolutions and matrix multiplications on fp32 tensors, TensorRT by default "permits" the use of TF32 tactics. TF32 is a fast format with the mantissa trimmed to 10 bits, used by Tensor Cores on Ampere and later. Not every fp32 operation becomes TF32 — ordinary fp32 implementations may still be chosen — but the assumption "built in fp32, therefore everything runs in fp32" does not hold.

A plausible cause of the per-build fluctuation is measurement noise at build time. Because TensorRT's build relies on on-device measurement, different tactics may be chosen on different builds from candidates with similar performance (the official documentation describes this as well). That said, we did not go as far as comparing which tactics were actually selected.

The workaround — and a surprising side effect

For engines that strictly verify fidelity, disable TF32 explicitly.

config = builder.create_builder_config()
config.clear_flag(trt.BuilderFlag.TF32)     # ★ remove TF32 tactics from the candidate pool

Here are the measured results.

The maximum absolute error improved by three orders of magnitude, from 2.55e-04 to 6.26e-07.

And interestingly, in this one pair of builds, the TF32-disabled side came out faster (0.874 ms → 0.772 ms). A different tactic may have been selected, but we did not isolate the cause (no comparison of the selected tactics). The difference is within reach of build-time measurement noise, so do not read this roughly 12% as a general trend. Still, it would be a shame to skip the experiment out of the assumption that "disabling TF32 always slows things down," so measure speed in that exact configuration as well. We should add that even with TF32 disabled, numerical differences from PyTorch arising from operation ordering and tactic choices do not disappear entirely.

Part 5: Swapped arguments in the ONNX export (the accident that hurt the most)

The last one is not a TensorRT problem but our own mistake. Yet it took longer to find than anything else in this series. The same shape of accident can happen to anyone, so we are sharing it. Note that this case occurred in a production model that confidentiality prevents us from publishing, and we have not run a reproduction experiment with a dummy model for this article.

Symptom

After converting a certain module to TensorRT, we hit a strange symptom: how the output broke depended on the input. Under some conditions it matched perfectly; under others it clearly degraded. Moreover, the amount of degradation grew in proportion to the "magnitude of motion" in the input.

fp16 or fp32, TF32 on or off — the degradation did not change.At that point we strongly suspected a structural problem rather than a simple precision difference.

Why it happened

torch.onnx.export had received its arguments in an order that disagreed with the parameter order of the model's forward.

# forward definition:   forward(self, feature, kp_driving, kp_source)
# what export was given:      (feature, kp_source, kp_driving)   ← 2nd and 3rd swapped
torch.onnx.export(model, (feature, kp_source, kp_driving), "m.onnx", ...)

These are positional arguments, so Python raises no complaint. The export succeeds, the ONNX file is generated, the TensorRT build passes, and it runs. Except what you end up with is an ONNX in which the roles of two inputs are swapped and frozen that way inside the model.

Because the internal computation took a difference form along the lines of "reference − input A + input B," the error comes out to exactly 2 × (the difference between the two inputs). Hence the symptom: inputs with a small difference match, and the larger the difference, the more spectacularly it breaks.

Why discovery took so long — the "self-reenactment" verification pitfall

This is the biggest problem. Our verification at the time was performed by feeding the same data to both inputs. Under that condition the two inputs are equal, so even with them swapped, the results match perfectly. The tests passed with full marks.

We call this the "self-reenactment parity" trap.If you verify equivalence only under conditions where the inputs coincide with the reference, structural bugs like swapped arguments are rendered completely harmless-looking.

How to find it — determining whether the culprit is TensorRT or the ONNX

What proved effective here was isolation using onnxruntime.

1. First, split the model into stages and binary-search for where the numbers start to diverge (comparing intermediate tensors)
2. Once the diverging subgraph is identified, run the same ONNX on onnxruntime's CPU execution
3. If onnxruntime diverges in the same direction, suspect the ONNX graph, the input binding at export, or operator specification differences first. If PyTorch and onnxruntime agree and only TensorRT diverges, prioritize TensorRT's parser, engine, and runtime-side configuration

This is not a procedure that logically convicts the culprit; it is a triage that decides where to look first (onnxruntime can have implementation differences of its own, and the issue can also be how input names or dtypes are passed on the TensorRT side). Even so, it dramatically changes the opening moves of an investigation. In our case, onnxruntime disagreed by exactly the same amount, so we examined the export side first and pinned down the positional-argument mix-up in the code.

For per-layer comparison, polygraphy's compare-all-layers mode (mark all) is convenient, but on large models it exhausts memory and dies. In practice, a binary search that carves out only the suspicious part was more dependable.

The fix — keyword binding via kwargs

Make the argument passing to export itself keyword-based. torch.onnx.export provides, separate from the positional args, an official `kwargs` parameter for passing inputs by keyword. Use it, and the very concept of tuple ordering disappears.

# Problem: args is a tuple, bound by position. Get the forward argument
#       order wrong and the export still succeeds, generating an ONNX with
#       the input roles left swapped (input_names merely attaches names;
#       it guarantees no semantic correspondence)
# Fix:  bind "name → tensor" explicitly via the kwargs parameter
torch.onnx.export(
    model,
    args=(),
    kwargs={
        "feature": feature,
        "kp_source": kp_source,
        "kp_driving": kp_driving,
    },
    f="m.onnx",
    input_names=["feature", "kp_source", "kp_driving"],
    output_names=["y"], opset_version=20, dynamo=True,
)

After the fix, the degradation vanished completely.

A verification design that catches silent degradation

Lay the five traps side by side, and shared countermeasures come into view.

1. Measure numerics before speed

Reverse the order, and the good news — "it got faster" — arrives first, reducing numerical verification to a formality.A speedup with wrong numbers has no value.

2. Do not trust agreement on random inputs — verify with real data

We were able to back this up with measurements as well. We built an engine from a block stacking LayerNorm and GELU with FP16 tactics allowed, and measured the error while varying only the input range. The results follow (the reference is PyTorch fp32; in both the "FP16 allowed / disallowed" columns, TF32 remains at its default, i.e., allowed).

As the input scale widened, the FP16 configuration's maximum absolute error grew substantially (though the growth is not simple proportionality to the input multiplier, because LayerNorm normalizes the scale). The nRMSE, meanwhile, actually fell from 8.9e-04 to 2.5e-04 — at the very least it does not worsen with input scale. In other words, absolute error and normalized error paint completely different pictures of the same result. What this tells us is that an acceptance bar of "absolute error below X" calibrated only on standard normal randoms is easily broken at real-data ranges.

In our real model, moreover, this went beyond mere error growth and appeared as every output turning NaN the moment real data went in (an intermediate value just before a LayerNorm exceeded fp16's representable range). The remedy was to disallow FP16 tactics for that module and build it with an FP32 configuration.

3. Do not verify equivalence by "self-reenactment"

As in Part 5, tests confined to conditions where the input coincides with the reference hide structural bugs completely.Make sure your verification data includes the large-difference cases that can occur in real operation.

4. Triangulate the culprit with PyTorch / onnxruntime / TensorRT

• PyTorch and onnxruntime agree, only TensorRT diverges → suspect TensorRT's semantics and kernels (Part 2)
• onnxruntime diverges as well → suspect the ONNX — that is, the export (Part 5)

Being able to make this cut changes investigation time by an order of magnitude.

5. Carve out the suspicious operation and test it alone

Chasing a vague "somehow off" across a whole model is inefficient. Part 2 was settled in a single stroke by carving out one grid_sample and cross-checking it exhaustively.

6. Toggle precision and watch whether the symptom changes

If switching among fp16 / fp32 / TF32 barely changes the amount of degradation, prioritize structural problems — input binding, operator semantics — over rounding error. It is not a proof, but as a signal for setting investigation priorities it served us extremely well.

7. Build in mechanisms that refuse to swallow failure

execute_async_v3 return-value checks, engine-to-device consistency checks, and a design that refuses to start when a component is missing.Never allowing a state that "looks normal while partly broken" is the last seawall against silent degradation.

What we have not yet confirmed

• The root location of `mode='linear'` becoming nearest — from the opset 16/20 comparison we strongly infer interpretation at the parser stage, but we have not examined the parser source code or the layer attributes immediately after parsing. Verification also covered a single version, TensorRT 10.16.1.11, and a future version may fix it
• The conditions under which cross-model engines break — with this article's dummy model, the warnings appeared yet the outputs matched. We have not identified at what scale or structure breakage begins
• The same class of problems under int8 quantization — this article covers fp16 / fp32 / TF32 only

Summary

Summed up in one sentence: "in a TensorRT migration pipeline, neither 'the build passed' nor 'it ran' nor 'it got faster' guarantees in any way that the output is correct.".

That is exactly why we measure numerics before speed, measure with real data rather than randoms, triangulate the culprit when things diverge, and build in mechanisms that refuse to swallow failure. We consider these the minimum conditions for putting TensorRT into production.

Next time (the final installment) takes on the biggest homework left over from Part 1: getting 5D grid_sample, which TensorRT does not natively support, through with a custom plugin. Building it against the new CUDA / TensorRT generation involves quite a few hurdles: what to do when your local nvcc cannot generate code for Blackwell; the problem that pip-installed TensorRT ships without the development headers; and the plugin's fp16 kernel breaking down numerically — once again, a story bordering on silent degradation.

See you next time!

References (primary sources)

• NVIDIA TensorRT documentation (Developer Guide / Release Notes)
• ONNX Operators: GridSample (specification of mode / align_corners / padding_mode)
• PyTorch: torch.nn.functional.grid_sample
• NVIDIA/TensorRT issue #3890: request for native 5D GridSample support
• NVIDIA: TensorFloat-32 (TF32) explained
• polygraphy (TensorRT debugging and comparison tool)
• Related article from Qualiteg: 2026 NVIDIA GPU quick-lookup tool

Hello!

On July 24, 2026, Anthropic released Claude Opus 5.

That is a generational change just two months after Opus 4.8 (released May 28). The pace here keeps accelerating.

Now, this blog has followed the Opus line through the Claude Opus 4.7 Complete Guide、Claude Opus 4.8 Complete Guide — but this Opus 5 is fundamentally different in structure from those two updates within 4.x.

What is different? First of all, Opus 5 is not the top-of-the-line model。

Anthropic's lineup already has, at the top, the June 9, 2026 release Claude Fable 5, and Opus 5 was slotted in below it and above Sonnet 5 — an upper-middle position.

The era when the name Opus meant "top tier" ended with the arrival of Fable 5.

On top of that, Anthropic positions Opus 5 as "a model that approaches Fable 5's frontier-class intelligence at half the price" — and has even published benchmark scores where it beats Fable 5.

Pricing is unchanged from Opus 4.8: $5 input / $25 output.

In other words, this is a character no previous Opus has had: not the top model, but the practical workhorse.

And remember: Fable 5 went through the turmoil of being suspended on June 12 by order of the U.S. government and returning on July 1 (we covered the full story in The Fable 5 / Mythos 5 Suspension: What Happened).

The "safety classifiers" at the center of that incident have, in fact, been added to Opus 5 as well。

Something Opus 4.8 did not have is now on board — a change you absolutely need to know about when migrating (covered in detail in Chapter 7).

This is a long piece, so there is no need to read it start to finish. Skim the chapters that interest you. Sources are collected at the end.

The intended readers: engineers considering a migration from Opus 4.8, tech leads deciding between Fable 5 and Opus 5, lead engineers running Claude Code across a team, and anyone optimizing API costs.

Table of Contents

Part 1 What Claude Opus 5 Is

• 1. Where Opus 5 sits in the Claude 5 family
• 2. Claude Opus 5 core specifications
• 3. What "beating Fable 5" actually means in the benchmarks
• 4. Two breaking API changes around thinking
• 5. Migration checklist (from the official guide)
• 6. New and related features

Part 2 Notes for API Use

• 7. Safety classifiers that 4.8 did not have
• 8. Choosing effort levels
• 9. Task budgets (beta): giving agents a sense of budget
• 10. Prompting to get the most out of Opus 5

Part 3 Opus 5 in Claude Code

• 11. Prerequisites for using Opus 5 in Claude Code
• 12. Watch the carry-over behavior of Claude Code's effort setting
• 13. What changed in Fast mode
• 14. What happens when you get flagged: the automatic model fallback
• 15. The 1M context and opusplan
• 16. A rhythm for day-to-day operation

The updates from 4.8, at a glance

From Opus 4.8, which we covered last time, the main changes in Opus 5 are the following.

Details come in each chapter; first, the overall picture.

Not incremental — a "step-change"

The official documentation itself describes the gain over 4.8 as "a step-change, not incremental." The biggest gains are in deep reasoning, long-horizon agentic tasks, and test-time compute scaling (Chapter 3).

thinking is ON by default

On 4.8, requests without the thinking field ran without thinking; on Opus 5, the same requests run with thinking.max_tokens settings need revisiting (Chapter 4).

Disabling thinking is possible only at effort `high` or below (breaking change)

thinking: {type: "disabled"} combined with effort xhigh/max returns a 400 error (Chapter 4).

The recommended starting effort has changed

The API default remains high, unchanged. What changed is the recommended starting point: from 4.7/4.8's "start coding with xhigh" to Opus 5's "start with high and step up to xhigh only for demanding work" — with active use of low/medium now spelled out explicitly (Chapter 8).

No Web Fetch server tool

The Web Fetch tool available on Opus 4.8 is not available on Opus 5 (it is separate from web search). API integrations using it need an alternative design (Chapter 5).

Safety classifiers on board

Model-specific cyber and biology classifiers absent from 4.8 are present on Opus 5. Interventions are said to be about 85% less frequent than Fable 5's, but for 4.8 migrants this is a new constraint (Chapter 7).

Knowledge cutoff of May 2026

Newer than Fable 5 and Sonnet 5 (both January 2026) — the freshest knowledge in the current lineup (Chapter 2).

New and related features

Mid-conversation tool changes (beta), the fallbacks parameter's "default" mode (beta), Task budgets (beta), and the minimum cacheable length dropping to 512 tokens, among others (Chapter 6, Chapter 7, Chapter 9).

Pricing unchanged

$5 input / $25 output, as before. Fast mode is also $10/$50, the same as Opus 4.8 (Chapter 2).

No Priority Tier

Priority Tier, usable with Opus 4.8, is not supported on Opus 5. Organizations with commitments should watch their capacity planning (Chapter 5).

One more note: code written for Opus 4.8 will not necessarily run with just a model ID swap。

With two behavior changes — the thinking default and the restriction on disabling thinking — we recommend running through the migration checklist (Chapter 5) once.

Part 1: What Claude Opus 5 Is

1. Where Opus 5 sits in the Claude 5 family

First, a recap of how Anthropic's lineup changed over the roughly two months from June 2026.

In that period, Anthropic shipped four models in rapid succession.

• June 9 Claude Fable 5 / Claude Mythos 5 released (the newly created top "Mythos class." Benchmark deep-dive)
• June 12 Both models suspended worldwide under a U.S. Commerce Department export-control directive (full story)
• June 26 Mythos 5 partially restored, limited to vetted U.S. organizations
• June 30 Export restrictions lifted. Same day, Claude Sonnet 5 released (introductory pricing $2/$10 through August 31, 2026; $3/$15 thereafter)
• July 1 Fable 5 fully restored on claude.ai, Claude Code, and elsewhere
• July 24 Claude Opus 5 released. Same day, Opus 4.7's Fast mode discontinued

As a result, the current lineup is a four-tier structure, from the top: Fable 5 ($10/$50), Opus 5 ($5/$25), Sonnet 5 ($3/$15), and Haiku 4.5 ($1/$5).

Opus 5's official positioning is "for complex agentic coding and enterprise work."

Anthropic made it the new default model for Claude Max and the top model available on the Pro plan.

The arrangement Anthropic itself recommends: Opus 5 as the daily mainstay, Fable 5 only for truly frontier work.

(Source: Anthropic's official model list. Prices and cutoffs as of July 27, 2026.)

Easily overlooked but important is the knowledge cutoff.

Opus 5's reliable knowledge cutoff is May 2026,
four months newer than Fable 5 and Sonnet 5's January 2026 — the newest in the current lineup.

When working with newer libraries and APIs, it may hold fresher knowledge than its "superior," Fable 5.

A newer cutoff does not guarantee accuracy on any specific detail, though. As noted below, Opus 5 lacks the Web Fetch server tool (Chapter 5), so supplement fresh information via the web search tool or your own retrieval, and keep cross-checking against official documentation.

For a comparison of LLM vendors' pricing, see our LLM API price roundup; on Fable 5's cost problem, see Fable 5: Costs and Outlook as well.

User complaints that Fable 5 "burns tokens aggressively" have been widely reported; per Fortune, Opus 5 leading with token efficiency is in part an answer to that complaint.

OpenAI's GPT-5.6, released July 9, also markets economical token consumption — the axis of competition is shifting from "intelligence" to "the balance of intelligence and cost."

2. Claude Opus 5 core specifications

Much carries over from 4.8, but two points signal a generational shift: "1M is the default and the maximum, with no smaller-context variant at all," and "thinking is ON by default."

Note that claude-opus-5 is a fixed snapshot, not an evergreen pointer (the dateless-ID scheme in place since the Claude 4.6 generation).

3. What "beating Fable 5" actually means in the benchmarks

Anthropic's launch announcement claims Opus 5 beats Fable 5 on several benchmarks.

In this blog's usual style, we will go through them while distinguishing who measured each number.

Frontier-Bench v0.1 (run internally by Anthropic)

In the System Card's evaluation summary table, Opus 5 tops the compared models at 43.3%, well above Fable 5's 33.7% and Opus 4.8's 18.7% — more than double 4.8. Cost per task is also said to be lower.

The official launch chart also shows multiple values by effort level, so 43.3% is not necessarily Opus 5's per-effort peak.

ARC-AGI 3 (verified external evaluation by ARC Prize)

Opus 5 scored, at high effort, 30.16% (rounded to 30.2% at publication) — about 3.9 times the 7.8% of the runner-up, GPT-5.6 Sol (Anthropic's announcement modestly calls it "3×").

Opus 4.8 was at 1.5% — essentially zero. Note that because the evaluation window was short, Opus 5's max has not been evaluated on ARC-AGI 3.

CursorBench 3.2 (external evaluation by Cursor)

max effort: Opus 5 at 70.0%, Fable 5 at 70.5% — a 0.5-point gap. Average cost per task is $8.23 for Opus 5 versus $17.32 for Fable 5, more than double.

Cursor itself notes, however, that small score differences may not be statistically significant.

OSWorld 2.0 (computer use; Anthropic evaluation, listed in the System Card)

70.57% (mean first-attempt success over 5 runs), up sharply from Opus 4.8's 55.7%. It is said to beat Fable 5's best result at roughly one-third the cost.

Zapier AutomationBench (results from an external evaluation provider)

A pass rate about 1.5× the runner-up model's (at comparable cost per task).

Life sciences (Anthropic's own evaluations)

+10.2 points over Opus 4.8 on organic chemistry tasks and +7.7 on protein prediction. Anthropic is pitching Opus 5 as "the most capable generally available model for scientific research."

Two caveats.

First, the numbers fall into three kinds of attribution: (a) evaluations Anthropic ran itself (Frontier-Bench, OSWorld, life sciences), (b) external evaluations run and published by benchmark providers (ARC Prize, Cursor, Zapier), and (c) press coverage of those.

(b) is not self-reporting, but neither is it independent replication by a third-party community.

Second, "beating Fable 5" does not apply across all benchmarks.

Anthropic itself continues to recommend Fable 5 for the hardest long-horizon autonomous tasks, and states plainly that in cybersecurity exploit capability Opus 5 "does not reach Mythos 5."

How it fares in blind voting (Arena-style) and in our LLM rankings will be settled once independent evaluations are in.

TechCrunch highlights the trait Anthropic emphasized: Opus 5 "verifies its own work and iterates carefully until it succeeds."

No premature victory declarations; verify, then fix. This is the change that symbolizes Opus 5's character — two sides of the same coin as the prompting changes described later (removing re-verification instructions that duplicate its self-verification; Chapter 5 and Chapter 10).

4. Two breaking API changes around thinking

Opus 4.7 to 4.8 was a "swap the model ID and go" migration.

This time there are two breaking changes in API parameters. On top of that, on the feature side, there are the effective incompatibilities noted above — no Web Fetch and no Priority Tier (covered in Chapter 5's checklist).

Change 1: thinking is now ON by default

On Opus 4.8, requests without the thinking field ran without thinking.

On Opus 5, the same requests run with thinking (adaptive thinking: the model decides, turn by turn, whether and how deeply to think).

Nothing changed on the wire — explicitly sending thinking: {type: "adaptive"} behaves the same.

The practical caution concerns max_tokens.

max_tokens is a hard limit on the combined total of thinking and visible text. Move a workload that ran thinking-free on 4.8 over as-is, and the added thinking tokens can truncate the visible text mid-response (stop_reason: "max_tokens").

Revisit anywhere you set a smallish max_tokens on the assumption of no thinking.

Change 2: disabling thinking now works only at effort `high` or below

If you want the old behavior, you can pass thinking: {type: "disabled"}, but Opus 5 accepts it only when the effort level is high or below.

thinking: {type: "disabled"} combined with effort xhigh/max returns a 400 error.

This combination was accepted on 4.8, so audit your requests for it before migrating.

The check runs on every single request, so "the first half of the conversation was fine, then a 400 the moment we raised effort mid-conversation" can happen.

# Passes on 4.8 but returns a 400 error on Opus 5
client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    thinking={"type": "disabled"},
    output_config={"effort": "xhigh"},   # disabled cannot be combined with xhigh
    messages=[{"role": "user", "content": "..."}],
)

# Fix A: drop the thinking field (back to the default, thinking on)
client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    output_config={"effort": "xhigh"},   # thinking is ON by default
    messages=[{"role": "user", "content": "..."}],
)

# Fix B: keep thinking disabled and lower effort to high or below
client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    thinking={"type": "disabled"},
    output_config={"effort": "high"},    # high / medium / low are acceptable
    messages=[{"role": "user", "content": "..."}],
)

The official documentation also spells out side effects of disabling thinking.

Without thinking, Opus 5 can occasionally write a tool call as body text instead of a tool_use block. The call is never executed, and in agent loops the leaked text remains in the conversation history and affects subsequent turns.

<thinking> and similar internal XML tags can also end up mixed into visible output.

The official recommendation: keep thinking enabled and control cost with lower effort.

For most tasks, "thinking on at low effort" is said to beat "thinking off" at comparable cost. Prompt-level mitigations for integrations that truly must disable it are covered in Chapter 10.

5. Migration checklist (from the official guide)

The official migration guide's checklist, reordered for practical workflow.

1. Update the model name from claude-opus-4-8 to claude-opus-5 in your code
2. Audit requests that use the Web Fetch server tool. It is unavailable on Opus 5, so design alternatives: the web search tool, client-side fetching, or routing just those requests to another model
3. thinking-less workloads: audit anything you ran without the field (it runs with thinking on Opus 5). Either revisit max_tokens, or keep the old behavior with thinking: {type: "disabled"} plus effort high or below
4. thinking: {type: "disabled"} with effort xhigh/max: fix requests combining them (they will return 400)
5. Re-evaluate effort from a blank slate. Do not carry settings over from the old model; re-run an effort sweep on your own evals.low/medium are worth trying aggressively as cost/latency controls, and max is worth trying on capability-first tasks.xhigh/max runs should start with max_tokens at 64k or higher
6. Check prompts near the cache boundary (cacheable from 512 tokens; 4.8 required 1,024)
7. stop_reason: "refusal" handling should be reviewed; consider adopting fallbacks: "default" (beta) (Chapter 7)
8. Organizations with Priority Tier commitments should plan capacity separately. Priority Tier is unsupported on Opus 5 and remains available on Opus 4.8
9. For agentic workloads, consider Task budgets (beta) and mid-conversation tool changes (beta) (Chapter 9 and Chapter 6)
10. Re-tune prompts for length and verbosity. Opus 5 tends toward longer visible responses and longer deliverable documents, and lowering effort does not shorten them (Chapter 10)
11. Remove re-verification instructions written for older models ("add a final verification step," "double-check," "verify with a subagent," etc.). Opus 5 self-verifies unprompted, so leaving them in wastes tokens on redundant verification. Do not remove project-specific acceptance criteria such as running designated tests, lint, and type checks (Chapter 10)
12. Re-baseline cost and latency on your own workloads

Of these, items 5, 10, and 11 are of the "won't break, but becomes suboptimal" kind — skipping them causes no errors.

That said, item 11 (removing redundant re-verification instructions) is one of the few the official guide flatly describes as "token savings with no quality loss," so it deserves higher priority.

6. New and related features

A roundup of the new and related features available with Opus 5.

This includes not only features added alongside Opus 5 but also existing features that now support it. Details in the relevant chapters.

Mid-conversation tool changes is an unglamorous change, but one that matters to agent developers.

Previously, changing the tool list invalidated the cached prefix, so the standard practice was to hand over every tool at the start of the session.

With the beta header mid-conversation-tool-changes-2026-07-01, you can progressively expose or retire tools as the task advances while keeping cache hits for earlier turns.

It is easiest understood as the sequel to the mid-conversation system messages introduced in 4.8.

Part 2: Notes for API Use

7. Safety classifiers that 4.8 did not have

First, pin down the key fact.

Model-specific safety classifiers of the same family as Fable 5's (cybersecurity and biology) are now on Opus 5 as well.

And Opus 4.8 does not carry these classifiers。

The only models the official documentation describes as classifier-equipped are Fable 5 and Opus 5. The very fact that flagged requests are routed to Opus 4.8 corroborates that 4.8 lacks the same classifiers.

To be fair, the 4.8 era also saw false positives from the platform-side Usage Policy safeguards.Our June article covered exactly that — a different layer from the model-specific classifiers.

So for users migrating from 4.8, this is a newly added constraint.

The accurate reading is not "it got safer" but "a classifier has been attached — though it intervenes far less often than Fable 5's."

That said, in the context of having just lived through work stopping during the Fable 5 suspension, the differences from Fable 5 carry practical meaning too. Four points.

(1) Alignment itself is the best in the line's history

In Anthropic's automated behavioral audit, Opus 5's misaligned-behavior score is 2.3 (lower is better).

That is the best among recent models including Opus 4.8, Sonnet 5, and Fable 5; Anthropic calls it "the most aligned Opus, and the hardest to steer toward misuse" (bear in mind this is self-evaluation).

(2) Classifier interventions about 85% fewer than Fable 5's (not zero)

Because of its high cybersecurity capability, Fable 5 runs strong safety classifiers at all times, and there were complaints that this obstructed even legitimate security work.

For Opus 5, Anthropic says it deliberately capped raw exploit capability below Mythos 5 and reduced classifier intervention frequency to "about 85% less than Fable 5."

Put the other way around: model-classifier interventions that never happened on 4.8 will happen at some frequency on Opus 5.

The line is drawn so that vulnerability discovery against source code is allowed, while vulnerability scanning against binaries and exploit generation are blocked.

For Project Glasswing — the special defensive-security framework under which Mythos 5 is offered — see our Glasswing explainer.

Companies and researchers already in the Cyber Verification Program (CVP) get immediate access to Opus 5 with relaxed restrictions.

(3) Automatic fallback on flags (API side)

When the classifier flags a request, there is now an option to route it automatically to another model instead of ending in an error.

fallbacks parameter's new "default" mode (fallbacks: "default") automatically selects the Anthropic-recommended fallback model per refusal category. Opus 5's cyber refusals, for example, flow to Opus 4.8.

The existing explicit model-list form remains available, and the "default" mode requires the server-side-fallback-2026-07-01 beta header.

(4) Data retention differences (confirmed in official documentation)

Anthropic's official "API and data retention" documentation names Fable 5 and Mythos 5 as "Covered Models" subject to mandatory 30-day data retention; those two cannot be used under ZDR (zero data retention). They do not appear in Claude Code's model picker in ZDR environments either.

Opus 5 is not among the Covered Models, and the launch announcement imposes no model-specific data-retention requirement on generally accessible Opus 5.

Organizations with ZDR agreements on the Claude API can therefore use Opus 5 within the scope of ZDR-eligible features.

Note that ZDR is an organization-level agreement; consumer plans like Pro/Max are not themselves covered. Features such as the Batch API and code execution sit outside ZDR, so check your contract terms and per-feature ZDR eligibility for actual coverage.

In sum, this is a change whose assessment depends on where you stand.

Seen from 4.8, adding the classifiers is purely an added constraint. Workloads touching security or biology may see processing that passed on 4.8 get stopped (staying on 4.8 is an option; Chapter 16).

Seen from Fable 5, though, interventions are about 85% fewer, there is no model-specific 30-day retention requirement, and cyber refusals get automatic fallback — clearly easier to live with.

Caveats remain: the automatic fallback is opt-in on the API, and Opus 5's biology category has no fallback destination (Chapter 14). Nor does any of this guarantee against future regulatory action.

8. Choosing effort levels

Since Opus 4.7, effort has remained the single most important lever for getting the most out of Opus.

With Opus 5, the wording of the official guide's recommendation changed.

To clear up a common misreading first: the API default has stayed high ever since 4.7. What changed is the description of the recommended starting point.

The 4.7/4.8 official guides said plainly: "Start with xhigh for coding and agentic use cases."

The Opus 5 guide instead reads: "Start with high, the default, and adjust based on evals; step up to xhigh for demanding coding and agentic work."

In table form.

The background to the change is the improved efficiency of Opus 5's lower effort levels.

The official guide says "low and medium deliver strong quality at a small fraction of the tokens and latency of the upper settings," and goes as far as "use low/medium liberally as the primary control over cost and response time."

Carry the 4.8-era habit of "default to xhigh" over unconditionally, and you can end up doubling tokens for a barely measurable quality gain.

The official recommendation is unambiguous: do not carry effort settings over from the old model; re-run sweeps on your own evals.

At the top end, meanwhile, max is more worth trying than in the 4.8 era, because Opus 5 converts additional effort into results (test-time compute scaling) better than any previous Opus.

The CursorBench run that came within 0.5 points of Fable 5's peak was also at max effort.

xhigh/max runs need headroom for thinking and tool calls, so start max_tokens at 64k or higher and tune from there.

Two effort-specific cautions.

effort does not control visible response length

Lowering effort on Opus 5 reduces thinking, but does not reliably shorten user-facing responses. If you want shorter responses, say so in the prompt (Chapter 10).

do not change effort mid-conversation

effort affects the rendered prompt, so changing the value between requests invalidates the prompt cache for the preceding prefix. In long cache-dependent sessions, fix the value you chose at the start.

Also, the restriction that thinking: {type: "disabled"} can only be combined with high or below (Chapter 4) affects effort selection too.

The configuration "thinking off but effort up" simply cannot be built on Opus 5.

9. Task budgets (beta): giving agents a sense of budget

Among the beta features usable with Opus 5, the most interesting one for agent-infrastructure developers is Task budgets.

The beta header is task-budgets-2026-03-13. Supported on Opus 5 / Fable 5 / Mythos 5 / Opus 4.8 / 4.7; Sonnet 5 is not supported. It is not an Opus 5-exclusive new feature but an existing beta gaining Opus 5 support.

output_config.task_budget given a {type: "tokens", total: N} conveys to the model a token budget for the entire agentic loop — thinking, tool calls, tool results, and output included.

Watching a server-injected countdown, the model prioritizes work against the remaining budget and lands gracefully before it runs out — summarizing findings, reporting progress, and so on.

with client.beta.messages.stream(
    model="claude-opus-5",
    max_tokens=128000,
    output_config={
        "effort": "high",
        "task_budget": {"type": "tokens", "total": 64000},
    },
    messages=[{"role": "user", "content": "Review the codebase and propose a refactor plan."}],
    betas=["task-budgets-2026-03-13"],
) as stream:
    response = stream.get_final_message()

Let's keep the concepts straight.

Where effort controls the depth of reasoning at each step, task_budget controls the amount of work across the whole loop.

Effort for depth, task budget for volume, and the per-request hard cap of max_tokens — a three-layer structure.

Four operational cautions.

The budget is advisory, not enforced

In the middle of work where interruption would be more destructive, it can overrun. Hard caps still go through max_tokens.

The minimum is 20,000 tokens

Anything below returns a 400 error.

Budgets that are too small trigger refusal-like behavior

Hand a 20k-token budget to an hours-scale task and the model may decline the task outright, over-trim scope, or wrap up early.

If you see mysterious refusals or early exits after setting a budget, raise the budget before suspecting any other parameter.

Only the model sees the countdown

There is no remaining-budget field in API responses.

Decrementing remaining client-side every turn breaks the cache, and the double counting makes the budget appear to shrink faster than reality — a cause of premature wrap-ups.

The basic pattern: set total once on the first request and leave the rest to the server.remaining is used only when compaction (context compression) rewrites the history and you need to carry over the tokens already consumed.

The official recommendation on setting budgets: do not guess.

First run representative tasks without task_budget, measure the distribution of loop-wide usage.output_tokens and tool-result tokens, then start around p99 and tune.

Note that Task budgets are not available in Claude Code / Cowork. This is a feature for agent platforms calling the Messages API directly (Claude Code's workflow feature has its own similar budget mechanism).

10. Prompting to get the most out of Opus 5

Opus 5 runs well as-is even on prompts written for 4.8.

But the official prompting guide names five behaviors that "tend to need adjustment." All are cases where the model now does things on its own, making legacy instructions excessive.

(1) Responses and deliverables got longer

Both conversational responses and reports or Markdown documents written to disk tend to run longer than 4.8's.

As noted, lowering effort does not shorten them; state it in the prompt.

Keep responses focused, brief, and concise. Keep disclaimers and caveats short, and spend most of the response on the main answer. When asked to explain something, give a high-level summary unless an in-depth explanation is specifically requested.

In long system prompts, layering a short reminder near the end (something like <tone_preference>Keep outputs reasonably concise.</tone_preference>) is said to work.

For document length, add a calibration instruction: cover the substance, and do not pad with filler sections, redundant summaries, or boilerplate.

(2) More progress narration

During agentic work it actively announces what it is about to do, and output per message runs longer.

Specifying frequency and format explicitly works well; the official example is this.

Before your first tool call, say in one sentence what you're about to do. While working, give a brief update only when you find something important or change direction. When you finish, lead with the outcome: your first sentence should answer "what happened" or "what did you find," with supporting detail after it for readers who want it.

Conversely, if you want more narration or a different style, showing positive examples of the desired form beats prohibitions.

(3) It self-verifies and self-corrects on its own

This is Opus 5's biggest character change.

The official recommendation is to delete re-verification instructions that duplicate the model's self-verification — "add a final verification step," "double-check," "verify with a subagent," and the like.

They overlap with the model's own verification behavior, producing over-verification: added cost with no quality gain.

But the only safe deletions are these duplicated instructions.

Project-specific acceptance criteria — "run the designated test suite after changes," "pass lint and type checks" — are completion conditions, not duplicated self-checks. Keep them.

It has also become more willing to voice corrections to its own earlier statements. For user-facing products, the guide shows a narrowing instruction: correct only errors that change the user's code, conclusions, or decisions concisely, and fix inconsequential misstatements silently.

(4) It tends to widen task scope

It may add steps you did not ask for, or reshape the task based on its own judgment of how things "really ought to be."

For narrow tasks, bound the scope explicitly.

Deliver what was asked, at the scope intended. Make routine judgment calls yourself, and check in only when different readings of the request would lead to materially different work. If the request seems mistaken or a better approach exists, say so in a sentence and continue with the task as asked rather than quietly narrowing, widening, or transforming it.

(5) More delegation to subagents

On multi-agent platforms, it spawns subagents more aggressively than 4.8.

A strength for genuinely independent, large work; for small tasks it multiplies cost and time. Recommended: explicit delegation criteria, or a deterministic cap on spawn count.

The official guide includes example lines like "do not delegate work that finishes in a few tool calls" and "do not use subagents to verify your own work."

Amusingly, the 4.8 prompting guide said the exact opposite, with examples nudging toward more subagents.

Default behavior swings like a pendulum between generations, so harness-side steering prompts need review every generation.

Two more strengths deserve mention.

Code review

Opus 5 is described as "finding real bugs at a high rate per pass, with the additional findings mostly non-spurious."

And accuracy holds even at low effort, enabling a two-stage setup: a fast pass at review time, a more thorough pass later.

As in the 4.8 era, an instruction like "report high-severity only" is obeyed literally and reduces reporting — so the standard practice remains: have it report everything at the discovery stage, and filter in a separate pass.

Vision (image understanding)

Understanding of charts, documents, and figures — and visual reproduction of UI and front-end work — has improved.

We recommend re-validating image workarounds added for older models.

Giving it tools to iteratively analyze, crop, and inspect images is said to be more cost-effective than piling on thinking.

Part 3: Opus 5 in Claude Code

11. Prerequisites for using Opus 5 in Claude Code

If you have never used Claude Code, start with our Claude Code primer (CLI and web).

Here we cover the operational prerequisites.

Version

Opus 5 requires Claude Code v2.1.219 or later (Sonnet 5: v2.1.197+; Opus 4.8: v2.1.154+).claude update to update.

Alias resolution

opusAlias resolution has advanced considerably since the 4.8 era.

Not only on the Anthropic API but also on Claude Platform on AWS, Amazon Bedrock, and Google Cloud's Agent Platform, opus resolves to Opus 5 (in the 4.8 era this varied by provider).

The exception is Microsoft Foundry, where opus remains Opus 4.6. On Foundry, use the full model name or ANTHROPIC_DEFAULT_OPUS_MODEL.

The `best` alias

An alias that resolves to Fable 5 if your organization has access, and otherwise to the latest Opus: best is available.fableDirect alias specification also works.

default (when no model is specified) resolves as follows.

Note that the Pro plan's default is now Sonnet 5.

Fable 5 is never the default for any account type (/model fable or similar explicit selection is required).

To pin a version, the standard practice is unchanged: use the full model name (claude-opus-5) or the environment variable (ANTHROPIC_DEFAULT_OPUS_MODEL) instead of an alias.

12. Watch the carry-over behavior of Claude Code's effort setting

The ways to control effort in Claude Code (/effort, --effort, CLAUDE_CODE_EFFORT_LEVEL, the settings-file effortLevel, and skill/subagent frontmatter) are the same as in the 4.8 era.

The thing to watch on Opus 5 is the carry-over behavior of the effort setting.

On Fable 5, Opus 4.8, and 4.7, the first use of a model force-applied the model default even if you had set a level on another model earlier (behavior we flagged in the 4.8 guide).

Opus 5 has no such hold: a previously set level carries over as-is.

So someone who lived at xhigh on 4.8 will, after switching to Opus 5, keep running at xhigh without noticing.

Chapter 8 explains that the official recommendation on Opus 5 is to start at high and adjust as needed — so right after switching, check the current value with /effort and re-choose deliberately.

You can also see the current effort in the session header's "with xxx effort" display.

Everything else is as before.

max is session-only (cannot be saved to settings files); ultracode is a Claude Code-side setting meaning "xhigh plus dynamic-workflow orchestration," not a model effort level; and for a one-off deep think you write ultrathink in the prompt — for these, see the 4.8 guide's Chapter 9 and Chapter 11.

Dynamic Workflows themselves work as detailed in the 4.8 guide; no mechanism changes in Opus 5.

13. What changed in Fast mode

/fast-toggled Fast mode (same model, same quality, up to 2.5× faster) covers Opus 5 by default from v2.1.219.

Pricing is $10 input / $50 output for both Opus 5 and Opus 4.8, keeping the level set by the 4.8-era price cut (flat across the whole 1M window).

The cautions carry over from the 4.8 era, with one new development.

• Opus 4.7's Fast mode was removed on July 24, 2026 (Opus 5 launch day). Moreover, Claude Code still treats 4.7 as Fast mode-eligible, so a 4.7 session with Fast mode left ON gets its requests rejected by the API (an error, not a fallback to standard speed). If you used Fast mode on 4.7, switch to Opus 5 or 4.8
• On subscription plans (Pro/Max/Team/Enterprise), it is usage-credits only. It is not part of the plan's normal allowance; Fast mode rates apply from the first token
• Turning it ON mid-conversation charges the uncached input rate once for the entire conversation context at that point, so if you use it, use it from session start
• Unavailable on Bedrock / Google Cloud / Foundry / Claude Platform on AWS. The VS Code extension does not support it (CLI only)

14. What happens when you get flagged: the automatic model fallback

Chapter 7 described the API-side fallbacks; separately from that, Claude Code has gained its own category-based automatic fallback (v2.1.219+).

Fable 5 and Opus 5 run with cybersecurity/biology safety classifiers, and when a request is flagged, the behavior is as follows.

As the table shows, Opus 4.8 serves as the fallback precisely because it lacks the classifiers.

After a fallback the session continues on that model, so to go back, use /model.

Three behaviors worth knowing operationally.

It can fire on the very first request of a session

The first request carries workspace context such as CLAUDE.md and git status.

In repositories with security or biology content, the classifier can react even when you asked for nothing unusual.

For isolation, claude --safe-mode (start with CLAUDE.md, skills, MCP, and hooks disabled) is useful.

If you don't want automatic switching, it is configurable

/config lets you turn off "switch models when a message is flagged."

When flagged, the session pauses and offers a choice: switch to the fallback model, or edit the prompt and retry on the current model.

In penetration testing, CTFs, and biology-adjacent codebases, frequent firing is "working as intended"

It does not mean your account has been flagged.

For organizations that need Fable-class capability in these domains, Anthropic's trusted access programs (CVP and others) are the indicated route.

Claude Code has also had confusing bugs around refusals in the past. Our guide to the "usage policy violation" message is collected in this article.

15. The 1M context and opusplan

Handling of the 1M context carries over the 4.8-era framework, and with "1M as default and maximum" on Opus 5 it got even simpler.

• On the Anthropic API, Opus 5 (and Opus 4.7+, Sonnet 5, Fable 5) always runs with the 1M window
• On Max / Team / Enterprise plans, Opus upgrades to 1M automatically with no extra configuration (included in the subscription; no premium billing beyond 200k)
• To disable it entirely, use CLAUDE_CODE_DISABLE_1M_CONTEXT=1

opusplan (an alias that switches automatically: plan mode on opus, execution on the sonnet alias) now means, on the Anthropic API, that Opus 5 plans and Sonnet 5 executes.

The actual combination follows the per-provider alias resolution explained in Chapter 11. Note that it becomes Opus 5 + Sonnet 4.6 on Claude Platform on AWS, Opus 5 + Sonnet 4.5 on Bedrock and Google Cloud Agent Platform, and Opus 4.6 + Sonnet 4.5 on Foundry.

Sonnet 5 bills itself as "the most agentic Sonnet," with execution capability approaching Opus class — and it is at introductory pricing of $2/$10 through August 31.

"Let Opus 5 think the design through, stream code generation to Sonnet 5" is an attractive setup cost-wise.

When the execution side is Sonnet 5 on the Anthropic API, Sonnet 5 always runs the 1M window, and auto-compaction kicks in around 967k tokens by default (per the Claude Code model-config docs; CLAUDE_CODE_AUTO_COMPACT_WINDOW changes the threshold).

16. A rhythm for day-to-day operation

Chapter 10's prompting changes apply directly to daily Claude Code operation.

Three operational patterns.

(1) Sort the verification instructions in CLAUDE.md

Split the verification-related instructions in your project's CLAUDE.md and skills into two kinds.

One: mandatory acceptance criteria (running the designated test suite, lint, type checks, the Definition of Done). Two: instructions that merely duplicate the model's self-checking ("double-check before completing," "verify once more at the end").

On Opus 5, only the latter should be deleted. The former are the project's completion conditions; keep them.

Opus 5 checks its own changes unprompted, so keeping the latter wastes tokens on over-verification.

The inversion also happens: it is precisely the situations where you don't want verification (fast draft-stage iteration, for example) that now require explicit instruction.

(2) Front-loading information into the first prompt is still paramount

The trait "hand over the complete task spec up front, then let it run" has strengthened further since 4.8.

Writing out intent, completion criteria, allowed scope, and constraints at the start remains as effective as ever.

Interim progress reporting, meanwhile, moved in the direction of more — so if you want a quiet run, put Chapter 10's narration-suppression prompt into CLAUDE.md.

(3) Redesign your model assignments

With Opus 5's arrival, the realistic division of labor inside Claude Code looks like this.

"Just use Fable 5 for everything" now pays off less often, both on cost (Fable 5: Costs and Outlook) and on classifier intervention frequency.

Reserving Fable 5 for the hardest problems and making Opus 5 the everyday ceiling is, we think, the right balance for now.

What we don't know yet / unconfirmed items

In the interest of honesty, here is what we have not been able to confirm at this point.

• The individual points in Frontier-Bench's by-effort chart (press readings put xhigh at about 44%) are shown only as an image; we could not confirm them in text form. The 43.3% in this article is the value listed in the System Card's evaluation summary table
• Independent evaluations are not yet in. We could not confirm Opus 5's position in Arena-style blind voting or on Artificial Analysis and the like as of writing (three days after release). We plan to check the answers in our August LLM rankings series
• Timelines for Opus 5 support in Priority Tier and Web Fetch have not been announced
• Fast mode, Task budgets, mid-conversation tool changes, and fallbacks are all beta / research preview, and specs and pricing may change

Summary

In one sentence: "Opus 5 is not the top model, yet it has become the practical workhorse." That is the essence of this release.

The practical takeaways condense to these five.

1. Migration is not "just swap the ID"

thinking is now ON by default, and disabling thinking is limited to effort high or below (a 400 error otherwise). Counting the Web Fetch and Priority Tier gaps as well, you need to revisit max_tokens and audit the affected requests.

2. Do not carry effort over — re-measure

The default is still high. The official starting point moved from "coding at xhigh" to "start at high, step up to xhigh only for demanding work," while the practical range of low/medium has widened. In Claude Code, note also that Opus 5 alone inherits your previous effort setting.

3. Delete duplicated re-verification instructions

Opus 5 self-verifies on its own. "Double-check" style instructions become a source of token waste through over-verification. Keep required tests, lint, and other acceptance criteria.

4. Understand the safety classifiers, then choose

Opus 5 carries model-specific classifiers (cyber, biology) that 4.8 lacks. For 4.8 migrants this is an added constraint, and staying on 4.8 remains an option for security workloads. Compared with Fable 5, interventions are 85% fewer, there is no 30-day retention requirement, and fallback exists — an easier landing spot for those moving off Fable 5.

5. Redesign the division of labor

Sonnet 5 for the everyday, Opus 5 as the mainstay (Max's default), Fable 5 for the hardest problems. The newest knowledge cutoff (May 2026) is Opus 5's quiet, exclusive advantage.

That Anthropic assembled this lineup just six weeks after the Fable 5 suspension is startling speed.

Once independent evaluations are in, we will revisit them in our regular rankings article.

See you next time!

Sources

The information in this article is based on official documentation, official announcements, and press coverage as of July 27, 2026.

Beta features and research previews may change; check each source for the latest.

Hello!

You get a new GPU, excitedly move your existing inference stack over, and

everything that ran fine yesterday now stops with errors across the board

— sound familiar? In migrations to the NVIDIA RTX 50 series and NVIDIA RTX PRO series (the Blackwell generation), this happens with rather high probability.

And here is the troublesome part: stopping with an error is actually the kind case. TensorRT's world also contains

"the build passes, execution passes, the speed is there — but the output alone is quietly broken"

— the hardest kind of failure to find.

This article is Part 1 of a series covering the whole picture.

Target environment
OS: Ubuntu 24.04 (WLS)
GPU: NVIDIA RTX PRO 4000 Blackwell / GeForce RTX 5060 Ti
(both Compute Capability 12.0 = sm_120)
Stack: PyTorch 2.11.0 + cu128 / TensorRT 10.16.1.11.

Part 1 delivers the following three things.

1. What TensorRT conversion actually is
   Why it is faster than running plain PyTorch — and what you pay in exchange
   
2. Why existing assets stop working on Blackwell (RTX 50 series, RTX PRO series)
   There is not one area to check but, broadly, three
   
3. Steps to get a minimal TensorRT 10 conversion through
   From environment setup to engine build, numerical comparison, and speed measurement, in a form you can reproduce by running the code in this article in order.

As the subject — so the same verification steps can be followed without depending on any particular product — we use a small dummy model whose full code is included in this article.

This is not about any specific product or model; read it as a story about TensorRT conversion itself.

The numbers, error logs, and behaviors shown in this article are actual results from real runs. The code targets the TensorRT 10.x API.
TensorRT 11 has changed or removed some APIs, including the precision flags (`BuilderFlag.FP16`, etc.), so the code does not run there as-is.
We deliberately use 10.16 to reproduce the migration path of existing assets.

To find out which generation (which SM) your GPU belongs to, you can use our "2026 NVIDIA GPU quick-lookup tool." Start there to check whether this article applies to your environment.

Part 1: What TensorRT conversion is — and why it beats PyTorch

In one phrase: ahead-of-time compilation

When you run PyTorch inference plainly — torch.compile and CUDA Graphs not in use, i.e. eager execution — think about what is happening inside the GPU.

Every operator call goes through dispatch, and a GPU kernel is launched per operation. Between unfused operations, intermediate results are written out to GPU memory (VRAM) and read back by the next operation.

TensorRT conversion changes this mode of execution at the root.

The model's entire computation graph is compiled ahead of time into an execution plan specific to that GPU, and at inference time the host merely submits that plan in a single call.

TensorRT calls this precompiled artifact an engine (engine / plan).

Note that the comparison target throughout this article is PyTorch eager execution. PyTorch itself has acceleration paths such as torch.compile and CUDA Graphs, and comparisons against those are a separate story.

The mental image is close to the difference between interpreting source code line by line and compiling it ahead of time into a native binary.

Five mechanisms behind the speed

Break down "why is it fast" and you get roughly these five.

[The cost] What you give up with TensorRT conversion

TensorRT articles tend to end with "it got faster!", but
in practice, understanding what you lose matters more.

1. Builds take time —
   Because kernels are measured on the real machine, builds can take minutes to tens of minutes depending on the model
   
2. Engines have poor portability —
   An engine built with default settings depends strongly on the TensorRT version and GPU used to create it; "copy the engine from the dev machine to production" is, as a rule, not possible. Compatibility modes (version compatibility, hardware compatibility) can widen the target range, at the cost of performance and feature constraints
   
3. An unsupported operation makes the build fail outright
   Part 4 of this article is exactly this
   
4. The numbers change — sometimes breaking silently
   Beyond precision loss, the very meaning of an operation can get swapped (the subject of Part 2 of the series)
   
5. Fixed shapes are easiest to handle —
   Dynamic shapes are officially supported via optimization profiles, but covering too wide a range can make per-shape optimal implementations harder to select and can increase memory
   
6. Harder to debug
   Unlike PyTorch, you cannot drop a print into the middle of the model to peek inside

Criteria for deciding "should we TensorRT this?"

Given all that, the decision becomes fairly clear.

Cases where it pays off

• An ONNX model still running on CPU remains in the pipeline
  (top priority — the gain is an order of magnitude)
  
• Models with lots of small operations where kernel-launch overhead is the bottleneck
  
• Inference services with fixed input shapes that keep running the same model at volume

Cases where it pays off less, or deserves caution

• Modules that spend most of their time on the computation itself — with the main operations' compute time already dominant, the per-item latency gain from a straightforward ONNX → TensorRT conversion was limited
  (increasing the batch barely changed the per-item time; this is the result for this particular module, though — with quantization or dedicated kernels the story changes, and throughput is another metric altogether)
  
• Workloads with wide shape variation, where the cost of designing optimization profiles or managing multiple engines does not match the benefit
  
• Models containing operations TensorRT does not support
  (→ apply it partially with Part 4's hybrid configuration)
  
• Workloads that can simply be batched
  This point deserves emphasis. If kernel-launch overhead is the bottleneck, you can sometimes gain several× just by batching on the PyTorch side, without TensorRT at all.
  In our case, running one module in batches shrank the per-frame time by more than 2× — faster than the configuration we had painstakingly converted to TensorRT. The optimal answer differs between "process one frame at a time in real time" and "render everything out in bulk"

In short,

TensorRT is not a universal speed button; it is
a tool you apply after measuring which stage is losing time, and to what

. Skip that and charge into converting every stage, and the result will not repay the build effort.

Part 2: Why assets that worked yesterday stop working on Blackwell

Now, on to Blackwell.

The RTX 50 series, and the RTX PRO Blackwell workstation GPU we tested, are sm_120 (Compute Capability 12.0) — a new generation, and this is where the causes of broken assets converge
(within the wider Blackwell family, some data-center products carry a different Compute Capability; check each product's generation with the GPU lookup tool introduced above).

In our test environment, problems appeared in the following three areas.

• TensorRT
  The first release to support sm_120 was TensorRT 10.8.0 (the release notes state "supports NVIDIA Blackwell GPUs, such as the GeForce 50-series").
  Put the other way around,
  TensorRT 8-series `.engine` / `.plan` files built with default settings fail the compatibility check and, as a rule, cannot be reused on sm_120
  (engines built on TensorRT 8.6+ with version compatibility and hardware compatibility enabled can be exceptions, but for existing assets whose compatibility settings cannot be confirmed, the safe assumption is a rebuild)
  
• PyTorch
  The first stable release with native Blackwell support is 2.7.0 (CUDA 12.8 wheel = cu128). Install an earlier wheel and it gets rejected with an error like sm_120 is not compatible with the current PyTorch installation..
  
• onnxruntime: In our test environment (onnxruntime 1.27), CUDAExecutionProvider failed to initialize and execution fell back to CPU.
  Because nothing stops with an error, it is easy to miss — the only symptom left is "things feel slow."
  This depends on the combination of onnxruntime version, distributed build, and CUDA, though — it does not mean official wheels uniformly fail on Blackwell. Verify with the versions you actually use
  

In other words, Blackwell migration is not just swapping the GPU; it is

"move PyTorch to the cu128 generation"
and "move TensorRT to the 10.x generation," plus
"re-examine which backend peripheral runtimes such as onnxruntime are actually running on"
— all run at the same time

.

The third item, onnxruntime, tends to be left alone with a shrug of "if CUDA won't work, CPU is fine" — but there is a good chance that is exactly where your pipeline's bottleneck sits, so beware.
Start by actually printing the providers and checking whether you have fallen back to CPU.

import onnxruntime as ort
print(ort.__version__)
print(ort.get_available_providers())   # if CUDAExecutionProvider is absent, the CUDA EP cannot be used in this environment

sess = ort.InferenceSession("model.onnx",
                            providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
print(sess.get_providers())            # providers registered to the session, in priority order

One caution.
sess.get_providers() tells you only "which providers were registered to the session" — it is not proof that every node ran on the GPU
(some nodes may be assigned to CPU). To confirm node-level placement, use onnxruntime's profiling or verbose logs.

Part 3: Getting a minimal TensorRT 10 conversion through — five steps

From here on, we get hands-on.

Do not start by converting your real model.

First push a minimal model of a few dozen lines through the five steps — export → build → run → numerical comparison → speed measurement — and confirm the environment itself is healthy.

With this self-check in place, when the real model later fails you can instantly separate "is it the environment, or the model."

Environment setup — the first trap lives here

All verification was done on WSL2 (Ubuntu 24.04).

We choose WSL2 because the Linux-first toolchain around CUDA / TensorRT works as-is. The venv is created with uv, and the system Python is left untouched.

# Create a working venv with uv (leaves the system Python untouched)
uv venv ~/work/trt_venv --python 3.11
source ~/work/trt_venv/bin/activate

# Pin PyTorch to the cu128 (CUDA 12.8) generation (confirmed to resolve to 2.11.0 as of 2026-07-23)
uv pip install torch --index-url https://download.pytorch.org/whl/cu128

# Pin TensorRT to the 10.x cu12 build (★ reasons below)
uv pip install "tensorrt-cu12==10.16.1.11"
# onnxscript is required by PyTorch's new exporter (dynamo=True)
uv pip install onnx onnxscript numpy

Trap 1: pip install tensorrt pulls in the CUDA 13 series

A bare pip install tensorrt resolved, at the time of testing, to TensorRT 11.1.0.106 (a CUDA 13-series build).

Meanwhile, this article's PyTorch environment is cu128 (CUDA 12.8). Having libraries from different CUDA major series in the same environment is not immediately invalid in itself, but it complicates dependencies and troubleshooting.

And above all, this article's code does not run on the TensorRT 11 API.

So, to keep the test conditions aligned, we match TensorRT's CUDA series to torch's and explicitly pin the cu12 build of TensorRT 10.16.

"Just install the latest" is not the rule here.

Once the environment is up, first confirm the GPU really is recognized as sm_120.

import torch
print(torch.__version__, torch.version.cuda)   # e.g. 2.11.0+cu128 / 12.8
print(torch.cuda.is_available())               # must be True
print(torch.cuda.get_device_capability())      # (12, 0) on RTX 50 series and the GPUs tested here

Step 1: Prepare the dummy model

We set up a small convolution block — unrelated to any product, just Conv + BatchNorm + ReLU stacked.

# tiny_model.py
import torch
import torch.nn as nn

class TinyBlock(nn.Module):
    """Minimal model for verifying TensorRT conversion (Conv+BN+ReLU ×3).
    Deliberately lines up small operations so the effect of layer fusion is easy to see."""
    def __init__(self, ch=64):
        super().__init__()
        layers = []
        for _ in range(3):
            layers += [nn.Conv2d(ch, ch, 3, padding=1), nn.BatchNorm2d(ch), nn.ReLU(inplace=True)]
        self.body = nn.Sequential(*layers)

    def forward(self, x):
        return self.body(x)

INPUT_SHAPE = (1, 64, 256, 256)   # handled as a fixed shape

Step 2: Export to ONNX

In later steps we compare numbers against "the very model we exported," so the `model` and `x` created here are reused throughout what follows.

Fixing the seed also makes the results reproducible.

torch.manual_seed(0)
model = TinyBlock().eval().cuda().float()
x = torch.randn(*INPUT_SHAPE, device="cuda")

torch.onnx.export(
    model, (x,), "tiny.onnx",
    input_names=["x"], output_names=["y"],
    opset_version=20,
    dynamo=True,          # PyTorch 2.x's new exporter (requires onnxscript)
)

For models with multiple inputs, there is the seed of an extremely hard-to-find accident right here.

torch.onnx.export's args parameter is a tuple, bound positionally, in order.

Get the parameter order of the model's forward wrong, and the export succeeds without error or warning, with the arguments swapped inside the ONNX.

That accident, and the permanent fix, are covered in detail in Part 2 of the series.

Step 3: Build the TensorRT engine

import tensorrt as trt

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

def build_engine(onnx_path, engine_path, fp16=True, workspace_gb=4, tf32=True):
    builder = trt.Builder(TRT_LOGGER)
    # TensorRT 10 defaults to explicit batch. The flag below remains for backward compatibility
    network = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))
    parser = trt.OnnxParser(network, TRT_LOGGER)

    # parse_from_file also resolves large models' external data (.onnx.data)
    # automatically from the same directory as the ONNX
    if not parser.parse_from_file(onnx_path):
        errs = [str(parser.get_error(i)) for i in range(parser.num_errors)]
        raise RuntimeError("ONNX parse failed:\n" + "\n".join(errs))

    config = builder.create_builder_config()
    config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, workspace_gb * (1 << 30))

    if fp16:
        # Add FP16 implementation candidates (tactics) to the choices. This does not guarantee
        # every layer becomes FP16; some layers may get FP32. The actual execution precision
        # can be confirmed by dumping layer information
        config.set_flag(trt.BuilderFlag.FP16)

    # ★ Even in fp32 builds, TensorRT uses TF32 (10-bit mantissa) by default.
    #    Disable it when strictly verifying fidelity (details in Part 2 of the series)
    if not tf32:
        config.clear_flag(trt.BuilderFlag.TF32)

    plan = builder.build_serialized_network(network, config)
    if plan is None:
        raise RuntimeError("build_serialized_network returned None (engine build failed)")
    with open(engine_path, "wb") as f:
        f.write(plan)
    return engine_path

build_engine("tiny.onnx", "tiny_fp16.plan", fp16=True)

As the comment says, BuilderFlag.FP16 does not mean "make everything fp16"; it means "allow FP16 implementation candidates". From here on, whenever the text says "built with fp16," read it as "built with FP16 tactics allowed."

Step 4: Run the engine (passing PyTorch tensors directly)

This is probably the part practitioners most want to know.

If device, dtype, shape, and memory layout (contiguous) already match the engine's requirements, you can register the PyTorch tensor's GPU memory address (`data_ptr()`) with TensorRT directly and avoid an extra copy at the boundary.

Once you can do this, the hybrid configuration — "convert only part of the model to TensorRT and leave the rest in PyTorch" — becomes realistic (we use it in Part 4).

For safety, the sample below automatically matches dtype and device before passing
(when the conditions do not match, a converting copy happens there; why this auto-conversion is needed is explained in the trap just below).

_TRT2TORCH = {
    trt.float32: torch.float32, trt.float16: torch.float16,
    trt.int32: torch.int32, trt.int64: torch.int64,
    trt.int8: torch.int8, trt.bool: torch.bool,
}

class TRTRunner:
    """Loads an engine and runs it using PyTorch tensors for I/O (fixed shapes assumed)"""

    def __init__(self, engine_path, device="cuda:0"):
        self.device = torch.device(device)
        # Runtime / engine / context are tied to the current CUDA device, so create them
        # with the target device made explicit (mandatory when using anything other than cuda:0)
        with torch.cuda.device(self.device):
            self.runtime = trt.Runtime(TRT_LOGGER)
            with open(engine_path, "rb") as f:
                self.engine = self.runtime.deserialize_cuda_engine(f.read())
            if self.engine is None:
                raise RuntimeError(f"deserialize failed: {engine_path}")
            self.ctx = self.engine.create_execution_context()

        self.inputs, self.outputs = [], []
        for i in range(self.engine.num_io_tensors):
            name = self.engine.get_tensor_name(i)
            if self.engine.get_tensor_mode(name) == trt.TensorIOMode.INPUT:
                self.inputs.append(name)
            else:
                self.outputs.append(name)

    def prepare(self, feed: dict):
        """Set input shapes and addresses, and allocate output buffers.
        If input buffers are reused, once is enough (in speed measurement, keep this outside the timed region)"""
        self._feed = {}
        for name, t in feed.items():
            # ★ Even with FP16 tactics allowed at build time, the engine's I/O types stay as defined
            #   in the ONNX (usually fp32). Pass a half tensor's address as-is and the fp16 bits get
            #   read as fp32, corrupting the output. And no exception is raised (details below)
            want = _TRT2TORCH[self.engine.get_tensor_dtype(name)]
            t = t.to(device=self.device, dtype=want).contiguous()
            # The set_* calls also return bool. Swallowing failures is strictly off-limits, given this article's theme
            if not self.ctx.set_input_shape(name, tuple(t.shape)):
                raise RuntimeError(f"set_input_shape failed: {name}, shape={tuple(t.shape)}")
            if not self.ctx.set_tensor_address(name, t.data_ptr()):
                raise RuntimeError(f"set_tensor_address failed: {name}")
            self._feed[name] = t

        self._outs = {}
        for name in self.outputs:
            shape = tuple(self.ctx.get_tensor_shape(name))
            if any(d < 0 for d in shape):
                raise RuntimeError(f"output shape unresolved: {name} {shape}")
            dtype = _TRT2TORCH[self.engine.get_tensor_dtype(name)]
            o = torch.empty(shape, dtype=dtype, device=self.device).contiguous()
            self._outs[name] = o
            if not self.ctx.set_tensor_address(name, o.data_ptr()):
                raise RuntimeError(f"set_tensor_address failed: {name}")
        return self._outs

    def enqueue(self):
        """Queue execution on the current stream (asynchronous). ★ Always check the return value"""
        ok = self.ctx.execute_async_v3(torch.cuda.current_stream(self.device).cuda_stream)
        if not ok:
            raise RuntimeError("TensorRT enqueue failed: possibly an engine/runtime mismatch. "
                               "The output cannot be trusted, so processing stops here")

    def run(self, feed: dict):
        """Convenience version for verification (prepare + synchronize every call). Not for speed measurement"""
        outs = self.prepare(feed)
        self.enqueue()
        torch.cuda.current_stream(self.device).synchronize()
        return outs

Trap 2: Even built with FP16 tactics allowed, the engine's I/O stays fp32

We actually stepped on this one during this verification.

config.set_flag(trt.BuilderFlag.FP16) enabled adds FP16 implementation candidates to TensorRT's choices (not every layer becomes fp16), but

the network's input/output tensor types follow the ONNX definition — fp32 in this example.

Pass a PyTorch half tensor's address here as-is, and TensorRT treats the region as an input buffer of fp32 elements. Since fp16 is 2 bytes per element and fp32 is 4, not only does the interpretation of the values change — TensorRT may also read beyond the region PyTorch allocated.

The behavior is undefined; beyond corrupted output, some environments may see CUDA errors or crashes
(set_tensor_address registers only the pointer and conveys nothing about the buffer's size).

Measured, it came out like this.

• With dtype correctly matched:
  maximum absolute error about 0.001
  (no major numerical breakdown under this FP16-tactics-allowed configuration)
  
• With the half tensor's address passed as-is:
  maximum absolute error about 13.8
  (on a model whose output range is a bit under 0.5. This value is one observation in our environment and does not represent a reproducible error magnitude)

And the worst part: in our measurements, even in this state `execute_async_v3` returned `True` (success), produced no NaN, and threw no exception. The reason the code above includes the two want = _TRT2TORCH[...] lines is to prevent exactly this silent degradation.

Trap 3: Never discard the return value of `execute_async_v3`

This method throws no exception on failure — it merely returns `False`.

Leave that unchecked, and the un-updated output buffer (garbage, or the previous frame's values) flows straight downstream.

Processing looks "successful" while only the output is broken — the worst possible shape of failure. As in the code above, we strongly recommend making False raise immediately.

Step 5: Match the numbers first, then measure speed

Before measuring speed, first confirm the numbers match.
Do not reverse the order.

The reference for the comparison is the very fp32 `model` and `x` used for the export in Step 2. Do not construct a new model here
(you would be comparing against randomly initialized "different weights"), and do not use an fp16-converted variant as the reference either (that would no longer verify "are the export and the engine correct").

# model / x are the same fp32 objects used for the export in Step 2
with torch.inference_mode():
    torch_ref = model(x)

runner = TRTRunner("tiny_fp16.plan")
trt_out = runner.run({"x": x})["y"]

diff = trt_out.float() - torch_ref.float()
print("max_abs :", diff.abs().max().item())
print("nRMSE   :", (diff.norm() / torch_ref.norm()).item())

Without the output's own value range alongside it, the size of a maximum absolute error cannot be judged. An untrained dummy model has a different value range from a real model, so a normalized relative error (nRMSE) should be checked alongside — that is the safe practice.

For this TinyBlock: max_abs about 0.001, nRMSE about 0.0016 — a small error level for an engine with FP16 tactics allowed, under this configuration.

Measurement is done with a dedicated CUDA stream + CUDA Events.time.time() cannot correctly measure the GPU's asynchronous execution.

import numpy as np

def bench_callable(fn, n_warmup=30, n_iter=200, device="cuda:0"):
    """Measures fn() and returns mean/median/min/max (milliseconds).
    Using a dedicated stream lets PyTorch and TensorRT be compared fairly under identical conditions"""
    dev = torch.device(device)
    # Complete any work queued earlier on other streams (model conversion, prepare, etc.).
    # Streams have no guaranteed ordering unless explicitly synchronized
    torch.cuda.synchronize(dev)
    stream = torch.cuda.Stream(dev)
    with torch.cuda.stream(stream):
        for _ in range(n_warmup):      # warmup is mandatory (first runs are slow)
            fn()
        stream.synchronize()

        times = []
        for _ in range(n_iter):
            s = torch.cuda.Event(enable_timing=True)
            e = torch.cuda.Event(enable_timing=True)
            s.record(stream)
            fn()
            e.record(stream)
            stream.synchronize()
            times.append(s.elapsed_time(e))

    t = np.array(times)
    return {"mean": float(t.mean()), "median": float(np.median(t)),
            "min": float(t.min()), "max": float(t.max())}

For the speed measurement, the PyTorch side uses an fp16-converted model as its realistic production candidate. We want to keep the fp32 model used for the comparison intact, so we copy before converting to fp16. The key point: on the TensorRT side, run `prepare()` (buffer allocation and address registration) once, outside the timed region, and call only `enqueue()` during measurement.

Call run() every iteration and the dtype conversion, output allocation, and synchronization all get counted into TensorRT's time — no longer a fair comparison.

import copy

# The fp16 model for speed measurement is made separately via deepcopy, to preserve the fp32 model used for export
model_half = copy.deepcopy(model).half()
x_half = x.half()

# PyTorch side (eager + inference_mode, fp16)
def torch_fn():
    with torch.inference_mode():
        model_half(x_half)

# TensorRT side: prepare runs once, outside the timed region. Only enqueue is measured
runner.prepare({"x": x})
def trt_fn():
    runner.enqueue()

print("torch:", bench_callable(torch_fn))
print("trt  :", bench_callable(trt_fn))

Here are the results.

Measurement conditions: GeForce RTX 5060 Ti / batch=1 / PyTorch 2.11 eager + inference_mode, fp16 model / TensorRT 10.16, fp32 I/O, FP16 tactics enabled. TensorRT preallocates I/O buffers and only the enqueue region is measured. Median of 200 runs with 30 warmups each.

For a small model that merely stacks Conv + BatchNorm + ReLU three deep, the GPU-execution region showed about a 2.5× difference.

Reading this number takes care, though.

This compares "fully fp16 PyTorch eager" against "a TensorRT engine with fp32 I/O and FP16 tactics allowed," over the GPU-execution region excluding upfront dtype conversion and buffer preparation.

It is neither a same-precision comparison nor an end-to-end speedup figure for a whole pipeline.

Even so, the fact that the same model structure shows this much difference when each side runs in its realistic production configuration is consistent with the effects listed in Part 1: layer fusion, kernel selection, and reduced runtime overhead.

For the comparison, look at median.

Averages get dragged around by the occasional outlier.

Also, keep the GPU exclusively yours during measurement. If other inference processes or desktop rendering share the GPU, the numbers swing by tens of percent easily and the comparison loses meaning.

Part 4: The first wall — the build falls over on grid_sample's dimensionality

When you start converting your real model, this is what most people hit first.If the model contains a 5-dimensional (volumetric) `grid_sample`, the TensorRT build does not pass.

Warping 3D volumes appears in many domains: medical image registration, NeRF-family methods, 3D feature warping. The minimal reproduction module is as follows.

import torch.nn.functional as F

class VolumetricWarp(nn.Module):
    """Minimal module containing a 5D (volumetric) grid_sample.
    input: [N, C, D, H, W] / grid: [N, D, H, W, 3]"""
    def forward(self, vol, grid):
        return F.grid_sample(vol, grid, align_corners=False)

VOL_SHAPE  = (1, 32, 16, 64, 64)
GRID_SHAPE = (1, 16, 64, 64, 3)

Export and build can be tried with the exact same tooling as Steps 2–3.

warp_model = VolumetricWarp().eval().cuda()
vol = torch.randn(*VOL_SHAPE, device="cuda")
grid = torch.rand(*GRID_SHAPE, device="cuda") * 2 - 1

torch.onnx.export(
    warp_model, (vol, grid), "volumetric_warp.onnx",
    input_names=["vol", "grid"], output_names=["y"],
    opset_version=20, dynamo=True,
)

# The ONNX export succeeds. What fails is the build that follows
build_engine("volumetric_warp.onnx", "volumetric_warp.plan", fp16=False)

Run it, and it fails with the following error (verbatim log from the real machine).

[TRT] [E] INetworkDefinition::addGridSample: Error Code 3: API Usage Error
    (Parameter check failed, condition: input.getDimensions().nbDims == 4.
     In addGridSample at /_src/optimizer/api/network.cpp:1803)
[TRT] [E] ModelImporter.cpp:138: While parsing node number 0 [GridSample -> "y"]:
[TRT] [E] ModelImporter.cpp:149: ERROR: ModelImporter.cpp:490 In function parseNode:
[6] Invalid Node - node_GridSample_0

input.getDimensions().nbDims == 4

In other words, TensorRT 10.16's native `GridSample` accepts only 4-dimensional inputs.

5D calls are rejected at the point where the parser adds the node.

This matches the primary sources. The onnx-tensorrt operators documentation states, for GridSample, "Input must be 4D input.", and the request for 5D support exists as NVIDIA/TensorRT issue #3890, which remains open / triaged with no native support provided.

"Raise the ONNX opset and it will pass" is wrong

This is a common source of confusion, so let's make it explicit.

The ONNX standard defines 4D grid_sample at opset 16 and 5D (volumetric) grid_sample at opset 20.

Therefore the export from PyTorch succeeds fine at opset 20, and a 5D GridSample node is duly generated inside the ONNX file. Indeed, the error log above shows the failure happening at build time, after a successful export.

What fails is not the export but TensorRT's build. What ONNX can express and what TensorRT can digest are separate questions.

The practical answer — leave only the failing module in PyTorch

One workaround is to write the operation TensorRT lacks as your own CUDA kernel and register it as a plugin. For 5D grid_sample there are existing OSS implementations (grid-sample3d-trt-plugin, among others), and rebuilding one for the new CUDA / TensorRT generation is a viable approach.

But that is expert work, so we cover it in the final installment of this series.

Making zero migration progress until the plugin is finished is not realistic. What works here is the hybrid configuration. The idea is utterly simple: convert to TensorRT only the modules that pass, and leave the failing modules in PyTorch.

# Configuration sketch (head_fp16.plan is a placeholder name meaning "the engine for the TensorRT-converted latter stage")
warp = VolumetricWarp().eval().cuda()      # 5D grid_sample → stays in PyTorch
head = TRTRunner("head_fp16.plan")         # latter stage → TensorRT engine

def forward(vol, grid):
    with torch.no_grad():
        feat = warp(vol, grid)             # PyTorch produces its output on the GPU
    return head.run({"x": feat})["y"]      # and that address is handed to TensorRT as-is

As Step 4 showed, when dtype, device, and shape match, PyTorch tensors and TensorRT can exchange addresses on the same device and the same stream, so bridging the two incurs no additional copies.

In our case, this configuration alone — without writing a single plugin — improved things nearly 2×, and as we progressively replaced the modules that load cleanly onto TensorRT, the pipeline as a whole reached roughly 2× within the plugin-free scope.

"First, secure reliable speedups in the low-risk scope. Carve the custom plugin out as a separate project."

That, we believe, is the realistic way to run this migration.

What we have not yet confirmed

In the interest of honesty: the verification in this article was done on a single software configuration, "TensorRT 10.16 + PyTorch 2.11 + cu128." The following have not been confirmed here.

• Reproducibility on other TensorRT versions
  — in particular, the behavior of 4D grid_sample (covered in Part 2 of the series) may be fixed in future versions. TensorRT 11 has also changed its APIs (strong typing), so this article's code does not run there as-is
  
• Comparison against PyTorch with torch.compile or CUDA Graphs
  — this article compares against eager execution only
  
• Combination with int8 quantization
  — this article covers fp16 / fp32 only
  
• Builds with dynamic shapes
  — everything in this article is fixed-shape.
  Speed and accuracy with optimization profiles require separate verification
  
• Numbers and speed for the 5D grid_sample plugin built for Blackwell
  — planned for the final installment

Summary

To sum it up in one sentence:

"Blackwell migration is not a GPU swap; it is updating PyTorch's and TensorRT's support status plus re-confirming which execution backend the peripheral runtimes are actually using — and TensorRT conversion is a separate investment decision layered on top."

That is the story
(not exactly one sentence, we admit.
Incidentally, our environment did hit the onnxruntime CPU fallback).

Now, to close, let us restate the important points.

1. Check the GPU's Compute Capability first
   (RTX 50 series is sm_120. Other Blackwell product lines carry different values, so check per product)
   
2. PyTorch on the cu128 generation
   (stable from 2.7.0)
   
3. Use TensorRT 10.8 or later.
   Existing TensorRT 8 engines should, after checking compatibility settings and target GPU, be rebuilt as a rule
   
4. Match TensorRT's CUDA series to torch's (pin to cu12)
   
5. Check whether onnxruntime has fallen back to CPU by actually printing the version and providers. If it has, it is your top TensorRT conversion candidate
   
6. Self-diagnose with a minimal model before moving to the real one
   
7. Pass tensors matched to the engine's I/O dtypes
   (even FP16-tactics builds usually keep fp32 I/O)
   
8. Always check the return values of execute_async_v3 and set_input_shape / set_tensor_address
   (no exception is thrown on failure)
   
9. Do the numerical comparison before the speed measurement, with the same weights and the same input used for the export.
   
10. Check the dimensionality of grid_sample inside your model
    (5D does not pass natively; raising the opset does not solve it)
    
11. Carve out the failing modules and go hybrid
    
12. Measure with the GPU exclusively occupied, using the median of CUDA Events. For engine-only comparisons, separate the buffer preparation and time the enqueue region; for production decisions, also measure end-to-end time including dtype conversion, module boundaries, and synchronization

Coming next (Part 2) — a passing build is not necessarily correct

Next up is not the finale but the middle installment.

What this Part 1 covered were problems of the kind that have the decency to stop with an error.

What is truly frightening in a migration lies beyond that.

We will cover the "silent degradation" we actually stepped on.

• The build succeeds, execution succeeds, and the output's shape and values look plausible — yet the meaning of an operation has been swapped for something else — one such case
  (this one we could fully reproduce with a publishable dummy model)
  
• A build that succeeds with good speed — yet the output breaks the moment real data goes in — one such module
• An engine that worked perfectly on the dev machine quietly emitting garbage on a different GPU of the same generation — that phenomenon
• Building in fp32 that turned out not to be fp32 — that story
• And the one that made us cry the hardest: the export-time argument mix-up accident

Every one of these is deeply troublesome when it surfaces after you have reported "it's faster now, all good" — so stay tuned.

See you next time!

References (primary sources)

• NVIDIA TensorRT documentation (Release Notes 10.8.0 — the Blackwell support entry)
• NVIDIA official forum: Software Migration Guide for NVIDIA Blackwell RTX GPUs (CUDA 12.8 / PyTorch / TensorRT)
• PyTorch issue #159207: sm_120 is not compatible with the current PyTorch installation
• onnxruntime issue #27875: report on CUDAExecutionProvider initialization in Blackwell environments
• NVIDIA/TensorRT issue #3890: request for native 5D GridSample support
• SeanWangJS/grid-sample3d-trt-plugin (custom plugin implementation of 5D grid_sample)
• ONNX Operators: GridSample (opset 16 / 20 specifications)
• Related article from Qualiteg: 2026 NVIDIA GPU quick-lookup tool

Hello!

On July 16, 2026, Moonshot AI of Beijing, China announced its new flagship model Kimi K3 and began offering it via its API and web services.

The sheer scale of 2.8 trillion total parameters, a 1-million-token context, and the declaration that

"it will become the largest open-weight model in history"

have set the AI community abuzz.

In our article this past May, "When Will a Mythos-Level Open Model Arrive?," we predicted when open models would catch up to the closed frontier.

Kimi K3 is one of the latest "answers" to that very question as of today.

On the other hand, as of July 20 — the day this article is being written — neither the model weights nor the technical report has been released yet。

That said, on X and other corners of social media, comments such as

"the guardrails are on the loose side — it handles prompts that Fable 5 would refuse"

and "it's easier to work with than Fable 5, which falls back to Opus 4.8 at the drop of a hat"

can be spotted here and there; against
the strong-guardrail policy of US-made models, a Chinese-made model with weaker guardrails may just hold the hidden potential to ultimately win users over.

That said, in this article we'll set aside the social-media chatter and instead cross-check primary sources (Moonshot's official blog, official API documentation, GitHub) against English-language press coverage and analyses from independent evaluation organizations, sorting out the full picture of Kimi K3 while distinguishing "officially announced," "independently evaluated," and "unconfirmed."

Part 1: What Is Kimi K3?

How the Release Unfolded

Kimi K3 was announced on July 16, 2026, and became available through the following channels the same day.

• Kimi.com(chat UI)
• Kimi Work(workspace product)
• Kimi Code(coding agent)
• Kimi API(OpenAI SDK compatible)

It was also added to the third-party API aggregator OpenRouter the same day — an API-first launch in the style of "make it available via API and web first, weights later."

The official blog explicitly states, "we will make the full model weights public by July 27, 2026 at the latest," and explains that the delay is "to ensure a smooth rollout with our inference partners."

Quick Reference: Basic Specs

Part 2: Architecture — An Extremely "Sparse" 2.8 Trillion Parameters

An MoE That Uses Only 16 Experts out of 896

Kimi K3 is a sparse MoE (Mixture of Experts) model built on a framework Moonshot calls "Stable LatentMoE." What stands out is its low activation rate: of the 896 experts, only 16 (about 1.8%) are activated per token.

The 2.8-trillion total parameter figure is rounded up in marketing as "the world's first open 3T-class model," but only a small fraction of it actually runs at inference time.

This
"open only the shelves you need in a vast warehouse of knowledge"
design aims to reconcile scale with inference cost.

Note that Latent.Space describes the active parameter count per token as "A50B" (roughly 50 billion), but Moonshot has not stated it officially, so this number is unconfirmed. Once the weights are released, it may become verifiable from the config files or model card.

Two Homegrown Innovations: KDA and AttnRes

The architectural highlights are two new mechanisms Moonshot developed in-house.

• Kimi Delta Attention (KDA)
  A hybrid linear attention mechanism, claimed to deliver up to 6.3x faster decoding at 1-million-token-class context lengths
  
  
• Attention Residuals (AttnRes)
  A mechanism that replaces conventional residual connections. Roughly 25% better training efficiency at under 2% additional cost is the claim.

Both are intriguing numbers, but at this point they are vendor claims originating from official announcements on X; with no technical report published, no third party has reproduced them.

Native Vision and an Always-On Thinking Mode

K3 is a natively multimodal model that handles text, images, and video.

On the vision side, noted AI blogger Simon Willison actually tested it and confirmed "Vision works well" — one of the few independent, hands-on data points. In his single test he also observed the thinking mode consuming 13,241 reasoning tokens (one example, not a typical value for the model in general).

The thinking mode is always on, and the reasoning_effort you can specify on the official Kimi API is currently limited to 'max'. Kimi Code, on the other hand, officially offers a choice of low/high/max — worth remembering that the options differ by channel.

Part 3: Benchmarks — Read "Self-Reported" and "Independent" Separately

When reading Kimi K3's benchmark numbers, the most important thing is distinguishing which figures are Moonshot's self-reported ones and which come from independent evaluations.

For independent comparative evaluations we have Arena.AI (blind voting) and Artificial Analysis (an evaluation organization); separately, Simon Willison has published an API hands-on.

Independent Evaluation #1: First Place in the Frontend Code Arena

This one was perhaps the biggest surprise.

In Arena.AI's Frontend Code Arena, which is decided by blind user voting, Kimi K3 took first place with 1,679 points as of the announcement (July 16).

That score surpasses even Claude Fable 5, and from the previous-generation Kimi K2.6's 18th place it shot up 17 ranks in one leap. Because it is blind voting, self-report bias is unlikely to creep in, though noise from early vote counts and prompt skew remains. Even so, it is one of the strongest external signals to date of the model's frontend generation capability.

Independent Evaluation #2: Artificial Analysis — "Opus 4.8 / GPT-5.5 Class"

On evaluation firm Artificial Analysis's Intelligence Index it scores 57, which — to borrow the firm's own phrasing — positions it as "comparable to Claude Opus 4.8 and GPT-5.5,

but not reaching Claude Fable 5 or GPT-5.6 Sol."

On the same firm's GDPval-AA v2 (an Elo evaluation on real-world work tasks), it also ranks third, right behind the two frontier leaders.

What deserves attention here is the agentic Elo in the right panel.

On GDPval-AA v2's agentic Elo, K3 (1,668) beats Claude Opus 4.8 (1,600) and GPT-5.5 (1,494) according to reports (The Decoder and others), which is consistent with Moonshot's own claim that it "trails Fable 5 / GPT-5.6 Sol overall, but surpasses Opus 4.8 / GPT-5.5 in coding and agentic work."

That said, not every independent evaluation is positive.

On Artificial Analysis's AA-Omniscience, while accuracy improved from K2.6's 33% to 46%, the hallucination rate rose from 39% to 51%, the firm reports.

Gains in knowledge and reasoning do not automatically translate into more cautious or more factual answers — a point worth remembering as input for enterprise adoption decisions.

Also, in the analysis Artificial Analysis published on July 17, Kimi K3 placed just behind Fable 5 on the firm's private agentic evaluation AA-Briefcase as well, at second place (Elo 1,547).

We use the 1,547 that the evaluator itself published in its analysis, rather than the 1,527 reported by VentureBeat (Elo is recomputed dynamically, so it drifts somewhat over time).

Position in Aggregate Rankings (For Reference)

On the benchmark aggregator BenchLM.ai, it sat in 4th place with an overall score of 80.96 as of our July 19 survey.

That said, BenchLM is not an evaluation body that runs every model itself under identical conditions; it is a service that aggregates multiple published scores.

Because the available benchmarks and evaluation conditions differ from model to model, "4th overall" is best treated as a reference value for getting a sense of direction.

Highlights from Moonshot's Official Evaluations

Among Moonshot's official numbers, the agentic web-browsing benchmark BrowseComp comes in at 91.2.

That figure is under the company's evaluation setup, which applies context compression upon reaching 300K tokens; without compression it is explicitly stated as 90.4.

External aggregations, moreover, place it just behind GPT-5.6 Sol (92.2), so it is not an unconditional "SOTA."

Meanwhile, independent evaluations by Arena.AI and Artificial Analysis are already underway via the API. At this stage, the accurate framing is: "the externally observable performance has some corroboration, but the model's internal specifications and the reproduction conditions of the official benchmarks cannot yet be verified."

Part 4: API Pricing — Cheaper Than the Closed Top Tier, but a Big Hike Within the Kimi Line

The API pricing is primary information confirmed on the official pricing page.

• Kimi K3: $3.00 input / $15.00 output (per million tokens), with cache-hit input at $0.30
• K2.7 Code (coding-focused) / K2.6 (general-purpose): $0.95 input / $4.00 output each
• No surcharge for long context. You can use the OpenAI SDK, and calling it is just a matter of swapping the base_url, API key, and model name

English-language coverage has described this as "Claude Sonnet-class pricing."

Indeed, K3's $3/$15 matches Claude Sonnet 5's standard pricing effective September 1, 2026 (though Sonnet 5 is offered at an introductory $2/$10 through August 31, so as of July 20, K3 works out 50% more expensive on both input and output).

Within the Kimi lineup, it is a substantial hike from K2.6's $0.95/$4, and Simon Willison likewise notes that Moonshot has "stepped into a higher price tier for a Chinese AI lab."

Moonshot also claims that its distributed inference architecture called Mooncake achieves "a 90%+ cache hit rate on coding workloads"; if that is close to reality, effective costs for agentic use would come down considerably (this figure, too, is a vendor claim).

Part 5: "Open Weights" Is Still Only a Promise

This is the single most important point in this article.

As of July 20, 2026, Kimi K3 is not an "open-weight model" but a "model that is scheduled to become open-weight."

Checking Moonshot's official GitHub organization, there is still no model repository for Kimi K3, and the pinned model repository remains Kimi-K2.5. There is no model card on Hugging Face either.

On quantization, the official tech blog explicitly specifies MXFP4 weights / MXFP8 activations, which counts as primary information; whether the files distributed on Hugging Face will be MXFP4 as-is, however, is unconfirmed since the model card has not been published.

The technical report is likewise "to be released in the future."

There is one more practically important caveat:
"the weights being released" and "being able to run it yourself" are separate problems.

Moonshot recommends super-node configurations with 64 or more accelerators as the deployment target for K3.

Even quantized, this is a 2.8-trillion-parameter-class model; self-hosting the full model is out of reach not just at home but probably in office environments too. For the time being, think of it as something for specialized data centers and large-scale inference providers.

Multiple reports put the license at "Modified MIT," but with no model card in existence, it cannot be confirmed against a primary source (unconfirmed).

If released as scheduled, it would overtake DeepSeek v4 Pro (1.6 trillion parameters) to become the largest open-weight model in history.

Part 6: Agentic Capabilities — K3's Main Battleground

What Moonshot pitches hardest for K3 is agentic use.

• BrowseComp (agentic web browsing): 91.2 (under Moonshot's evaluation conditions; external aggregations place it just behind GPT-5.6 Sol)
• 2nd place on AA-Briefcase (Artificial Analysis's agentic evaluation)
• Beats Opus 4.8 and GPT-5.5 on the GDPval-AA v2 agentic Elo
• Kimi Code — offered as its coding-agent product line
• The API supports tool calling, and migrating from the OpenAI SDK is easy

In Kimi Code, a new plan structure — Starter / Explorer / Expert / Master — was introduced on July 20. K3 itself is available on all plans, and the 1M context requires Explorer or above (existing Moderato / Allegretto subscribers keep the terms of their old plans).

Caveats for Production Adoption

There are also constraints that the official documentation and blog themselves acknowledge.

• Full re-sending of thinking history is required
  Unless you return assistant messages verbatim — including the thinking from past turns — quality can become unstable
  
• Excessive autonomous judgment
  The official docs explicitly note that, given ambiguous instructions, it may make unexpected decisions on the user's behalf
  
• Sampling settings are fixed, and the web-search feature is not recommended for production use for the time being

Part 7: What's New Since the K2 Generation

The changes from the previous generation sum up as follows.

1. A leap in scale
   — a massive expansion to 2.8 trillion total parameters from the previous flagship K2/K2.5 on GitHub
   
2. Homegrown new mechanisms
   — introduces KDA and AttnRes, with an extremely sparse Stable LatentMoE design at roughly 1.8% activation
   
3. A jump in capability
   From K2.6's 18th place to 1st place with K3 in the Frontend Code Arena
   
4. Native vision
   — image and video understanding as standard
   
5. Always-on thinking mode
   
6. Product-line reorganization
   — a three-tier structure: K3 (flagship) / K2.7 Code (coding-focused) / K2.6 (general-purpose budget tier)

Comparisons with K2.6 are partly verifiable through the Arena.AI and Artificial Analysis evaluations (18th place → 1st in the Frontend Code Arena, 33% → 46% accuracy on AA-Omniscience, and so on). However, cross-generation comparisons under identical conditions — including K2 and K2.5 — remain limited, and details including the architecture and training methods await the technical report.

Part 8: What We Still Don't Know

Let's also organize what remains unannounced or unconfirmed at this point.

• Training data scale, training token count, and training methods (pre-training corpus, RL/post-training setup) → undisclosed in both primary and secondary sources
• Active parameter count → "~50B (A50B)" is an analyst's notation, not officially disclosed
• Technical report / paper → unpublished (only an official statement that one is forthcoming)
• Hugging Face model card and distribution file format → nonexistent (weights unreleased)
• Exact license terms: "Modified MIT" is mentioned only at the level of press reports

Once the July 27 weight-release deadline passes, check three things: (1) whether the weights were actually released, (2) whether the evaluation conditions behind the official scores are disclosed in enough detail in the technical report to allow retesting under equivalent conditions, and (3) the license terms. We plan to cover any follow-up news in detail on this blog.

Conclusion

To sum up Kimi K3 in one line:

「an ultra-large-scale, "open-weights-to-be" MoE that falls one step short of the frontier's very top (Claude Fable 5 / GPT-5.6 Sol) overall, yet is positioned to aim at or above the Opus 4.8 / GPT-5.5 class in coding and agentic work」

.

The independent result of first place in the blind-voted Frontend Code Arena carries real weight as corroboration of its ability; on the other hand, for the model's internal specifications and the reproduction conditions of the official benchmarks, we must wait for the weight release scheduled by July 27 and the technical report to come. Whether the "largest open weights in history" billing becomes real — the answer starts arriving in about a week.

In our May article "When Will a Mythos-Level Open Model Arrive?," we predicted when open models would catch up to the frontier based on the slope of benchmark trends.

If K3 releases its weights as planned, "the weights of a model in the same league as Opus 4.8 on Artificial Analysis's overall index" will be obtainable in the summer of 2026 — meaning the trend we sketched in that article is running along the upper edge of our projected band.

However, for the cybersecurity capabilities that were the centerpiece of that article (CyberGym and the AISI real-environment tests), no evaluation data for K3 exists yet. The reckoning with our line that "capability attainment and the decision to release are separate problems" will have to wait for independent evaluations after the weights are out — and we intend to revisit it then.

Coming Up Next

Next time, we plan to cover Qwen's officially announced next flagship, Qwen3.8. With 2.4 trillion total parameters and an open-weight release planned, it is another ultra-large open-series model standing right alongside Kimi K3. Just as with Kimi K3, we'll dig in with thorough, primary-source-based research — stay tuned.

See you next time!

References (Primary and Key Sources)

• Moonshot AI official blog: Kimi K3 Tech Blog
• Kimi API official documentation: K3 Quickstart
• Kimi API official pricing page
• Kimi Code What's New (new plan structure)
• Moonshot AI official GitHub
• Moonshot AI official X (source of the KDA / AttnRes efficiency figures)
• Artificial Analysis: Kimi K3 analysis (Intelligence Index / GDPval-AA v2 / AA-Briefcase / AA-Omniscience)
• Simon Willison: Kimi K3 hands-on
• Tom's Hardware: Moonshot releases 2.8-trillion-parameter Kimi K3
• VentureBeat: China's Moonshot AI releases Kimi K3
• Arena.AI official X: Frontend Code Arena results
• OpenRouter: moonshotai/kimi-k3
• Latent.Space: Kimi K3 analysis
• BenchLM.ai: Kimi K3
• Alibaba Qwen official X: Qwen3.8 announcement (source for the preview above)
• Bloomberg: Alibaba's Qwen unveils preview of flagship AI model

* This article is based on research as of July 20, 2026. All charts were drawn by Qualiteg from published figures and are not reproductions of official charts.

Hello! This is the Qualiteg Product Development Team!

Today we would like to discuss the de-identification of PII (Personally Identifiable Information).

To date, we have published technical explainers on personal-information detection and AI security, including high-accuracy PII detection, staged PII masking when using LLMs、test design for PII detection, and more.

Today, we develop and offer our PII detection and masking technology, the "PII-FI Engine," along with "PII-FI Scan」「PII-FI API," a PII masking and de-identification service built on it.

In this article, we take up the design principles behind "once PII has been detected, how should it be rewritten?" — using a single example sentence as a touchstone, we present the framework we have actually adopted in our products.

One caveat up front: what we call "de-identification (de-identification)" in this article refers to technical processing that makes documents and logs safe to share and analyze (processing data so that individuals can no longer be identified).

It is not a guarantee that the result qualifies as "pseudonymized information" or "anonymized information" under Japan's Act on the Protection of Personal Information (we touch on the relationship with these legal terms later in the article).

What Is De-identification For?

Let us start from first principles.

If you can use your logs and documents "as they are," you do not need de-identification.

If the original is in your hands and you are the only one who will see it, there is no reason to process it.

The need for de-identification arises at

"we want to hand this data to someone so the work can proceed, but we cannot hand it over raw"

— precisely that moment.

For example:

• You want to outsource an analysis task, but customer information is mixed in
• You want to feed operational logs into an LLM for incident analysis
• You want to send a full set of logs to a security vendor for analysis
• You want to review files containing PII before sending them to another company
• You want AI to organize medical-record information, but need to mask PII before it goes into the AI

Data is a mixture of "what happened" and "who it is about."

It is the former that creates value in analysis and sharing.

It is the latter that can lead to harm for the individual (impersonation, discrimination, reputational damage, and so on).

In other words, de-identification is the technique of

keeping the "events" while severing the "who"

.

And it is not only the individuals appearing in the data who are protected.

The provider is protected from liability for leaks, and the recipient reduces both the risk and the management burden of holding "information that is a liability merely to possess."

The Touchstone: 「甲野太郎は高血圧です」 ("Kono Taro has hypertension")

Now for the main topic. Consider de-identifying the following sentence.

「甲野太郎は高血圧です」 — "Kono Taro has hypertension."

There is more than one way to process it.

Let us compare three representative patterns.

Pattern 1: Replace Only the Name with a Pseudonym

乙山次郎は高血圧です ("Otsuyama Jiro has hypertension")

The link between 甲野太郎 (Kono Taro) and hypertension has been severed.
The reader receives only the event that "someone has hypertension." And this event is still true, and still usable. For a reader examining the case details or the appropriateness of the response, this sentence continues to function as working material.

This is the basic form of de-identification. Note, however, that what succeeded here is only "severing the link within this one sentence." In real documents, the surrounding context can still suggest who the person is even after the name is changed — a limitation we confront head-on in Pitfall 3.

Pattern 2: Fake the Disease Name Too

乙山次郎は花粉症です ("Otsuyama Jiro has hay fever")

At first glance this looks even safer.

But this processing has a serious problem.

The reader has no way to tell that "花粉症 (hay fever)" is fake。

Whoever tallies the cases, and whoever evaluates the appropriateness of the response,
ends up doing their work on top of a falsehood。

"Hiding" is a fail-safe operation: information only decreases, and no lies are added.

"Substituting another plausible-looking value," by contrast, is an injection of falsehood. One might fairly call it poisoning the data.

Pattern 3: Redact Everything

■■■■は■■■です ("■■■■ has ■■■")

Yes, this is safe.

But the sentence now carries zero information — "someone is something" — and as data it is dead. There is no longer any point in handing it over at all.

The Principle: There Are Two Kinds of Words

This comparison can be distilled into the following principle.

There are two kinds of words.

Words that point to "who" (identifiers) — names, employee numbers, phone numbers, email addresses. These may be replaced with natural-looking fakes. The reader does not consume their truthfulness; the only thing that needs to be severed is the "link."

Words that tell "what happened" (attributes) — disease names, monetary amounts, performance reviews, events. For these there are only two choices: keep them, or honestly redact them. They must never be faked, because the reader's work depends on their being true.

In reality, of course, words do not split cleanly into these two categories.

"Head of Accounting" is an attribute — and at the same time, within the organization, a clue that points to a specific person.

Ages and dates, when combined, also edge closer to a "who" (these are called quasi-identifiers). This is precisely why the yardstick for the decision is not the word's type but its role.

It is also why the question in the flow above is not "is this word a name?" but "will the reader use the truth of this value to do their work?"

Rather than deciding the method mechanically from the detected type alone, we decide it by combining the role the word plays within the document with the recipient's intended use.

"Honestly redacting" means replacing the value with a label such as [Disease 1].

The point is to number the labels sequentially: the same disease name always receives the same number (we call this coreference preservation).

That way, the fact that "this document discusses two different diseases" — the structure the analysis needs — is preserved, while the reader is honestly told that "this part has been redacted."

There is also a middle path between the two choices: "generalization." As in "age 47 → 40s" or "full street address → city level only," it is a way of keeping the truth while lowering only its resolution.

Because it injects no falsehood and reduces only the identification risk, it is an honest middle option usable for both identifiers and attributes.

Putting all of this together, the de-identification methods can be organized as follows.

One note of caution here.

Credentials such as passwords and API keys — the last rows of the table — should be treated as "secrets," regardless of whether they qualify as PII.

On the shared file they are redacted, but if there is any possibility they have leaked externally, redaction is not the end of the story: revocation and rotation — that is, incident response — are separately required.

Conversely,

applying dummies to attributes (substituting one disease name for another, for example) is, as a rule, prohibited

.

If there is ever a situation that calls for realistic fake data, it is one where the recipient does not rely on the factuality of the original — parser testing, demos, load testing, and the like.

We do not call that de-identification; we treat it as a separate job called "synthetic data generation."

In the standards world, synthetic data generation is indeed counted among de-identification techniques (NIST SP 800-188), but

"masking that preserves meaning for handover" and "the job of crafting plausible fakes" should never be mixed into the same process

— that is the design decision we have adopted.

Note also that for anonymizing statistical databases there are standard techniques that deliberately alter attribute values, such as noise addition (adding random error to numeric values) (ISO/IEC 20889); these are positioned as tools for hiding each record's true value while preserving statistical properties in aggregate.
Bringing them into the domain this article addresses — documents and logs read directly by people and tools — causes the same problem as the injection of falsehood described above.

Four Pitfalls That Are Easy to Step Into in Practice

Now let us look at the points to watch when actually designing PII processing.

Here are four challenges you will genuinely encounter when implementing a PII-processing application.

1. Across Files, the Same Person Becomes Different People

De-identify indiscriminately and you can end up converting one person into several different people. The same person should remain traceable as the same person after de-identification. This is what we call coreference preservation.

Consider, for example, de-identifying person names that appear across multiple logs.

If coreference preservation is applied only "within a single file," then when a set of logs is processed, file A may map 甲野 (Kono) → 乙山 (Otsuyama) while file B maps 甲野 → 丙川 (Heikawa).

That breaks any analysis that follows the actors' movements across multiple logs.

To deal with this class of problem, you need

coreference preservation at the level of the whole set (the job)

.

There is a flip side, however.

The wider the coreference scope, the more the "appearance pattern of the same pseudonym = the original person's behavior pattern" survives — which also means more clues for identification. For that reason, we believe the scope of consistency should be selectable to match the use case.

This point applies not only to person pseudonyms but equally to the numbering of labels such as [Disease 1].

2. A Pseudonym Collides with a Real Person

If the generated pseudonym "乙山次郎 (Otsuyama Jiro)" happens to match a real Mr. Otsuyama who already appears in the document, the analysis is not merely confused — you risk pinning a false accusation on that real person.

This challenge requires a mechanism to

exclude values that actually appear in the document from the pool of dummy candidates

.

Natural person names, however, can never be guaranteed "not to exist." For recipients who cannot accept the risk of misattribution, choose a label such as [Person 1] instead of a pseudonym — this, too, is a "destination" decision, discussed below.

For machine-formatted values, moreover, the standard move is to generate "values that cannot exist".

Use reserved domains for dummy email addresses (example.com), documentation-reserved test ranges for IP addresses (such as 192.0.2.0/24), and check-digit-invalid values for various ID numbers — then even an accidental contact or lookup reaches no one.

3. De-identification Cannot Erase "Everyone Knows Who That Is"

Even with every name and number removed, the context alone — "the head of Accounting received a disciplinary action on July 3" — tells people inside the company exactly who it is. This frequently happens when de-identifying PII contained in email.

De-identification is not magic; it is a technique for reducing identification risk.

In that sense, the assumption that "processed = perfectly safe" may be the biggest pitfall of all.

Note that "pseudonymized information" and "anonymized information" are legal terms, with definitions and requirements under Japan's Act on the Protection of Personal Information.

For referring to technical processing, we consider it safer to use the terms "de-identification" and "masking," as this article does.

4. A Processed File Is an "Altered Document"

Deliverables from incident response can later become material for audits or legal proceedings.

Since a de-identified file is not the original,

a processing record documenting "when, under what rules, what was processed and in how many instances," together with a hash of the original,

should accompany the deliverable, so that its relationship to the original can be demonstrated to third parties. The record is not a mere disclaimer; it is an audit document that guarantees the transparency of the alterations.

The Right Answer Is Decided by the "Destination," Not the "Type"

Let us move on to the next point:

what to prioritize in PII processing.

Naturalness (analysis tools keep working), safety, and honesty (fakes are recognizable as fakes) — these three cannot all be maximized at once.

Which one to prioritize is determined by where the data is going.

• Deliverables read by people → prioritize honesty. Label replacement at the core, plus a processing record
• Logs analyzed by tools → prioritize preserving formats and digit counts. Dummies built from values that cannot exist
• Data fed into LLMs → prioritize naturalness. Dummies at the core (some designs pair this with restoration applied to the LLM's responses)

Incidentally, the LLM case is not complete with input-side substitution alone.

The substitution mapping table is never sent to the LLM; it is managed inside your own trust boundary.

Restoration is limited to managed tokens, never free-text string matching.

And because an LLM's responses can contain PII that was not present in the input, the output side must be run through PII inspection again. All of this together forms one complete set.

In other words, a de-identification configuration is not "a preference per data type" but
a declaration of "to whom, and for what purpose, this data is being handed"
.

In our PII-FI, we design this declaration as a per-destination "profile."

Conclusion

In this article we have laid out our thinking on what de-identification essentially is, illustrating the design challenges of PII processing along the way.

To close, let us restate the key points.

• De-identification is a technology for "handing data over": keep the "events," sever the "who"
• There are two kinds of words: identifiers may be pseudonymized; attributes offer three options — keep, honestly redact, or generalize — and must never be faked
• Injecting fake data into attributes is an "injection of falsehood"; the only place it belongs is the separate job of synthetic data generation
• In practice, watch four things: cross-file coreference preservation, pseudonym collisions with real people, residual identification risk, and the processing record
• The final answer is determined not by the word's type but by "to whom, and for what purpose, the data is handed".

We hope this article has been of some use in thinking through PII processing.

At Qualiteg, as a fast, high-accuracy PII detection and masking solution built on the design philosophy in this article, we are developing and offering "PII-FI Scan," a service that de-identifies whole sets of logs and documents via drag and drop, along with the "PII-FI API," which makes the same technology available through an API.

If you are interested in PII-FI's detection, masking, or de-identification capabilities, please feel free to reach out via our contact form.

See you next time!

References (Industry Standards and Guidelines)

The framework in this article is consistent with the thinking in the following standards and guidelines on de-identification. Please refer to them as well.

• NIST IR 8053: De-Identification of Personal Information — A report from the U.S. NIST broadly organizing research and practice on de-identification and re-identification. Covers not only structured data but also free text and images
• NIST SP 800-188: De-Identifying Government Datasets — A practical guide to de-identifying government datasets (final version, 2023). Covers quasi-identifier transformation, synthetic data, re-identification risk assessment, and governance
• ISO/IEC 20889:2018 — The international standard for the terminology and classification of de-identification techniques. Systematizes pseudonymization, generalization, suppression (deletion/masking), noise addition, and more
• ENISA: Pseudonymisation Techniques and Best Practices — A practical guide to pseudonymization from the EU cybersecurity agency. Details the trade-off between the usefulness of consistent pseudonyms (coreference preservation) and re-identification risk
• Personal Information Protection Commission: Guidelines on the Act on the Protection of Personal Information (Pseudonymized and Anonymized Information volume) — Definitions, requirements, and obligations for "pseudonymized information" and "anonymized information" under Japanese law

Related Resources

• High-Accuracy PII Detection (Part 1)
• Staged PII Masking When Using LLMs (Part 2)
• What the Confusion Matrix Cannot Show in PII Detection — Inter-Recognizer Collisions and Test Design
• Enterprise Information Defense in the LLM Era
• The Complete Guide to LLM Security in the Zero Trust Era
• DLP Technology in the AI Era (Part 1)
• DLP Technology in the AI Era (Part 2)

Introduction

This report is a comprehensive analysis of the performance of Japanese-capable LLMs, based on benchmark data from the Nejumi Leaderboard 4 (2026/7/10 edition).

Last time, we published an analysis report on the 2026/3/6 edition;
now, roughly four months on, this edition too has proved turbulent, with the lineup at the top changing dramatically.

(We update this LLM ranking regularly. You can receive update notifications by following our X (formerly Twitter) account.)

Nejumi Leaderboard 4 is known as a reliable benchmark that evaluates LLM performance on Japanese-language tasks from multiple angles. It is built on two axes — General Language Performance (GLP) and Alignment (ALT) — and covers a wide range of aspects, from translation, summarization, reasoning, and coding to toxicity, bias, and truthfulness.

This analysis covers both commercial API models and open models, examining the characteristics and trends of each. First, here are the three biggest topics of this edition.

• Claude Opus 4.8 breaks the 0.85 overall-score barrier, a first in the leaderboard's history
  Together with Claude Opus 4.7, Anthropic takes a one-two finish
  
• Nineteen models now score 0.80 or above overall
  (11 last time, 4 the time before) — the near-doubling pace continues
  
• Among open models, Gemma 4 surges into view
  New domestic entrants also appear, including RakutenAI-3.0 and llm-jp-4
  

On "open-source" models

Models with open weights are sometimes called "open-source models" or "OSS models," but since not all of them disclose their training data and training code in full, this article uses the term "open models" throughout.

On benchmark analysis

This report presents trends and characteristics that can be read from benchmark data, as reference information for LLM selection. For actual deployment, we recommend validating in a real environment suited to your use case.

This edition's tally is based on export data for the leaderboard's top 100 entries. Evaluation runs of the same model with different settings, such as reasoning/thinking modes, may appear as separate entries.

With that, let us look at the overall ranking of Japanese-capable LLMs as of 2026/7/10.

Overall Score Ranking: TOP 50

* Rows in light blue are open models.
* As of today (7/10), Fable 5 and GPT 5.6 have rolled out, but there is a lag before new models appear in the benchmark, so they are not yet listed. Please bear this in mind.

Overall Score: Trends and Analysis

The biggest news this time is, without question,
the one-two finish by Claude Opus 4.8 (0.8523) and Claude Opus 4.7 (0.8509).

This is the first time an overall score has exceeded 0.85 on Nejumi Leaderboard 4 — and two models from the same vendor achieved it simultaneously.

• The top two both score 0.85 or above, an unprecedented level (the previous edition's leader stood at 0.8430)
• The top 19 score 0.80 or above (11 models last time)
• Six of the TOP 10 slots are newcomers absent from the previous ranking

Anthropic finally breaks through the "0.85 wall"

The leader, Claude Opus 4.8, achieved high marks on both axes: GLP (General Language Performance) 0.8328 and ALT (Alignment) 0.8933. Its mathematical reasoning of 0.975, abstract reasoning of 0.87, and 0.6875 on the SWE-Bench-style evaluation of practical coding ability are top-class among all models — the strength of its reasoning is what lifts the score.

Claude Opus 4.6 (0.8394), second last time, holds on at sixth, leaving Anthropic with three of the TOP 10 slots (Opus 4.8, Opus 4.7, and Opus 4.6). Gemini 3.1 Pro (0.8430), the previous leader, slips to third with its score unchanged.

OpenAI gives chase with GPT-5.5 and GPT-5.4

GPT-5.4 (0.8397), which we noted last time as "too new to have scores listed," comes in fifth, and its successor GPT-5.5 (0.8411) takes fourth. With GLP 0.8303, GPT-5.5 closes in on Opus 4.8 in language performance, and its SWE-Bench-style score of 0.6875 matches Opus 4.8.

Interestingly, for GPT-5.4 the "high" reasoning-effort setting (0.8397) outscored the "xhigh" setting (0.8286) — a good illustration that more reasoning does not always mean a higher score.

* The true latest models are yet to come

As of today (2026/7/10), the true latest and highest-end models — Claude Fable 5 and ChatGPT 5.6 — do exist, but they are expected to appear in the benchmark a little later, so please note that they are not yet in this ranking.

Six of the TOP 10 Slots Change Hands

As the chart above shows, six of the TOP 10 are faces that were not in the previous ranking. Here are the newcomers most worth noting.

• Qwen3.6 Max (7th, 0.8295)
  Alibaba's flagship API model. Its ALT of 0.9181 is the highest of any model, with a standout balance of controllability, low toxicity, and robustness
  
• Gemini 3.5 Flash (10th, 0.8249)
  A light, low-cost Flash-class model that outscores the previous edition's Opus 4.5 (0.8064). Its truthfulness of 0.550, however, is markedly the lowest among the top group, so hallucination countermeasures are needed on the user side
  
• GLM-5.2 (20th, 0.8040)
  Z.ai's latest API model. Newly released in June, it outscores the open GLM-5 (0.7884)
  
• DeepSeek V4 Pro (23rd, 0.7956)
  DeepSeek's latest API flagship. Reasoning has been reinforced, with mathematical reasoning at 0.94 and abstract reasoning at 0.79

"High Performance, Yet Sinking Overall" — the Lesson of grok-4.20

The scatter plot above places the major models on two axes: GLP (language performance) and ALT (alignment).

Every model at the overall top sits in the upper right — that is, they combine "intelligence" with "safety and controllability".

The contrast is xAI's grok-4.20-0309-reasoning (0.7732).

Its GLP of 0.7942 is TOP 10-class language performance, but its ALT is low at 0.7659 (notably toxicity 0.566 and truthfulness 0.485), sinking it to 40th overall.

Because alignment carries substantial weight in the overall evaluation, "being smart alone" does not get a model into the top ranks — this case illustrates that structure clearly.

On interpreting benchmark results

The scores presented in this report are, in the end, results of benchmark tests. Benchmarks are a useful tool for objectively comparing LLM performance, but please note the following.

• Benchmarks evaluate against a specific set of tasks, so models well suited to that task mix tend to score higher
• The practical feel in real work, and usefulness for specific purposes, cannot be fully measured by benchmark scores alone
• Some models have unique strengths and characteristics that the benchmark does not measure

Next, let us narrow the field to open models and see how they fare.

Open Models: Overall Score Ranking TOP 20

* Rankings are per model (where a model has multiple evaluation runs, only its best score is used).

Open Models: Trends and Analysis

Qwen3.5-397B-A17B holds the lead, but the gap to the API camp widens again

The top open model is once again Alibaba's Qwen3.5-397B-A17B (0.8191).

The Qwen3.5 series still carries its scores from its February 2026 release, meaning the open-side frontier saw no update over these four months.

Meanwhile the API-side leader climbed from 0.8430 to 0.8523, so the gap between the open and API leaders has widened again, from roughly 0.024 to roughly 0.033.

A reversal from the excitement last time that "open models are closing in on the APIs" — this time the frontier API camp has pulled away.

That said, the open field is steadily deepening: the number of open models above 0.80 grew from three to four.

The Gemma 4 Shock — a 31B Dense Model Above 0.80

That fourth model is the biggest news in this edition's open division: Google's Gemma 4 family. A new generation released in March 2026, published on Hugging Face under the Apache 2.0 license.

• Gemma 4 31B (0.8077)
  A 31B dense model that surges to third among open models. It delivers the same level as GPT-5.1 (0.8085) and Claude Opus 4.5 (0.8064) at a size that runs on a single GPU
  
• Gemma 4 26B-A4B (0.7872)
  An MoE version with 26B total and 4B active parameters. Roughly 0.79-class performance while sharply reducing inference cost
  
• Gemma 4 E4B / E2B
  Compact on-device versions round out the lineup (covered in the small-model section)

Until now, the leading candidate for a 30B-class local LLM was Qwen3.5-27B (0.8049); with Gemma 4 31B now surpassing it, the single-GPU class has two strong options side by side.

New Chinese entrants: GLM-5, Kimi K2.5, and MiniMax-M2.1

As before, the depth of China-origin open models is overwhelming. This edition brings three large newcomers.

• GLM-5 (0.7884)
  Z.ai's latest generation: an MoE with 744B total and 40B active parameters, adopting DeepSeek Sparse Attention (DSA). Generously released under the MIT license
  
• Kimi K2.5 (0.7785)
  Moonshot AI's 1T-total, 32B-active MoE. A natively multimodal model with an integrated vision encoder, designed with agentic use squarely in mind
  
• MiniMax-M2.1 (0.7556)
  A 229B-total MoE, with a character specialized in SWE-Bench-style coding and agentic processing

The pattern of five Chinese players — Qwen, DeepSeek, GLM (Z.ai), Kimi (Moonshot), and MiniMax — filling the open TOP 12 holds again this time, with Google's Gemma 4 now wedged in among them.

New Developments Among Japanese Open Models

This edition also brought noteworthy movement among open models from Japan.

• RakutenAI-3.0 (0.6761)
  An MoE with 671B total and 37B active parameters released by Rakuten Group in March — among the largest domestic open models. It adopts a DeepSeek-V3-family architecture and was built on Japanese-English bilingual data (Apache 2.0)
  
• llm-jp-4-32b-a3b-thinking (0.6679)
  An MoE reasoning model with 32B total and roughly 4B active parameters from the National Institute of Informatics (NII). An ambitious, fully domestic effort trained from scratch on 11.7 trillion tokens
  
• Reinforcement-learning versions of the Swallow series
  The RL versions from the Swallow team at Institute of Science Tokyo (formerly Tokyo Tech) have expanded. GPT-OSS-Swallow-120B-RL-v0.1 (0.6914) is top-class among domestic models, followed by Qwen3-Swallow-32B-RL-v0.2 (0.6782) and others

Their overall scores still sit short of the 0.70 wall, but training data optimized for Japanese and ease of domestic operation are value the scores do not show.

It is also welcome news for Japanese users that K-EXAONE-236B-A23B (0.7186) from Korea's LG AI Research includes Japanese among its officially supported languages.

Next, let us look at mid-size models, roughly 10B to 30B.

The appeal of mid-size models is that they can run on GPUs that are relatively affordable even for individuals.

Mid-size Models (10B-30B): Overall Score Ranking

* Because the tally is based on the top-100 export data, some previously listed models (such as the gemma-3 family) fall outside this edition's scope.

Mid-size Models: Trends and Analysis

In the mid-size category, the previous champion Qwen3.5-27B (0.8049) defended first place, but Gemma 4 26B-A4B (0.7872) debuts in second, turning this into a two-horse race.

Because Gemma 4 26B-A4B is an MoE with only 4B active parameters, its inference load is far lighter than a dense 27B model's.

A sensible division of labor seems to emerge: Qwen3.5-27B for performance, Gemma 4 26B-A4B for inference cost.

Fourth-place GPT-OSS-Swallow-20B-RL-v0.1 (0.6424) is OpenAI's open model gpt-oss-20b further trained for Japanese with reinforcement learning — a valuable Japanese-specialized mid-size model.

Finally, let us look at small models. These compact models run on relatively inexpensive consumer GPUs, making them the prime candidates for edge and on-premises use.

Small Models (Under 10B): Overall Score Ranking

Small Models: Trends and Analysis

Among small models, Qwen3.5-4B (0.7352) leads again. At just 4B, it posts a score level that a year ago would have taken a 30B-class model — emblematic of how far miniaturization has come.

For Japanese specialization: Nemotron Nano 9B v2 Japanese

In second place again, NVIDIA Nemotron Nano 9B v2 Japanese (0.7111) is NVIDIA's Nemotron Nano — a Mamba-2/Transformer hybrid — further trained for Japanese. It supports toggling reasoning on and off and controlling the volume of thinking tokens (thinking budget), and it is published under a commercially usable license. It is arguably the best Japanese-specialized small model available today.

Gemma 4's on-device versions: E4B / E2B

Gemma 4 E4B (0.6692) and E2B (0.6564) are compact versions intended to run on smartphones and edge devices. They reach this score band while keeping effective parameters low — the practical bar for "Japanese LLMs that run entirely on-device" keeps rising steadily.

Seventh-place Qwen3-Swallow-8B-RL-v0.2 (0.6552), a Japanese reinforcement-learning version from the Swallow team at Institute of Science Tokyo, also made its presence felt among domestic small models.

Conclusion: a Guide Toward Full-Scale Adoption

Once again we have analyzed the Nejumi Leaderboard 4 benchmark data. Here are the key takeaways from this edition.

The dawn of the "0.85 era" — and the democratization of 0.80

The biggest point this time is that Claude Opus 4.8 broke the 0.85 overall-score barrier for the first time in the leaderboard's history.

At the same time, models above 0.80 reached 19, continuing a near-doubling pace from 11 last time and 4 the time before.

A score of 0.80 is no longer "proof of the frontier" but "an admission ticket to the top group."

• Commercial APIs: the big three of Anthropic (Opus 4.8/4.7), Google (Gemini 3.1 Pro), and OpenAI (GPT-5.5/5.4) become a big four, with Alibaba (Qwen3.6 Max) forcing its way in
  
• Open models: Qwen3.5 holds the lead while newcomers such as Gemma 4, GLM-5, and Kimi K2.5 deepen the field
  
• Domestic models: Japanese-specialized options are steadily expanding, with RakutenAI-3.0, llm-jp-4, and the Swallow RL versions

Characteristics by Model Size

Toward Full-Scale LLM Adoption: Choosing the Right Model for Each Use Case

This report has mapped each model's position around overall scores, but real adoption decisions are not that simple.

Benchmarks are strong reference material, yet practice demands multifaceted evaluation: not just accuracy but deployment cost, inference cost, response speed, handling of confidential data, and integration with existing systems. Licensing, too — what is called "open source" or an "open model" in name can conceal pitfalls on closer inspection.

Especially in a moment like this one, when the options have expanded rapidly, what matters on the ground is less "picking the single highest-scoring model" than "building the ability to use multiple models appropriately across use cases."

To support this kind of multi-model operation in practice, we provide our integrated AI platform Bestllam , which lets you use multiple LLMs on a single platform.

We can support you end to end — from clarifying the issues around commercial use and selecting models, to embedding them in business workflows and designing operations built around AI agents.

Beyond simply providing tools, we also offer BPR consulting aimed at AI-enabling your business operations, accompanying you not only on AI technology but from upstream business analysis through AI-native BPR planning, success-metric design, and deployment support.

Please feel free to get in touch.
https://qualiteg.com/contact?inquiry=consulting

Hello!

Claude Fable 5, which arrived in June 2026, has had an eventful first month: a suspension under export controls right after release, a global re-rollout, and then its departure from the subscription allowance.

This blog has followed the situation at each turn.

For the overall picture, see Finally Generally Available: Reading Claude Mythos 5 / Fable 5 from a Practitioner's Perspective; for the suspension drama right after release, see Suspended Three Days After Release — the U.S. Government Directive on Fable 5 / Mythos 5 and the New Availability Risk of AI; and for pricing and the outlook, see What Happens Next for Claude Fable 5? Background, Costs, and Outlook.

This article builds on those to serve as "the definitive guide for practical use."

In particular,

as of July 12, 2026 (July 13 Japan time), it leaves the subscription allowance and cannot be used unless usage credits are enabled
(This deadline was initially set at July 7, 2026, and was later extended by five days to July 12.)

— with this major pricing change as the axis, this article brings together the model specifications, the realities of pricing, and the operational points to watch when using it in Claude Code. Including how to divide roles with Opus 4.8, we provide the material for judging

"how much Fable 5 is worth paying for"

for yourself.

This guide therefore takes Anthropic's official announcements and documentation as its baseline, and organizes
Claude Fable 5 (claude-fable-5) around the three questions of

"understanding it as a model,"
"grasping the pricing and the treatment of the subscription allowance,"
and "operating it economically in Claude Code by combining Fable with less expensive models"

.

With that, let us get into the main content.

Incidentally, in the early morning of this article's publication date (2026/7/10), the weekly rate limit was abruptly lifted, effectively expanding Fable 5's usage allowance once again

Table of Contents

• Part 1: What Is Claude Fable 5?
  • Where Fable 5 fits
  • Timeline: announcement, export controls, re-rollout
  • Specifications as a model
  • Constraints to check before enterprise use (data retention)
  • How to read the benchmarks
  • Key API differences
  • Relationship to Mythos 5
• Part 2: Pricing and the move out of the subscription allowance (most important)
  • The change in delivery and the deadline extension from July 7 to July 12
  • How usage credits work and the management screen
  • Limits, balance, and prepaid discounts (each a separate thing)
  • Cost estimates, and how to measure for yourself
  • Price comparison (as of July 9, 2026)
• Part 3: Fable 5 in Claude Code
  • Prerequisites and model selection
  • Enabling credits and setting limits
  • effort, the 1M context, and prompt caching
  • Refusals and fallback in practice
  • Using Fable 5 realistically in Claude Code — splitting models into commander and workers
  • Handling false positives from the safety classifier
• Conclusion — how to live with Fable 5

Part 1: What Is Claude Fable 5?

Where Fable 5 Fits

Claude Fable 5, announced by Anthropic on June 9, 2026, is the most capable model the company currently makes generally available (GA).

The official documentation positions Fable 5 as

"Anthropic's most capable broadly available model, designed for the most advanced reasoning and long-horizon autonomous (agentic) work"

.

The key concept is "long-horizon agentic work" (long stretches of autonomous work).

What distinguishes Fable 5 is that its lead over other models widens as tasks grow longer and more complex, more than in one-off Q&A.

As an example cited at announcement, the payments company Stripe stated that it "completed a migration of a Ruby codebase exceeding 50 million lines — work that would normally take a team more than two months — in one day," reported as an early customer example (note that this is Stripe's report, not a general performance guarantee).

Fable 5 is "Mythos-class" — that is, it carries capabilities from the same lineage as Anthropic's top research model, released with safeguards built in to make it suitable for general use.

Why It Is Not "an Upgraded Opus 4.8"
Fable 5 is not Opus's successor. Opus 4.8 (claude-opus-4-8) remains "the newest and highest model in the Opus tier," and for most practical work it is the standard choice. Fable 5 sits above the Opus tier — a separate bracket in both price and capability. So the answer to the common request "I want to upgrade to the latest model" is still Opus 4.8. Understand Fable 5 as the model you choose explicitly when you want the hardest problems solved, with the cost accepted up front.

Timeline: Announcement, Export Controls, Re-rollout

Fable 5 followed an unusual path entangled with national security and export controls.

Here is the chronology.

For the re-rollout, Anthropic introduced a new safeguard classifier. Per the official explanation, "the specific technique described in Amazon's report is now blocked in more than 99% of cases," while "the safety margin has been expanded substantially" — with the result that more benign requests are also blocked than before.

Anthropic explicitly states

"In the near term, some routine tasks like coding and debugging may be flagged more often."
(meaning: for the time being, everyday tasks such as coding and debugging may be flagged more often than before)

. Because this matters in practice for using Fable 5 in Claude Code, we revisit it in Part 3.

The suspension under export controls and the re-rollout themselves — and the "new availability risk of AI" they reveal — are explored in depth in

Suspended Three Days After Release — the U.S. Government Directive on Fable 5 / Mythos 5 and the New Availability Risk of AI

.

Specifications as a Model

Here are the key specifications, based on the official documentation. Unit prices are as of July 9, 2026, as noted below.

• The 1M context comes at standard pricing, with no surcharge (premium) for long context.
• The tokenizer is identical to Opus 4.8's, so if you migrate from Opus 4.7 / 4.8, the token count for the same text is essentially unchanged.
  What changes is the unit price. If you move from Opus 4.6 or earlier, Sonnet, or Haiku, token counts will shift, so re-measure with count_tokens.

Constraints to Check Before Enterprise Use (Data Retention)

Fable 5 comes with the following constraints.
If they do not immediately click on first reading, that is fine.
Below them we have added a plain-language explanation.

Fable 5 / Mythos 5 are "Covered Models," and 30-day data retention is mandatory. They cannot be used under zero data retention (ZDR).

More than performance or price, this is the constraint that decides whether enterprise adoption is possible. Before adopting, check at least the following.Do your contracts and internal rules permit sending confidential code and data to Fable 5?Existing ZDR-based workloads cannot be migrated as they are. Thirty-day retention is required on every serving platform, but 400 invalid_request_error is officially documented only for Claude API (when the organization/workspace retention settings do not meet the requirement). Display and error behavior via Claude Code or each cloud varies by endpoint (for example, Fable disappearing from the model choices).For each connecting platform — Bedrock / Vertex / Foundry and so on — confirm where and under what conditions data is retained.

Plain-language explanation → the basic rule

What the above means is this: the text we send to Fable 5, and the text Fable 5 returns, is always stored for 30 days. You cannot delete it. You cannot opt out.

Other Claude models (Opus 4.8 and so on) can be set to "do not store." Only Fable 5 cannot.

Why is it stored?

Because Fable 5 is powerful, it is monitored for abuse. Abuse cannot be spotted in a single exchange; only when hundreds of exchanges are laid side by side does it become clear that "this is an attack." The stored data is not used to train AI.

What is the problem?

Some companies hold contracts (ZDR) stating "none of our data is ever stored." That contract does not apply to Fable 5. The retention requirement takes precedence over the contract.

In other words, if you send confidential source code or customer data to Fable 5, it will remain somewhere for 30 days. And if it is judged dangerous, a human may look at its contents.

How to Read the Benchmarks

For benchmark details, we defer to our separate article below.

Finally Generally Available: A Practitioner's Read on Claude Mythos 5 / Fable 5

Hello! This is the Qualiteg Product Development Team. On June 9, 2026, Anthropic announced Claude Fable 5 and Claude Mythos 5. In this article, we sort out what…

Qualiteg BlogQualiteg コンサルティング

Key API Differences

First, let us look at the API-level differences from Opus.

Fable 5 largely shares the API request surface with the Opus tier (4.7 / 4.8), but there are differences that can turn existing implementations into 400 responses on migration.
Here are the main ones (not exhaustive).

1. Thinking is always on

Adaptive thinking is the only thinking mode, and omitting thinking enables it automatically.Explicitly sending thinking: {type: "disabled"} returns a 400 (specific to Fable 5; disabled is accepted on Opus 4.7 / 4.8).budget_tokens has also been removed and returns a 400 if sent. Depth is controlled with output_config.effort (low to max).

2. Sampling parameters are not supported

temperature / top_p / top_k return a 400 if you send non-default values. Steer output tendencies through the prompt.

3. max_tokens is the total of thinking plus the final answer

max_tokens is a cap that includes both thinking tokens and the final answer. At high effort, thinking consumes many tokens, so a small cap will cut the answer off midway.

4. Raw chain of thought is not returned

What comes back is a normal thinking block; with display: "summarized" it contains a summary, and with "omitted" (the default) it is an empty string. Thinking runs and is billed regardless of the display setting.

5. A stop reason called refusal

Fable 5 runs a safety classifier over the input. When refused, you get an HTTP 200 success response, not an error, with stop_reason: "refusal" returned and stop_details carrying the category.

• Before output — refusal: content is empty, and there is no charge.
• Mid-stream refusal: the portion already generated is billable (discard the partial output; do not use it).

response.content[0] — code that reads this unconditionally will break.Always check stop_reason first. In practice, judge in the following order: (1) HTTP status → (2) stop_reason → ③stop_details logged → (4) content confirmed non-empty → (5) on refusal, do not use the partial output.

6. Fallback (rescuing refusals with another model)

Server-side fallbacks automatically re-runs the same request on another model upon a policy refusal — an opt-in beta feature (server-side-fallback-2026-06-01) that must be specified explicitly (it is not on by default).

Currently it is available on the Claude API and Claude Platform on AWS;

on the Message Batches API, Amazon Bedrock, Google Cloud, and Microsoft Foundry it is not available,

requiring client-side re-execution or SDK middleware. The model to fall back to is specified explicitly in the request's fallbacks array.
(The official example is Opus 4.8; the Anthropic API does not automatically pin the fallback to Opus 4.8.)

What differs by endpoint is "whether server-side fallbacks can be used." If you are writing new claude-fable-5 code, we recommend putting this fallback in explicitly.

Relationship to Mythos 5

Claude Mythos 5 (claude-mythos-5) shares the same base model, context length, maximum output, and pricing structure with Fable 5.

However, it does not carry the safety classifiers added to Fable 5, so the actual API behavior, including refusals and automatic fallback, is not identical
(the refusal and fallback descriptions apply to Fable 5 only).

Mythos 5 is limited to Project Glasswing participant organizations and the like; general users use claude-fable-5.

It is the successor to the formerly invitation-only claude-mythos-preview. For the difference in positioning between Mythos 5 and Fable 5 and a reading of the benchmarks, see also Finally Generally Available: Reading Claude Mythos 5 / Fable 5 from a Practitioner's Perspective.

Part 2: Pricing and the Move out of the Subscription Allowance (Most Important)

The Change in Delivery and the Deadline Extension from July 7 to July 12

When Fable 5 was re-released on July 1, its inclusion in subscriptions came with a deadline.

That deadline was initially July 7, 2026, but just before it arrived, Anthropic abruptly extended it by five days, finally setting it at July 12, 2026 (23:59:59 U.S. Pacific Time; July 13 Japan time).

The cap of up to 50% of the weekly allowance, and the switch to credit-based use after the deadline, have been unchanged since the July 1 re-release.

The cutoff is 23:59:59 on July 12 (U.S. Pacific Time; the afternoon of July 13 in Japan). Confirm the actual switchover timing on your account screen (described below) as well.

On the possibility of returning to subscriptions
Anthropic has indicated it would like to bring Fable 5 back into the subscription allowance if capacity can be secured, but neither the timing nor the implementation is guaranteed. Deadlines have moved once already, so the realistic approach is to keep checking the latest official guidance while planning operations on a credit basis for now.

How Usage Credits Work and the Management Screen

Usage credits are a pay-as-you-go layer available on paid plans such as Pro / Max. Originally designed so you could "keep working after hitting your plan limit," after July 12 (from July 13 Japan time) they become the entry point to Fable 5.

Note, however, that "general extra usage credits (for continuing after hitting the plan cap)" and "the delivery of Fable 5 after July 12" are officially separate announcements.

Anthropic describes the latter as "provided via usage credits," and we will not assert the specific consumption order — whether Fable 5 draws down the normal allowance first before moving to credits, or consumes credits from the start. Confirm actual allowance consumption on your account screen and in the latest official guidance.

Management happens in claude.ai under Settings → Usage. Looking at the actual screen (with the subscription allowance used up and credits turned on) makes the mechanics concrete.

Who can configure this differs by plan. The screen examples and steps below assume a Pro / Max individual account.Team and seat-based Enterprise plans: the Owner / Primary Owner enables credits and sets limits from Organization settings → Usage.usage-based Enterprise is not a use-up-included-allowance scheme; in principle it is billed per token from the first token.

* Display example from our own account in early July 2026. You can see the Fable 5 subscription allowance has been used up and usage has moved to the credit allowance.

Here is what can be read from this screen.

• Usage allowances are displayed per model. "All models" and "Fable" appear as separate bars; in the example above, the Fable allowance is 100% used (the subscription allowance is exhausted).
• Usage credits can be toggled ON/OFF. The framing is that you turn credits on "to keep using Claude even after reaching your limit."
• Usage is displayed in real time (e.g., $68.69 used / 27% of the limit, with the reset date).

Limits, Balance, and Prepaid Discounts (Each a Separate Thing)

"If I turn credits on, won't spending run away without limit?"

You may worry about that, but
you can set a spending limit on credits

With credits on, there are three concepts to know

• Monthly spending limit
  The cap on credit spending for the month (e.g., $250.00). You can change it via "Adjust limit." However, "no limit" is also selectable, so merely enabling credits does not guarantee a spending cap. Especially if you use auto-reload, always confirm the monthly limit is set to the amount you intend.
  
• Credit balance
  The balance you currently hold (e.g., $181.31).Auto-reloadAn ON/OFF switch controlling whether the balance is automatically topped up as it runs down, or replenished manually (auto-reload combined with "no limit" can effectively become unbounded).
  
• Discount bundles (prepurchase)
  A "bulk purchase" product separate from limits and balance. Think of it as charging up your billing allowance in advance.
  
  $50 worth = 10% off (pay $45)
  $250 worth = 20% off ($200)
  $1,000 worth = 30% off ($700)
  
  For example, buying $250 worth applies the 20% discount, so you pay $200. In practice, 10% consumption tax is added, for a payment of $220.
  
  

In short, Fable 5 costs are managed with separate levers: the monthly limit (no limit unless you set one), the held balance, auto-reload, and prepurchased discount bundles.

Set a monthly limit to avoid unbounded spend, and charge up in advance to buy your billing allowance at a modest discount — that is the shape of it.

Cost Estimates, and How to Measure for Yourself

Here are simple estimates based on Fable 5's unit prices (input $10 / output $50 per 1M).All assume no caching, no retries, no fallback, and no tax or exchange-rate effects.

Case A: chat-centric (200K input + 50K output tokens per day)

• Input: 0.2M × $10 = $2.0
• Output: 0.05M × $50 = $2.5
• About $4.5 per day → about $135 per month

This is a simple estimate that assumes all Fable 5 usage is billed to credits.

Case B: autonomous coding in Claude Code

What drives cost is less chat than agentic coding. Long Claude Code sessions repeatedly load context, call tools, and self-verify, so token consumption is large.

Actual consumption, however, varies greatly with the model, codebase size, parallelism (number of subagents), caching, retries/fallback, and usage pattern.

The official documentation likewise explains that costs vary substantially with these factors.

Generalizations like "X dollars per hour" are unreliable unless measurement conditions match.Measuring in your own environment is the dependable way.

Concretely, record the following per session and apply the price table (input $10 / output $50 / cache reads $1 / cache writes $12.50–$20, all per 1M):

• Model used / input tokens / output tokens
• Cache-creation tokens / cache-read tokens
• Fallback count / number of subagents / session duration

Fable 5 costs twice Opus 4.8 per token, so with regular use, credit consumption can reach a scale that cannot be ignored.

That is exactly why the Part 3 question of "when to use Fable 5" is itself cost management.

Price Comparison (as of July 9, 2026)

• Sonnet 5 carries an introductory price of $2 input / $10 output through August 31, 2026 (the price currently in effect). The standard price from September 1 is $3 input / $15 output.
• Fable 5's unit price is exactly double Opus 4.8's, placing it among the most expensive of the mainstream generally available models (not "the most expensive ever" — models with higher unit prices, such as Opus 4.1, have existed, and Anthropic itself describes Fable 5 as less than half the price of the old Mythos Preview).

"Double the unit price" and "double the effective cost" are different things

This is the easiest point to misread. The $10 / $50 above is a comparison of token unit prices, and the actual cost per task will not necessarily stay within 2x. There are two main reasons effective cost tends to exceed the 2x price ratio.

1. Fable 5 has Adaptive thinking always on.
   Thinking tokens are billed as output ($50 / 1M on Fable 5), and max_tokens is the sum of thinking plus the final answer. Opus 4.8, by contrast, can turn thinking off (thinking omitted runs without thinking), so for the same task Fable 5 tends to consume more of the expensive side — output tokens. Because tokens grow on the high-priced output side, the cost compounds.
   
2. Fable 5 is designed to think deeply and at length.
   On hard tasks a single request can easily run several minutes to well over ten, and thinking and output tokens accumulate accordingly. The longer the autonomous task, the larger this gap grows.

In other words, effective cost is determined by "2x unit price × growth in generated tokens." The multiplier can be 2x or more (higher still at high effort). Budget on the assumption that "paying double Opus 4.8 buys the same work" and the invoice will exceed expectations.

Levers that actually reduce cost

• effort: turn it down. Fable 5 remains strong even at low effort, and this directly reduces thinking tokens (the most effective lever on effective cost).
• max_tokens: constrain it appropriately, or control the tokens of the whole loop with Task Budgets (API only).
• Use the measurements from the previous section to track input, output, thinking (included in output), and cache separately.
• Note: prompt caching discounts only the input side. The thinking and output tokens that tend to grow are not made cheaper by caching, so do not assume "caching will keep it within 2x."

The question of whether to choose Fable 5 is therefore not "the unit price is double" but
"do I want to solve problems Opus 4.8 cannot reach, even at a cost that can effectively exceed 2x?".

Part 3: Fable 5 in Claude Code

Here we consider the scenarios for putting Fable 5 to serious use in Claude Code.

Prerequisites and Model Selection

Fable 5 is a supported model in Claude Code. First, the prerequisites.

• Check the supported version. Per the official documentation, Fable 5 requires a relatively recent Claude Code. In practice: use the latest Claude Code CLI.
  
• The default model differs by plan
  Fable 5 is not the default model on any plan, so if you want it, select Fable 5 explicitly with the /model command.

Enabling Credits and Setting Limits

As noted above, after July 12 (from July 13 Japan time), calling Fable 5 from Claude Code requires credits to be enabled on the claude.ai side.

The 1M Context and Prompt Caching

Fable 5 offers a 1M-token context at standard pricing, but the more you load, the more tokens (= cost) you consume. For large fixed context (codebases, specifications), use prompt caching. Its pricing needs to be understood, though.

• Cache-hit reads are $1 / 1M — inexpensive.
• On the other hand, the initial cache write costs $12.50 / 1M at 5-minute retention, or $20 / 1M at 1-hour retention.

In other words, unless the fixed context is referenced repeatedly, caching actually costs more. It pays off precisely in workflows that reuse the same prefix many times.

Task Budgets cannot be used in Claude Code
The API offers Task Budgets (beta), which conveys a token budget for the entire agentic loop to the model, but this is a Messages API feature and is not available in Claude Code / Cowork. It is also a "guideline" conveyed to the model, not a hard cap. It applies only if you implement your own Messages API agent.

Refusals and Fallback in Practice

Because the re-released Fable 5 has widened its safety margin, benign tasks such as coding and debugging are more likely than before to be refused as false positives.

• If you use Claude Code as-is
  When the endpoint can correctly identify Fable 5 and the fallback Opus model, it falls back automatically on refusal.
  It is, frankly, quite sensitive and produces false positives fairly often. More on that below.
  
• If you build your own agent against the API
  stop_reason == "refusal" must be handled, and the opt-in server-side fallbacks should be specified explicitly. It is available on the Claude API and Claude Platform on AWS; elsewhere (Batches, Bedrock, Vertex, Foundry) you need client-side re-execution or SDK middleware. Specify the fallback model explicitly in the request's fallbacks array (the official example is Opus 4.8). What differs by endpoint is availability.

Using Fable 5 Realistically in Claude Code —

Commander (Fable) and Workers (Opus, Sonnet): Split Your Models by Role

After the pricing change, Fable 5 is expensive (2x Opus 4.8, and with thinking always on the effective cost tends to climb further); having it do absolutely everything — file reading, trial and error, long autonomous runs — is frankly not economical.

What we at Qualiteg actually do is split a single Claude Code session into a "commander" and "workers," assigning a different model to each role.

The basic idea is simple

• Commander = Fable 5
  Devotes itself to deciding policy, decomposing tasks, issuing work orders, accepting or rejecting reports, and judging the next move. It does not read code or logs itself, and it does not implement.
  
• Workers = Opus 4.8 / Sonnet 5 subagents
  Take on all the hands-on work — exploration, investigation, implementation, testing, documentation — and return only short reports to the commander.
  
• Use two tiers of workers
  Work requiring judgment or design understanding goes to Opus; fully specified, mechanical routine work (bulk renames, log collection, and the like) drops to Sonnet.

The aim is plain: the expensive model's value is concentrated in "judgment".

Let the process (large volumes of file reading, trial and error, logs) be digested inside the subagents' context and have only conclusions returned to the commander, and Fable 5's credits and context are spent only on the "moments of judgment."

Moving from "Fable 5 does everything" to "Fable 5 thinks, cheaper models do the hands-on work"

alone cuts credit consumption substantially.

Of course, for small day-to-day fixes it is faster and cheaper to skip the commander entirely and work directly with the default models (Opus 4.8 / Sonnet 5).

The commander method pays off in the "hard and heavy" situations: large migrations, large implementations with clear acceptance criteria, and large investigations using parallel subagents.

This Fable commander method — including setup steps, the shape of delegation prompts, orchestrating parallel subagents, and pinning models via CLAUDE_CODE_SUBAGENT_MODEL — will be covered in detail in a separate article, so stay tuned.

For this article, the one idea to take away is: put the expensive Fable 5 in the commander's seat, and leave the hands-on work to Opus / Sonnet.

Handling False Positives from the Safety Classifier

The re-released classifier widens the safety margin around areas such as cybersecurity and biology/chemistry.

Here it is important not to conflate "intentional blocks" with "false positives".

1. Treat refusals as "content results," not "errors"
   stop_reason — check it, and on refusal route to fallback or a switch to Opus 4.8.
   
2. Clearly intended block targets
   Building exploit chains, generating malware, creating attack tools — requests that directly provide offensive cyber capability are, naturally, what Fable 5's classifier intentionally restricts. These are design-level restrictions, not false positives.
   
3. Safety-margin blocks and false positives
   Vulnerability research, penetration testing, and PoC development are ambiguous areas that can serve attack or defense depending on content. Because Fable 5 judges broadly on the safe side, legitimate defensive work can also be blocked. In addition, ordinary coding, debugging, configuration changes, dependency updates, and other inherently unproblematic work can get caught up as well.
   Anthropic characterizes refusals of such harmless requests as false positives and says it will improve toward "better distinguishing true abuse from legitimate requests." In ambiguous areas or suspected false positives, restate the request with the legitimate purpose, ownership, and scope of authorization made explicit; if it is still refused, consider fallback or switching to Opus 4.8.

Related articles: our firsthand account of Fable 5 on Claude Code detecting, refusing, and reflecting on "an attack that never came" is in The AI "Detected," "Refused," and "Reflected On" an Attack That Never Came, and our treatment of legitimate operational work being judged a Usage Policy violation is in Why Legitimate Operations Work Gets Flagged as a "Usage Policy Violation" in Claude Code.

Conclusion — How to Live with Fable 5

Claude Fable 5 is the most capable model Anthropic makes generally available. Five key points.

1. Fable 5 is not Opus's successor; it is a separate bracket above it
   The default models (Opus 4.8 / Sonnet 5) for everyday work, Fable 5 only for the hard parts — that is the baseline.
   
2. It stays within the subscription allowance only through July 12, 2026 (July 13 Japan time)
   Past that deadline, it cannot be used unless credits are enabled on claude.ai.
   That said, dates and policy may change again at the last minute; this is not necessarily permanent and deadlines can move, so keep checking the latest official guidance while planning around credits for now.
   
3. Manage credits by setting your own limit
   (without one, spending can be "unlimited"). The monthly spending limit, held balance, auto-reload, and discount bundles (up to 30% off) are separate levers. The unit price is double Opus 4.8, but always-on thinking increases token consumption, so effective cost can exceed 2x (effort is the most effective lever to turn down). Always measure.
   
4. The longer, more autonomous, and more complex the task, the stronger it is
   State the goal and constraints up front, use appropriate effort, and exploit parallel subagents. But wherever possible, use Fable 5 for "thinking and judgment" and Opus / Sonnet as the hands — that keeps things economical.
   
5. Mind the safety classifier
   Benign coding/debugging can be refused (false positives). Attack code, authentication bypasses, and the like, by contrast, are intentional restrictions rather than false positives, and may not pass even for legitimate business.stop_reason must be handled; prepare a fallback, and route the affected areas to another model such as Opus 4.8.

"Do I want to solve problems Opus 4.8 cannot reach, even at a cost that can effectively exceed 2x?"
Commit Fable 5 only to tasks where the answer is yes. And rather than letting Fable 5 do everything, split the commander and worker models. That is the realistic way to live with this powerful — and expensive — model.

Closing — Staying Steady amid a Shifting Model Landscape

The whole sequence around Fable 5 (suspension under export controls, global re-rollout, departure from the subscription allowance, and hard-to-read effective costs) once again drove home the risk of depending heavily on a single model.

The higher a model's performance, the faster its availability and pricing can change. That is precisely why the question "if this model becomes unavailable or expensive, can we switch to an alternative immediately?" matters.

Qualiteg's consulting services draw on a deep store of practical AI-native development knowledge, including the insights introduced here, and we provide support for AI-native software development process innovation leveraging AI agents, and for making full use of Claude Code.

If you are interested, please get in touch with Qualiteg.

Building on our knowledge of frontier AI models, AI-native development processes, and AI security, we will propose the form that fits your organization best.

See you next time!

Primary Sources Referenced

• Claude Fable 5 and Claude Mythos 5 — Anthropic
• Introducing Claude Fable 5 and Claude Mythos 5 — Claude Platform Docs
• Redeploying Claude Fable 5 — Anthropic
• Claude Fable 5 promotional access — Anthropic official support
• Models overview / Pricing — Claude Platform Docs