Hello!

Welcome to the second installment of our coding agent series.

In the previous installment, Part 1, we took a bird's-eye view of the AI coding agent landscape as of December 2025—a field in full bloom.

The State and Future of Coding Agents, Part 1: The Landscape and Fundamentals

Hello! This time, we present a grand tour of the more than 20 flourishing AI coding tools out there—introduced and explained all in one go! AI-assisted coding is by now simply how things are done, and new tools appear at a dizzying pace.

Qualiteg BlogQualiteg コンサルティング

We introduced more than 20 tools, from commercial services to open source, and classified them into four categories: CLI-based, IDE-integrated, AI-native IDE, and autonomous.

We also explained the fundamental architecture: a coding agent is essentially an orchestration system of "LLM + tool layer"—the LLM itself only generates code and makes decisions, while the agent framework handles the actual file writes and command execution.

This time, we focus on the challenges that only become apparent once you really put these tools to work.

To be honest, every one of these tools has moments that make you say "wow"—and then, after a while, moments that make you go "hmm."

Performance suddenly degrades as a session grows long; something you taught it yesterday is forgotten today; a tool that scored high on benchmarks stumbles on your own company's code.

These cannot be dismissed as mere growing pains—they are structural challenges.

In this article, we first compare Claude Code, Codex CLI, and Aider—the leading CLI-based agents—in detail, and then dig into three fundamental problems shared by today's coding agents

——the limits of the context window, memory loss between sessions, and the gap between benchmarks and the real world——

—exploring each with concrete numbers.

Series Overview (Recap)

2-1. A Detailed Comparison of CLI-Based Agents

Part 1 surveyed the overall landscape; from here on, we take a closer look at CLI-based coding agents.

2-1-1. Claude Code

Anthropic's Claude Code is a terminal-based coding agent officially released in February 2025. It leverages high-performance models such as Claude 3.5 Sonnet and Opus 4.1 and offers a context window of up to 200,000 tokens. It has posted a high score of 72.7% on SWE-bench Verified and excels at understanding and modifying complex codebases.

What defines Claude Code is its developer-driven approach. It works collaboratively with the developer through an interactive CLI and always asks for approval before destructive operations. It adopts the CLAUDE.md file system for preserving project context, allowing an entire team to share coding conventions and design policies.

2-1-2. OpenAI Codex CLI

OpenAI's Codex CLI was released as open source in April 2025. It is published under the Apache 2.0 license and accepts community contributions. It leverages the latest models such as GPT-5 and o3, achieving a 69.1% score on SWE-bench Verified.

A standout feature of Codex CLI is its support for multimodal input. It can take screenshots and UI diagrams as input and generate code based on them. It also offers a sandboxed execution option for safety and a Full Auto mode that provides a high degree of autonomy.

2-1-3. Aider

Aider has earned a strong reputation as an open-source pair-programming tool. While terminal-based, its deep Git integration makes tracking and rolling back changes easy. It supports multiple LLM providers (Anthropic, OpenAI, local models, and more), allowing flexible model switching.

Aider's strengths lie in its automatic commits and diff display. It records code changes as Git commits automatically and lets you visually review before-and-after diffs. Its ability to manage changes spanning multiple files consistently also makes it well suited to real-world work.

2-1-4. Comparison Table

2-2. The Limits of the Context Window

From here, as in Part 1, we will use Claude Code as our example while we organize the challenges. Other coding agents face broadly similar issues, so feel free to map these observations onto your tool of choice.

2-2-1. The Nature of the Problem

Every coding agent today operates under one fundamental constraint: the context window. Claude Code has a large 200,000-token window, but for complex tasks even that is not enough.

The figure above illustrates this problem.

The upper chart plots the number of tool calls on the horizontal axis and cumulative token consumption on the vertical axis.

As you can see, consumption is not linear—it accelerates with each additional call, curving ever more steeply upward. Like the person in the illustration on the right, the LLM gradually becomes overwhelmed by the sheer volume of data.

2-2-2. The Concrete Numbers

According to reports in GitHub Issue #2545, Claude Code frequently reaches the 200K-token limit after roughly 50 tool uses.

After only about 50 exchanges, cumulative consumption reaches the neighborhood of 100K tokens. This is the first major wall. Each tool call consumes 1,000 to 10,000 tokens, and it all accumulates.

To maintain conversational coherence, an LLM re-reads the entire past exchange (the history) as context every single time. In other words, each new exchange requires processing "the full past history + the new input + the new output," so consumption snowballs quadratically.

That is, as the session progresses, past conversation history, file contents, and command outputs accumulate, and each request grows larger and larger.

Then, once the remaining context window capacity runs low (the final 20%), the LLM's performance begins to degrade. By around the 80th call, it hits the 200K-token ceiling and can no longer process additional information correctly.

2-2-3. Attempts at Context Compression

Various approaches have been attempted to address this problem. In May 2025, Anthropic announced a context management feature introducing automatic summarization and smarter information selection. For example, a community plugin called claude-mem takes the approach of having AI compress the AI's own work. It compresses tool outputs of 1,000 to 10,000 tokens into semantic observations of roughly 500 tokens, aiming to extend the ~50-tool-use limit to around 1,000 uses (a 20x increase). The trade-off is a 60-to-90-second delay for generating each observation.

These are attempts to shrink token counts by summarizing an ever-growing history—but summarization can drop details that turn out to be critical for implementation.

2-3. Memory Loss Between Sessions

2-3-1. The "Starting from Zero Every Day" Problem

Claude Code's biggest challenge is that it cannot retain memory across sessions.

When a session ends, the next session is

a completely fresh start

as far as Claude Code is concerned, so you have to explain everything from scratch all over again.

This is where it differs from a human—it is like working with an engineer who forgets everything from yesterday, every single day. That said, it picks things up so quickly that teaching it from zero every day is still somehow manageable.

2-3-2. The Limits of CLAUDE.md

To address this, a persistent memory system based on CLAUDE.md files has been introduced. But this is ultimately a manually maintained Markdown mechanism—it does not learn automatically from conversations. And because the contents of CLAUDE.md also consume the context window, there is a dilemma: the more detailed the information you record, the less capacity remains for the actual work.

2-4. The Gap Between Benchmarks and the Real World

2-4-1. SWE-bench: Its Limits

SWE-bench is widely used as the standard benchmark for measuring coding agent performance. But it has serious limitations.

2-4-2. SWE-bench Pro: A Wake-Up Call

On SWE-bench Pro, released by Scale AI in 2025, models that had achieved scores above 70% on the traditional SWE-bench Verified were reported to see their performance fall to around 23%. The causes cited include data contamination (the possibility that benchmark code was included in the training data) and over-adaptation to open-source repositories.

2-4-3. Performance Drops on Private Code

What is especially interesting is that performance drops even further in evaluations using private commercial codebases: Claude Opus 4.1 fell from 22.7% to 17.8%, and GPT-5 from 23.1% to 14.9%. This shows that today's coding agents do not generalize to unseen codebases as well as one might expect.

2-5. Claude Code's Four Structural Problems (Summary)

Based on the analysis so far, here is a summary of the structural problems Claude Code faces.

Rooted in the current architecture, these are structural problems that improvements in model performance alone cannot readily solve.

Fundamentally, the answer would be to increase the amount of memory per session—that is, to push the context size beyond 200K tokens. But that is far from easy even for AI providers, and it tends to require long-term technical breakthroughs. Nor is it purely a technology problem: a larger context size also demands more infrastructure on the provider side, so the cost balance has to be weighed carefully, which makes it a hard problem. Today, Anthropic's Claude offers 200K tokens, while Google's Gemini 3 offers 1 million. A larger context size is an advantage for coding tasks like these, but even 1 million tokens falls far short for large-scale development. In the short term, then, the important theme is what clever techniques are possible under the assumption that context sizes will not grow much.

(Even so, considering that early LLMs—including ChatGPT, which appeared in 2022—had context sizes of only around 30K tokens, the progress has been steady.)

Part 2 Summary

In this article, we compared the major CLI-based agents and, using Claude Code as our example, analyzed the structural challenges facing today's coding agents.

Key Points

• Claude Code, Codex CLI, and Aider each take a distinct approach
• The context window hits its limit after roughly 50 tool uses (O(N²) growth)
• Memory loss between sessions is not fundamentally solved by CLAUDE.md
• A 70% score on SWE-bench Verified drops to the 20% range in the real world
• These are structural problems that model improvements alone cannot solve
• Larger context sizes would help, but are unlikely in the short term

Coming Next — Part 3: Amplifier and the Road Ahead

In the third and final installment, we take a detailed look at how Microsoft's Amplifier tackles these challenges.

• Amplifier's design philosophy: specialized agents, a Knowledge Graph, and a microkernel
• The differences between the old version (piggybacking on Claude Code) and the new version (an independent framework)
• A major shift in the pricing model: from the flat-rate Max plan to pay-as-you-go
• Three approaches to persistent memory: file-based, RAG, and Knowledge Graph
• Open questions ahead: the world after the context limit, and multi-agent collaboration

In particular, we will look closely at the pricing issue: usage that fit within the $100–200/month Max plan under the old version can exceed $500–1,200/month under the new one.

See you in the next installment!