The GPU-Rich and the Reality in Japan

The global GPU shortage is becoming increasingly severe, and securing high-performance graphics processing units (GPUs) in particular has become difficult. This shortage is having a major impact on many technology industries, beginning with AI research and development, and is giving rise to a new form of competition among companies and research institutions.

The Current State of the GPU Shortage

Procuring GPUs has become so difficult that voices within the industry can be heard saying things like, "GPUs are in such short supply that the fewer people using our product, the better," and "With GPUs this scarce, we would honestly be glad if usage of our product went down."

Elon Musk has described GPUs as being "considerably harder to get than drugs."

Big Tech companies and mega-startups in the United States are securing GPUs by the tens of thousands; as one example, securing 10,000 GPUs is said to require an investment of roughly 60 billion yen. These companies own such large quantities of high-performance GPUs, such as NVIDIA's A100 and H100, that they have come to be called "GPU-rich."

The Impact of Being GPU-Rich

This GPU-rich environment is accelerating the AI R&D race within the United States. Top AI researchers in the Bay Area boast about their access to GPUs, and this has begun to significantly influence their choice of workplace. Companies such as Meta are using GPU resources as a recruiting tactic, leveraging their deep pockets to secure large volumes of high-performance GPUs in a strategy aimed squarely at winning the race.

The Situation in Japan

In Japan, on the other hand, even ABCI (AI Bridging Cloud Infrastructure) at the National Institute of Advanced Industrial Science and Technology (AIST), which could be called a national initiative, shows no sign of holding the new H100 model and has only been able to secure older-generation GPUs. This situation shows that Japan is at a disadvantage in international competition, and no matter how hard it tries, it may only be able to build small-scale Japanese-language LLMs.

(There is also the underlying issue that Japanese-language resources are far scarcer than English ones.)

How Japan Should Compete

How Japan should compete in this GPU-rich environment is an important question. We believe that Japanese companies and research institutions will discover highly optimized algorithms, efficient data processing, and creative problem-solving strategies within their limited resources. Furthermore, by using LLM platform companies like ours as a hub, mutual partnerships can form and strengths can be pooled in the development of new technologies. To that end, we intend to actively support networking and share case studies. In other words, where the United States competes with raw power, Japan competes by bringing together skill upon skill.

Still a Harsh Environment for LLM Startups

We must not forget the ventures and startups researching LLMs, which are key players in providing that "skill."

Their situation is even more serious; one might call them "GPU-poor." For example, an NVIDIA A100 (80GB) costs 3 million yen per unit, and an H100 (80GB) costs more than 6 million yen. When a university spin-off startup tries to raise equity, its seed-stage post-money valuation might be a few hundred million yen, and the actual amount raised is at most a few tens of millions of yen. At the funding levels of Japan's current startup ecosystem, buying just a few high-performance GPUs would exhaust the entire round.

Because this industry is fundamentally built around GPUs, its business model and cost structure are decisively different from the traditional SaaS model of starting small and scaling up, yet we hear that explaining this and getting it properly understood is quite difficult. Moreover, even if a startup could raise a few hundred million yen, that amount is nowhere near enough to train a truly "large-scale" LLM. GPU cloud environments are also expensive, and since there is no guarantee that a training run will succeed in the first place, it is common to spend millions of yen on training and end up with nothing to show for it. It is a genuinely tough situation. If things continue as they are, we believe that apart from startups lucky enough to find a sponsor or those acquired by large companies, most will be weeded out before they even get a turn at bat. One could call that competition, but at the very least, they need a chance to step up to the plate (that is, to use GPU resources freely). Generous relief measures such as AWS's support programs have begun to appear, but for more challengers to get their turn at bat, an environment is needed where abundant GPU resources can be accessed casually, and we hope for a national policy of "free GPU usage." This is by no means someone else's problem: although we do not use resources on the scale required for training, we too are struggling to secure the GPU resources needed for our inference environment, and we share the same sentiment.

All you all need is GPU! (^_-)