Introducing Llama 3.1, the Latest Large Language Model Series Announced by Meta

On July 23, 2024, Meta announced its latest large language models, the Llama 3.1 series. In this article, we introduce the features and performance of the Llama 3.1 series, along with a hands-on look at using it.

We have also covered this topic in the video below, so feel free to watch it as well.

Key Features of the Llama 3.1 Series

The Llama 3.1 series is available in three model sizes: 8B, 70B, and 405B. Its key features are as follows:

• The first open LLM to rival top AI models in general knowledge, steerability, math, tool use, and multilingual translation
• A context window of 128K tokens
• A multilingual model supporting eight languages (Japanese, however, is not among them)
• Trained on more than 15 trillion tokens

Features by Model Size

• 8B model: well suited to mobile devices and smaller systems, delivering strong performance even in resource-constrained environments
• 70B model: offers an excellent balance, approaching the performance of the 405B model on many tasks while requiring far less compute to operate
• 405B model: the best choice when top-tier performance is required or when handling complex tasks

Performance Comparison of the Llama 3.1 405B Model

Meta evaluated these models using more than 150 benchmark datasets. The specific comparison results for the 405B model are as follows:

1. MMLU (general language understanding): 88.6, beating GPT-4's 85.4 by 3.2 points
2. HumanEval (coding ability): 89.0, beating GPT-4's 86.6 by 2.4 points
3. GSM8K (mathematical ability): 96.8, beating GPT-4 Omni's 96.1 by 0.7 points
4. ARC Challenge (reasoning ability): 96.9, beating GPT-4's 96.4 by 0.5 points
5. ZeroSCROLLS/QuALITY (long-context processing): 95.2, beating GPT-4 Omni's and Claude 3.5 Sonnet's 90.5 by 4.7 points
6. Multilingual MGSM (multilingual capability): 91.6, beating GPT-4's 85.9 by 5.7 points

These results show that Llama 3.1 405B delivers state-of-the-art performance across many areas. The 70B and 8B models likewise perform impressively for their respective sizes.

Let's Actually Chat with Llama 3.1

You can chat with Llama 3.1 for yourself on chatstream.net, which we operate.

https://chatstream.net/?model_id=meta_llama_3_1_8b_instruct&ws_name=chat_app_en

Wrap-Up

Even with the 8B model, Llama 3.1 showed a general tendency to answer user questions from multiple angles. Compared with the previous 8B model, it came across as smarter and more attentive.

Going forward, we expect a steady stream of models built on Llama 3.1 with continued pre-training in Japanese. As AI continues to evolve, we look forward to the arrival of these new models.