Hello!

OpenAI's vision-capable models (that is, LLMs that accept image input) use two different methods for converting images into tokens.

The latest GPT-5 and GPT-4.1 families introduced a patch-based method that differs from the traditional tile-based approach. This change significantly improves the efficiency of image processing and allows for finer-grained control.

The Two Calculation Methods

OpenAI currently operates two calculation methods in parallel: the patch-based method and the tile-based method.

The patch-based method is used by the newer generation of models such as GPT-4.1-mini, GPT-4.1-nano, GPT-5-mini, GPT-5-nano, and o4-mini. This method divides an image into very small patches of 32×32 pixels. Considering that the traditional tile method used 512×512 pixels, each patch is roughly 1/256th the size, enabling more precise image understanding.

Meanwhile, flagship models such as GPT-4o, GPT-4.1, GPT-5, o1, and o3 continue to use the tile-based method. This approach divides images into 512×512-pixel tiles and resizes them appropriately before processing for efficient handling.

Token Calculation Tables by Model

Patch-Based Method (32×32 pixels)

Tile-Based Method (512×512 pixels)

*In GPT Image 1's high-fidelity mode, 4,160 additional tokens are added for square images and 6,240 for portrait or landscape images.

The Patch-Based Method in Detail

The patch-based method first calculates how many 32×32-pixel patches are needed to cover the image. For example, a 1024×1024 image requires 1,024 patches of 32×32 pixels.

However, there is an upper limit of 1,536 patches. Larger images that exceed this limit are automatically scaled down. The scaling factor is computed so that the entire image fits within 1,536 patches, and the image is shrunk while preserving its aspect ratio.

The final token count is obtained by multiplying the number of patches by a model-specific multiplier: 1.62 for GPT-5-mini and GPT-4.1-mini, 2.46 for GPT-5-nano and GPT-4.1-nano, and 1.72 for o4-mini. These different multipliers reflect differences in each model's internal architecture.

Example Implementation of the Patch-Based Calculation

import math

def calculate_patch_tokens(width, height, model="gpt-4.1-mini"):
    """Token calculation for the patch-based method"""
    multipliers = {
        "gpt-5-mini": 1.62,
        "gpt-5-nano": 2.46,
        "gpt-4.1-mini": 1.62,
        "gpt-4.1-nano": 2.46,
        "o4-mini": 1.72
    }
    
    # Calculate the number of patches needed (in 32x32-pixel units)
    raw_patches = math.ceil(width / 32) * math.ceil(height / 32)
    
    # Resize if the image exceeds 1536 patches
    if raw_patches > 1536:
        # Compute the scaling factor
        r = math.sqrt(32 * 32 * 1536 / (width * height))
        
        # Adjust to align with patch boundaries
        width_scale = math.floor(width * r / 32) / (width * r / 32)
        height_scale = math.floor(height * r / 32) / (height * r / 32)
        r = r * min(width_scale, height_scale)
        
        resized_width = width * r
        resized_height = height * r
        
        # Number of patches after resizing
        final_patches = math.ceil(resized_width / 32) * math.ceil(resized_height / 32)
    else:
        final_patches = raw_patches
    
    # Cap at 1536 patches and apply the model-specific multiplier
    final_patches = min(final_patches, 1536)
    return int(final_patches * multipliers.get(model, 1.62))

The Tile-Based Method in Detail

The tile-based method performs a more complex, multi-stage process.

In high-resolution mode (detail="high"), the image is first resized to fit within a 2048×2048-pixel square. It is then resized again so that its shorter side becomes 768 pixels (512 pixels for GPT Image 1). This two-stage resizing allows images of various aspect ratios to be processed efficiently.

The resized image is divided into 512×512-pixel tiles, and the number of required tiles is calculated. The final token count is the sum of the base tokens plus the number of tiles multiplied by the tokens per tile.

If you choose low-resolution mode (detail="low"), no resizing or tiling takes place; a fixed token count defined per model is used instead—85 tokens for GPT-4o, 2,833 tokens for GPT-4o-mini, and so on.

Example Implementation of the Tile-Based Calculation

import math

def calculate_tile_tokens(width, height, detail="high", model="gpt-4o"):
    """Token calculation for the tile-based method"""
    model_config = {
        "gpt-5": {"base": 70, "tile": 140, "min_side": 768},
        "gpt-4o": {"base": 85, "tile": 170, "min_side": 768},
        "gpt-4o-mini": {"base": 2833, "tile": 5667, "min_side": 768},
        "o1": {"base": 75, "tile": 150, "min_side": 768},
        "gpt-image-1": {"base": 65, "tile": 129, "min_side": 512}
    }
    
    config = model_config.get(model, model_config["gpt-4o"])
    
    # Low-resolution mode uses a fixed value
    if detail == "low":
        return config["base"]
    
    # Step 1: fit within 2048x2048
    if max(width, height) > 2048:
        scale = 2048 / max(width, height)
        width = int(width * scale)
        height = int(height * scale)
    
    # Step 2: adjust the shorter side to the target size
    scale = config["min_side"] / min(width, height)
    width = int(width * scale)
    height = int(height * scale)
    
    # Step 3: count the 512x512 tiles
    tiles = math.ceil(width / 512) * math.ceil(height / 512)
    
    # Total tokens = base + (number of tiles x tokens per tile)
    return config["base"] + (tiles * config["tile"])

Reducing Costs Through Image Size Optimization

Sending huge images as-is causes token consumption to spike, driving up costs. A particular problem is wasted tokens due to padding. For example, a 513×513-pixel image is processed as four tiles (2×2) under the tile-based method, even though most of each tile is actually blank—an inefficient outcome.

You can minimize padding by adjusting image dimensions to be close to multiples of 32 for the patch-based method, or multiples of 512 for the tile-based method. It is also important to choose an appropriate resolution based on the image content. If character recognition is not required, you can often downsize the image aggressively without any problems.

Code for Optimizing to an Efficient Size

from PIL import Image

def optimize_image_size(image_path, model_type="tile", max_dimension=2048):
    """
    Optimize an image to an efficient size that minimizes padding
    
    Parameters:
    - image_path: path to the image file
    - model_type: "patch" (multiples of 32) or "tile" (multiples of 512)
    - max_dimension: maximum size limit
    """
    img = Image.open(image_path)
    width, height = img.size
    
    # Base unit depending on the model type
    unit = 32 if model_type == "patch" else 512
    
    # If the current size exceeds the maximum, shrink it first
    if max(width, height) > max_dimension:
        scale = max_dimension / max(width, height)
        width = int(width * scale)
        height = int(height * scale)
    
    # Compute the optimal size (minimizing padding)
    def find_optimal_size(size, unit):
        # Find the multiple of the unit closest to the current size
        lower = (size // unit) * unit
        upper = lower + unit
        
        # Choose whichever requires less padding
        if size - lower < upper - size:
            return lower if lower > 0 else upper
        else:
            return upper
    
    optimal_width = find_optimal_size(width, unit)
    optimal_height = find_optimal_size(height, unit)
    
    # Perform the resize
    optimized_img = img.resize((optimal_width, optimal_height), Image.LANCZOS)
    
    # Calculate and print the token count
    if model_type == "patch":
        patches = (optimal_width // 32) * (optimal_height // 32)
        estimated_tokens = int(patches * 1.62)  # for GPT-4.1-mini
        print(f"Optimized: {optimal_width}x{optimal_height} ({patches} patches, ~{estimated_tokens} tokens)")
    else:
        tiles = (optimal_width // 512) * (optimal_height // 512)
        estimated_tokens = 85 + tiles * 170  # for GPT-4o
        print(f"Optimized: {optimal_width}x{optimal_height} ({tiles} tiles, ~{estimated_tokens} tokens)")
    
    return optimized_img

# Usage example
# optimized = optimize_image_size("large_photo.jpg", model_type="tile")
# optimized.save("optimized_photo.jpg")

Calculation Examples for Various Sizes

*Values after resizing due to the 1,536-patch limit

Optimization Tips

It is important to choose the right model for the job: GPT-4.1 for OCR and text recognition, GPT-4o-mini for fast analysis, GPT-5 for detailed image understanding, and GPT-4.1-nano when cost matters most.

Optimizing image size is especially important. For example, when processing a 3000×4000-pixel photo, sending it as-is consumes roughly 2,500 tokens under the tile-based method, but resizing it to 2048×1536 brings that down to 1,445 tokens. Going further to 1536×1024 reduces it to 935 tokens. For most use cases, this level of resizing has almost no impact on quality.

A staged approach is also effective: process images in low-resolution mode first, then reprocess at high resolution only when necessary. In many cases this can cut costs substantially.

Guidelines for Cost-Efficient Image Sizes

When using the tile-based method, the following sizes are efficient: 512×512 (1 tile), 1024×512 (2 tiles), 1024×1024 (4 tiles), 1536×1024 (6 tiles), and so on. Aligning dimensions to multiples of 512 avoids wasteful padding.

For the patch-based method, multiples of 32 are efficient. Sizes such as 960×960, 1280×960, and 1920×1080 fit patch boundaries exactly, so nothing is wasted. 1920×1080 in particular is a common image size, which has the added benefit of being processable as-is.

Limitations and Caveats

The API has the following limits: a maximum file size of 50MB, up to 500 images per request, and supported formats of PNG, JPEG, WEBP, and non-animated GIF.

The models also have weak spots. Accuracy may degrade for interpreting medical images, recognizing non-Latin text, reading text that is too small, understanding rotated images, and precise spatial reasoning.

Summary

To use OpenAI's Vision API efficiently, it is essential to understand each model's characteristics and optimize images to appropriate sizes. Adjusting dimensions to multiples of 32 for the patch-based method or multiples of 512 for the tile-based method minimizes waste from padding.

Rather than sending huge images as-is, resizing them to an appropriate size for the task can significantly reduce costs. Keep in mind that in many cases, images at 50-70% of the original size still retain more than enough quality.

See you next time!

Related Articles

- Calculating image token consumption for Google's Gemini series

A Guide to Multimodal Token Calculation (Image, Video, and Audio Tokens) in Gemini 2.5 Pro/Flash

Hello! When using Gemini 2.5 Pro and Gemini 2.5 Flash, accurately understanding token counts is essential for pricing calculations and context window management. This article explains in detail how tokens are calculated for multimodal content such as images, video, and audio. Basic concept: what is a token? In the Gemini 2.5 series, one token corresponds to roughly 4 characters, and 100 tokens correspond to roughly 60-80 English words. All input and output is processed in token units, and billing is based on token counts as well. Gemini 2.5 series models and pricing. Available models: Gemini 2.5 Pro, the flagship model with advanced reasoning; Gemini 2.5 Flash, a fast, cost-efficient model; Gemini 2.5 Flash Image, a dedicated image-generation model. Context window: both models offer a large context window of 1,000,000 tokens.

Qualiteg BlogQualiteg プロダクト開発部

- Calculating image token consumption for Anthropic's Claude series

Claude 4.5 API: A Guide to Calculating and Optimizing Image Input Tokens

Hello! This time, we take a detailed look at how image token counts are calculated when using Claude 4.5 Sonnet/Haiku and Claude 4.1 Opus via the API. How image token counts are calculated: images sent to the Claude 4.5 API are counted as tokens just like text, forming the basis of pricing. If an image is within the API's size limits and requires no resizing, you can estimate its token count with this simple formula. Basic formula: tokens = (width px × height px) ÷ 750. Using this formula, you can predict costs before uploading and optimize images as needed. For example, a 1000×1000-pixel image consumes about 1,334 tokens, so under Claude 4.5's pricing you can calculate the cost per image in advance. A 1092×1092-pixel image (1.19 megapixels) comes to about 1,590 tokens, which you can use as a baseline to estimate batch-processing costs as well. Image size limits and optimization: the Claude 4.5 API has several important

Qualiteg BlogQualiteg プロダクト開発部