Image and Audio Input Limits for Multimodal Models in Devices

Concept render

Vendors limit images by pixels, bytes and count, and audio by token rate and request size. Claude charges ceil(W/28) x ceil(H/28) visual tokens per image on its high-resolution tier and caps a base64 image at 10 MB. OpenAI rejects images over 30,000 patches. Gemini counts audio at 32 tokens per second with a 20 MB request. As of October 31, 2026.

A camera or microphone sends bytes. The model API counts pixels, patches, tokens and megabytes. This guide puts the official limits side by side, turns them into a budget you can check against a sensor mode, and lists what to measure on your own board. It covers Claude and OpenAI image input and Gemini audio input. It does not cover Gemini image limits or OpenAI audio limits. It contains no Duobay price and no measured result.

What are the published limits?

Each row comes from the vendor's own documentation, read on October 31, 2026. Limits change, so re-read the page before you decide.

How many tokens or patches is one frame?

Claude counts ceil(W / 28) x ceil(H / 28) visual tokens per image (Anthropic vision page, as of October 31, 2026). OpenAI rejects an image that exceeds 30,000 patches after processing, and the patch size depends on the model, so this guide gives no OpenAI patch table (OpenAI images and vision guide, as of October 31, 2026). We did the arithmetic below ourselves. The 2,691 figure for 1920 x 1080 matches the Anthropic page only for the high-resolution tier, which applies to Claude 4.7 and later models. Other models resize that frame first and the vision page lists about 1,560 tokens for it, so a budget built on 2,691 over-counts on those models.

These counts apply to the frame as the API sees it after any resize on the vendor side. For OpenAI, look up the patch size for your model on the OpenAI page and keep ceil(W / p) x ceil(H / p) at or below 30,000.

How do you check a request against the byte limits?

raw_limit_bytes      = Limit_img × 3 / 4           (base64 turns 3 bytes into 4)
frame_fits           = S ≤ raw_limit_bytes
tokens_per_request   = K × ceil(W/28) × ceil(H/28)     (Claude)
patches_per_frame    = ceil(W/p) × ceil(H/p)         (OpenAI, must be ≤ 30,000; p = patch size for your model, from the OpenAI page)
audio_bytes_per_sec  = B / 8
audio_seconds_inline = (20,000,000 − P − image_bytes) / audio_bytes_per_sec   (Gemini)
gemini_audio_tokens  = 32 × seconds

Units: W and H in pixels. S, P and image_bytes in bytes. B in bits per second. K is a count. The 3/4 factor is a general encoding rule, and it is our assumption that the vendor MB is 1,000,000 bytes. If it is 1,048,576, the real limit is slightly higher, so the check stays safe.

Worked example with assumed inputs. Suppose a wearable sends K = 6 frames at 1280 x 720 in one Claude request. That is 6 x 1,196 = 7,176 visual tokens. On the 5 MB Bedrock path, raw_limit_bytes is 5,000,000 x 3 / 4 = 3,750,000 bytes per frame. A frame passes if S is at or below 3,750,000. The values K = 6 and the frame size are assumptions for illustration, and S is yours to measure. For audio, 1 hour is 3,600 x 32 = 115,200 tokens, and the 9.5-hour prompt limit is 9.5 x 115,200 = 1,094,400 tokens.

How much audio fits inline in 20 MB?

The Gemini request limit is 20 MB total including the prompt and all files. The table shows the ceiling when audio is the only file and the prompt is negligible. The bitrates are example choices of ours. They are not recommendations, and the vendor page does not say whether the 20 MB is measured before or after any encoding of the upload.

Source of the 20 MB limit: Google audio page, as of October 31, 2026. The vendor states that audio is downsampled to 16 Kbps on its side.

Inputs, and which are yours to replace

Only the limits come from a source. Every other input is yours. We do not supply a typical frame size, because a made-up number would look like a fact. The vendor does not publish one, so it needs a test on your board.

What are the common input traps?

These follow from the published limits above. They come from engineering reasoning about how the numbers behave. None describes an incident at any company.

• One firmware, two byte limits. Claude allows 10 MB per base64 image on the first-party API and 5 MB on Amazon Bedrock. A frame budget tuned to the larger limit fails on the smaller one. If your backend might move between the two paths, budget for 5 MB.
• Base64 inflates the file. The limits are stated after base64 encoding. Base64 turns every 3 bytes into 4 characters, which is a general encoding rule and not a number from the vendor page. A 5 MB base64 limit holds about 3.75 MB of raw JPEG, and 10 MB holds about 7.5 MB. The vendor page does not say whether MB means 1,000,000 or 1,048,576 bytes, so keep a margin.
• More than 20 images changes the pixel rule. Claude guidance says that above 20 image and document blocks, each image should stay at or under 3,000 px per side. A device that sends a burst of full-resolution frames can cross that line without any change to a single frame.
• Rejection instead of resizing. OpenAI rejects an image that exceeds 30,000 patches after processing. It does not shrink it for you. A sensor upgrade late in development can turn working uploads into errors unless the device checks size before sending. How the pre-processing step counts for your model is a question for the vendor page and a test.
• Animated formats lose frames. Claude uses only the first frame of a GIF. A short animated capture sent as GIF delivers one still. Send separate frames if the model needs motion.
• A microphone array collapses to one channel. Gemini combines multi-channel audio into one channel and downsamples to 16 Kbps. Any beamforming or channel selection must happen on the device before upload. Sending a high sample rate and more channels adds bytes without adding what the model hears.
• Image and audio share one request budget. The Gemini 20 MB request limit counts the prompt and all files together. Audio plus a few frames can pass it while each part looks small.
• Token count grows with pixels. The Claude formula charges per 28 px block, so doubling both dimensions roughly quadruples the visual tokens. The vision page also lists a high-resolution tier at 2,576 px with 4,784 tokens for Claude 4.7 and later models. Read the page for what happens to larger frames before you choose a sensor mode.

What should you measure on your own board?

The vendor pages give limits and formulas. They give no latency, image quality or accuracy results for your sensor, and we have not measured these for you. No result appears in this article. Fix every pass line before you run the test.

What should you ask before you commit?

Take this list to your team, and to any supplier who proposes a sensor or encoding choice.

• Which API path does the firmware call, first-party or a cloud marketplace, and what is the per-image byte limit on that path?
• What is the largest encoded frame our sensor can produce, and does it fit the smaller limit with the base64 factor applied?
• How many frames can one request carry before the pixel rule changes?
• Does our pipeline check size on the device before upload, and what happens to the user when a request is rejected?
• What bitrate does the audio leave the device at, and has a person scored transcripts at that bitrate in noise?
• Where is the multi-channel to single-channel step done, on the device or at the provider?
• Does the request size include the prompt and every other file, and have we added them up?
• Which limits did we read from the vendor pages, on what date, and who rechecks them before launch?
• Gemini image limits and OpenAI audio limits are not covered here. Have we read those pages for the models we plan to use?
• If the device records people, how does the product show that recording is on, and where is that written in the spec?

If you want help choosing the sensor, microphone and encoding for your device, tell us about your device. You can also read about our AI recording wearable development and how our engineering gates work.

Sources

All pages accessed October 31, 2026. Limits and model lists change, so re-read them before you decide.

• Anthropic, Claude vision documentation: https://platform.claude.com/docs/en/build-with-claude/vision (accessed 2026-10-31)
• Google, Gemini API audio understanding: https://ai.google.dev/gemini-api/docs/audio (accessed 2026-10-31)
• OpenAI, images and vision guide: https://developers.openai.com/api/docs/guides/images-vision (accessed 2026-10-31)