Concept renderSkip the LLM when your product needs a short, closed list of actions. Local recognizers handle fixed command sets offline, a speech-to-intent engine returns intent and slots directly, and OpenAI's own guide says not to default to an LLM. Use a model only for requests no list can cover.
A language model is easy to add to a device and hard to take out later. It brings a per-token bill on every use, a network dependency and variable replies. Many products only need a handful of actions. This guide lists the published facts that bear on that choice, gives a cost formula you can fill in, and shows what to measure on your own board. It contains no Duobay price and no measured result from us.
Which approach fits which job?
This table compares three paths using only what each vendor publishes. Each row was read on October 31, 2026. Where a vendor did not publish a number, the cell says so.
| Approach | What the vendor page says | Cost per use | Limits and unknowns | Source |
|---|---|---|---|---|
| Fixed command set (example: Espressif MultiNet on ESP32-S3) | Recognizes a closed list. The vendor page says it is designed to recognize multiple speech command words offline on ESP32-S3, supports up to 200 commands including customized ones, and takes commands from a text file or the esp_mn_commands_add API. | No cloud model call per command, so no per-call model fee. | The list is closed. The page states a 200-command ceiling. Flash and RAM use on your chip: vendor figures not read for this article, measure on your board. | Espressif documentation, as of October 31, 2026 |
| Speech-to-intent engine (example: Picovoice Rhino) | Returns intent and slots directly from speech within a defined context. The vendor page says it directly infers intent from spoken commands within a given context of interest, in real time, and lists Arm Cortex-M targets (STM32, Arduino) among its platforms. | The page does not state a per-call cost. Check the license terms with the vendor. | The context is defined in advance. The page does not use the words "on-device", so confirm with the vendor where inference runs before you design around it. Latency and memory: vendor did not publish figures on that page, measure on your board. | Picovoice documentation, as of October 31, 2026 |
| Cloud LLM (example: OpenAI realtime audio) | Open-ended conversation, no fixed list. | Billed per token on every interaction, from the table in the next section. | Needs a network path on every use. Latency, accuracy and answer variation: not measured for this article, measure on your product. | OpenAI pricing page, as of October 31, 2026 |
A rule of thumb from the criteria
This is our engineering reasoning from the table. No vendor stated it. A local path is a candidate when the list of things a user can say is closed, the count sits under the engine's stated ceiling, and the product must work offline or answer fast. A model is a candidate when requests are open-ended and no list can cover them. Many products need both: a local list for the common actions and a model for the long tail. Decide the split from your own logs.
What does a cloud model cost per interaction?
These are the official list prices per million tokens from the OpenAI pricing page, read on October 31, 2026. The right-hand column adds one million input tokens and one million output tokens, a unit chosen only to make the arithmetic visible.
| Model | Input, per million tokens | Output, per million tokens | Input plus output, one million each | Source |
|---|---|---|---|---|
| gpt-realtime-2.1 | Audio: $32.00 | Audio: $64.00 | $96.00 | OpenAI pricing page, as of October 31, 2026 |
| gpt-realtime-2.1-mini | Audio: $10.00 | Audio: $20.00 | $30.00 | OpenAI pricing page, as of October 31, 2026 |
| gpt-5 | Text: $1.25 | Text: $10.00 | $11.25 | OpenAI pricing page, as of October 31, 2026 |
The pricing page states no page date, so the date above is when we read it. Re-read the page before you decide.
The formula
cost_per_interaction = (a × p_in + b × p_out) / 1,000,000
cloud_per_year = D × K × cost_per_interaction
local_command_path = no per-call model fee
(you pay in flash, RAM, firmware effort and the command ceiling)Units: a and b are tokens, p_in and p_out are dollars per million tokens, K is interactions per device per year, D is devices, and the results are in dollars.
Worked arithmetic with the published prices. At one million input tokens and one million output tokens, gpt-realtime-2.1 costs 32.00 + 64.00 = $96.00, and gpt-realtime-2.1-mini costs 10.00 + 20.00 = $30.00. The first is 96.00 / 30.00 = 3.2 times the second. The same volume on gpt-5 text costs 1.25 + 10.00 = $11.25, so the realtime audio model is 96.00 / 11.25, about 8.5 times the text model. To get your cost per interaction, scale by a / 1,000,000 and b / 1,000,000 with your own token counts.
Inputs, and which are yours to replace
Only the prices come from a source. Every other input is an assumption for you to replace. We supply no example values for them, because a made-up number would look like a fact.
| Variable | Meaning | Status | What to do |
|---|---|---|---|
| a, b | Input and output tokens per interaction | Your assumption | The pricing page does not give an audio-seconds-to-tokens conversion. Take it from the provider realtime documentation and then confirm by logging real sessions on a prototype. |
| p_in, p_out | Price per million input and output tokens, in dollars | Published price | Take from the table above. Check the date and the model name. |
| K | Interactions per device per year | Your assumption | Replace after you log real usage. Count every wake, every retry and every misheard command. |
| D | Devices in the field | Your assumption | Replace with your own plan. |
| L | Number of distinct commands your product needs | Your assumption | Count the real list. Compare it with the 200-command figure on the Espressif page if you consider that engine. |
Which LLM features should you cut?
These follow from the sources above. They are engineering reasoning. None describes a failure at any company.
- A chat mode on a device whose real job is five or ten actions.
- A model-written confirmation for every action. Store the phrase.
- A cloud call to decide which of your fixed commands the user meant, when the command list is closed and a local recognizer already covers it.
- A refusal message generated on every out-of-scope request. Store it.
- A voice path that bills audio tokens for wake-up or silence you could have handled locally.
What are the common traps?
Each trap is derived from the published facts above. None describes an incident at any company, and we found no public, attributable postmortem to cite.
- Sending a bounded task to a billed model. If your product has a list of commands, each use of a cloud model adds a per-token charge that a local recognizer does not. The Espressif page documents offline recognition of up to 200 commands on ESP32-S3, so a short, closed list is a candidate for the local path.
- Letting the model write fixed replies. OpenAI's latency guide says that action confirmations, refusal messages and requests for standard input are all great candidates to be hard-coded. If your device says "Timer set" or "Sorry, I cannot do that", a stored phrase costs no tokens on a path you may call thousands of times.
- Defaulting to a model because it is available. The same guide has a section titled "Don't default to an LLM." It is the vendor of one of the models above telling builders to check whether a simpler method fits the task first.
- Hiding a hard ceiling. A command engine with a stated maximum of 200 commands will not grow past it with a firmware update. If your roadmap adds commands every quarter, plan for that ceiling, or plan for a different path.
- Pricing voice like text. The realtime audio rates above are $32.00 and $64.00 per million tokens for gpt-realtime-2.1, against $1.25 and $10.00 for gpt-5 text. Do not reuse a text estimate for a voice feature.
- Trusting a feature name. The Picovoice page says Rhino infers intent directly from speech and lists Arm Cortex-M targets. It does not use the words "on-device". If your privacy claim or offline promise depends on where inference runs, get it confirmed in writing.
What should you measure on your own board?
The sources do not give latency, accuracy, memory or power for your chip and your phrases. We have not measured these for you, and no result appears in this article. Measure them yourself before you commit.
| What to measure | How | How to set the pass line |
|---|---|---|
| End-to-end latency, fixed command set vs cloud LLM | Put a GPIO toggle or a logged timestamp at the end of the spoken phrase and a second one at the moment the action starts. Use the same recorded phrase set, played from a speaker at a fixed distance, on both paths. Report the median and the 95th percentile over at least a few hundred runs, and run the cloud path on your worst realistic network too. | Set the maximum tolerable delay from the product use case before you run the test, and write the reason next to it. |
| Command recognition accuracy | Record a phrase set from speakers who match your market, in the noise conditions of the real use (quiet room, kitchen, street). Score hits, misses and wrong-command errors separately, per condition. | Pick limits per error type. A wrong command that triggers an action usually deserves a stricter limit than a missed command. State the basis. |
| False wake and false trigger rate | Play several hours of background audio (television, conversation, music) with no intended command and count unintended activations. | Derive the limit from how often a user can tolerate an unintended action. Write it before the run. |
| Flash and RAM footprint on your chip | Build the firmware with and without the recognition engine and read the linker map. Record peak RAM during recognition, with your other tasks running. | Compare with the free memory your own design budget leaves. The vendor pages read for this article did not give figures for your chip. |
| Power draw while listening | Put a current meter on the battery rail. Record always-listening idle for the local engine and for the cloud path (microphone, radio and any wake detector). | Convert to battery hours with your real usage profile and compare with your product promise. |
| Cost per interaction from real logs | Log token counts per interaction from prototype sessions, including retries, using the provider tokenizer. Compute the average and the 95th percentile. | These replace a and b in the formula. Compare the result with your margin or subscription plan. |
| Coverage of real requests by a closed list | Collect what test users actually say to a prototype with an open microphone. Count how many requests would map onto your candidate command list. | Set the minimum coverage you accept before you give up open conversation. If the rest is long-tailed, that argues for a model. |
Fix every pass line before you run the test. A threshold chosen after you see the result tends to land just under the result.
What should you ask before you commit?
Take this list to your team and to any supplier who proposes a model for your device.
- How many distinct things must the user be able to say, and is that list closed?
- Which requests in our prototype logs fall outside any list we could write, and how often?
- What does one interaction cost at the published audio price, using token counts from our own logs, with no guessing?
- Does the product still work with no network? What do we promise the user then?
- What is the longest delay a user will accept for this action, and which path was measured against it?
- Which fixed phrases can be stored on the device so the model never writes them?
- Does our command list fit under the ceiling of the engine we picked, with room for the next two releases?
- Where does the chosen engine actually run, and who confirmed that in writing?
- How much flash and RAM does the local engine take on our chip, and is that budget reserved in the design?
- Which assumptions in our model are measured on our own board, and which are guesses?
Questions about LLMs and fixed command sets
Is a fixed command set always better than an LLM?
No. This article does not claim a winner. A closed list suits a short, bounded set of actions. Open-ended conversation needs a model. The measurement table above shows how to decide with your own phrases and your own board.
How many commands can an on-device recognizer handle?
It depends on the engine and the chip. The Espressif MultiNet documentation states up to 200 speech commands, including customized commands, on ESP32-S3 (Espressif documentation, as of October 31, 2026). Other engines list other limits, so read the vendor page for the one you choose.
Can I hard-code the replies my device speaks?
Yes. OpenAI's own latency guide lists action confirmations, refusal messages and requests for standard input as good candidates to hard-code (OpenAI latency guide, as of October 31, 2026).
How do I estimate the cloud cost of a voice feature?
Multiply your measured input and output tokens per interaction by the published per-million-token prices, then multiply by interactions per device per year and by fleet size. The formula and the price table above show the steps. The pricing page (OpenAI pricing page) has no page date, so we record the date we read it: October 31, 2026.
Can Duobay tell me which approach fits my device?
We do not publish prices. A Gate 0 Feasibility study states what each feature adds in bill of materials and firmware effort, so you can decide with real inputs. Tell us about your device and we will explain how it works.
If you want to know what each feature adds to the bill of materials and firmware effort for your device, tell us about your device. You can also read how our firmware work is organized.
Sources
All pages accessed October 31, 2026. Vendor pages change, so re-read them before you decide.
- Espressif, ESP-SR speech command recognition (MultiNet): https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/speech_command_recognition/README.html (accessed 2026-10-31)
- Picovoice, Rhino speech-to-intent documentation: https://picovoice.ai/docs/rhino/ (accessed 2026-10-31)
- OpenAI, latency optimization guide: https://developers.openai.com/api/docs/guides/latency-optimization (accessed 2026-10-31)
- OpenAI, API pricing: https://developers.openai.com/api/docs/pricing (accessed 2026-10-31)
Planning an AI recording wearable? Tell us what it needs to do. A Duobay engineer replies within 1 business day. Send a project brief →