Concept renderTinyML means running small machine learning models on microcontrollers with only kilobytes of memory, without a cloud round trip. It fits narrow tasks such as keyword spotting, wake words, person detection and anomaly detection. A common workflow quantizes the model to int8, budgets flash and RAM on the real chip, then tests accuracy and latency on the target board.
TinyML puts a small model next to the sensor. The device decides on its own and sends only the result, which can save radio time and keep raw data local. This guide helps founders and engineering leads judge what fits, pick tools, size memory and power, and prove the model works on the real board. For where TinyML sits among on-device AI options, from microcontrollers to NPUs, see what edge AI is.
Figures come from the vendor, runtime and MLCommons pages in the sources list. None is a Duobay measurement.
What is TinyML, and what hardware runs it?
TinyML is machine learning inference on microcontrollers. Google describes its LiteRT for Microcontrollers runtime as designed to run models on microcontrollers and other devices with only a few kilobytes of memory. It says the core runtime fits in 16 KB on an Arm Cortex-M3. It needs no operating system, no standard C or C++ libraries and no dynamic memory allocation (as of November 2026).
Google announced the rename of TensorFlow Lite to LiteRT on September 4, 2024. Its documentation now says LiteRT for Microcontrollers, while the code repository README is still titled TensorFlow Lite for Microcontrollers, or TFLM (as of November 2026).
Some microcontrollers add instructions for neural network math. Espressif says the ESP32-S3 is a dual-core Xtensa LX7 that runs at up to 240 MHz with 512 KB of internal SRAM, and that its vector instructions accelerate neural network and signal processing work. It also supports external octal SPI flash and PSRAM (as of November 2026). Those numbers apply to the ESP32-S3 only.
Which on-device AI tasks fit on a microcontroller?
Narrow tasks fit: one sensor, a few outputs, frequent decisions. The MLCommons MLPerf Tiny benchmark is a useful map of such tasks. Its page lists version 1.3 of each benchmark. Each one measures the latency of a single inference and checks model quality against a target, and energy measurement is optional (as of November 2026).
| Task | Typical sensor | MLPerf Tiny reference, as of November 2026 | Signs it has outgrown a microcontroller |
|---|---|---|---|
| Keyword spotting | Microphone | Speech Commands dataset, DS-CNN model, 90% top-1 target | Full speech recognition or open vocabulary. |
| Streaming wake word | Microphone, always on | Speech Commands plus MUSAN, 1D DS-CNN, at most 8 false positives and 8 false negatives | Many wake words or many languages. |
| Person detection (visual wake word) | Low-resolution camera | COCO dataset, MobileNet model, 80% top-1 target | Tracking or naming people, or high-resolution frames. |
| Small image classification | Camera | CIFAR-10 dataset, ResNet model, 85% top-1 target | Many classes, large images, or a frame rate the chip cannot hold. |
| Anomaly detection | Vibration, sound or current sensor | ADMOS Toy Car dataset, dense model, 0.85 AUC target | Naming which fault occurred, with few fault examples to train on. |
| Gesture or activity detection | Accelerometer or IMU | Not in the MLPerf Tiny suite | Fine motion tracking or many similar gestures. |
These targets describe benchmark models on public datasets. Use them as a sanity check on whether a task is in range. Speech recognition beyond a few keywords is a different budget. See on-device ASR memory and compute budgets before you plan it on a microcontroller.
Which TinyML toolchain should you use?
Most projects combine a runtime that runs the model, a kernel library that makes each layer fast on a given core, and a vendor tool that converts the model for one chip family.
| Tool | What it is | Targets | Formats and limits, as of November 2026 |
|---|---|---|---|
| LiteRT for Microcontrollers (repository name: TensorFlow Lite for Microcontrollers) | Interpreter for converted models. No operating system or dynamic memory needed. | Requires a 32-bit platform. Google says it is tested extensively on Arm Cortex-M and ported to ESP32. The repository lists Arm Cortex-M (CMSIS-NN), Cadence Xtensa HiFi 3/4/5 and Vision P6, and Qualcomm Hexagon as Tier-1 targets tested in its CI. | A subset of TensorFlow operations. No on-device training. |
| CMSIS-NN | Arm kernel library that speeds up neural network layers. | Cortex-M without SIMD (such as M0), with the DSP extension (such as M4), and with Helium/MVE (such as M55). | Follows the int8 and int16 quantization spec of TensorFlow Lite for Microcontrollers. Float support is experimental. |
| STM32Cube.AI | ST tool that turns a trained model into optimized code. | STM32 microcontrollers. | Takes .tflite, .h5 and .onnx files with float32 or int8 weights. Reports RAM and flash needs, and validates on the desktop or on a connected STM32 board. |
| ESP-DL | Espressif inference framework for its own chips. | Espressif SoCs. | Uses its own .espdl format. Its ESP-PPQ tool quantizes models from ONNX, PyTorch and TensorFlow. A static planner fits layers into the internal RAM size you set. |
Pick the chip first, then the tool its vendor supports best. An unusual layer can lock you to one runtime, because LiteRT for Microcontrollers supports only a subset of TensorFlow operations. In that runtime, your code registers each operation the model uses through an op resolver, so list the model's operations and check each one early.
How much RAM and flash does a TinyML model need?
Split the budget into flash and RAM, and measure both on the board.
Flash: the model and the runtime
In LiteRT for Microcontrollers, the converted model becomes a C byte array stored in read-only program memory. Flash holds that array, the runtime and the kernels for the operations you register. Quantization shrinks the array. Google lists full integer quantization as 4x smaller with 3x or more speedup, for CPUs, Edge TPUs and microcontrollers. It needs a representative dataset to calibrate (as of November 2026). Our int8 quantization accuracy check covers the calibration set and the go/no-go routine.
Google also says creating integer-only models is a common use case for LiteRT for Microcontrollers. For integer-only devices, such as 8-bit microcontrollers, its docs enforce full integer quantization for all operations, including input and output. A model that falls back to float operations and keeps float input and output is not compatible with those devices (as of November 2026). If your target core is integer-only, check that the exported model is fully int8 before you size anything.
RAM: the tensor arena
RAM holds the input, output and intermediate tensors. LiteRT for Microcontrollers asks you to preallocate this memory as a tensor arena. Google says the size depends on the model and may need to be found by experiment. Its getting-started example uses a 2 KB arena for a tiny demo model (as of November 2026). ESP-DL takes a similar approach: its static memory planner places layers within an internal RAM size you specify.
Add the rest of the firmware on top: audio or sensor buffers, the radio stack, the stack for each task, and any logging. Measure the arena on the board with the full firmware running.
How does TinyML change power and battery life?
Energy per inference is roughly active current times inference time. Multiply by how often it runs. A keyword detector on every audio frame costs far more than a gesture model that wakes only on motion.
Three levers matter most. Run the model less often, behind a cheap trigger such as a motion threshold. Make each inference faster with int8 and an optimized kernel library, so the core sleeps sooner. Keep the always-on front end small. Then measure the whole device on the battery rail at the duty cycle real users produce.
How do you validate TinyML accuracy on the device?
A model that scores well in a notebook can still fail on the board, because the sensor, the int8 math and the noise all differ. Plan these checks before the first prototype. The int8 article linked above goes deeper on the first row.
| Check | How to run it | Pass rule you set |
|---|---|---|
| Float vs int8 on your test set | Run both versions on a PC against the same held-out set recorded with your own sensor. | The largest accuracy drop the product can accept, set before the run. |
| Target output matches the PC | Feed stored test inputs to the model on the board and compare outputs with the PC run of the same int8 model. | Outputs match within a tolerance you write down. |
| Latency per inference | Time one inference on the target at the clock speed the product ships with. | Fits inside the sensor window or reply deadline with margin. |
| Peak RAM | Find the smallest tensor arena that runs, then add a margin. Check the stack and buffers too. | Arena plus firmware fits internal RAM with room for growth. |
| Energy per inference | Measure current on the battery rail while the model runs, times inference time. | Fits the battery budget at the real duty cycle. |
| Field behavior | Run the device in real rooms, on real wearers, for hours. Count misses and false triggers. | Rates per hour or per event that match what you will promise. |
Vendor tools help with the first steps. ST says STM32Cube.AI can validate the generated code against the original model using a dataset or random values, on the desktop or on an STM32 board connected to the computer, and it reports detailed RAM and flash needs (as of November 2026). With LiteRT for Microcontrollers you write that harness yourself: stored inputs in, outputs logged, compared with the PC.
Record test data on your own hardware. A public speech dataset was not captured through your microphone port or your enclosure. For anything that runs all the time, count errors per hour of real background audio. Our guide to wake word false accept and false reject testing shows how many hours a target needs.
When is a microcontroller the wrong place for the model?
TinyML stops fitting when the model outgrows internal RAM, when one inference takes longer than the sensor window, or when the task needs open language or large images. Then you have three paths: a microcontroller with a neural processing unit, an application processor with more memory, or sending data to a server. Each changes cost, power and privacy. Our microcontroller vs Linux SoC comparison works through that choice for a wearable. Check which of your model's operations an NPU supports before you count on its speed. See what an NPU is in edge devices for that check.
What should you ask your hardware team?
Bring these to the first design review, and record who answered each one.
- Which task does the model do, and what is the accuracy or error rate we will accept on our own data?
- Which runtime and which chip are we assuming, and are all model operations supported there?
- How large is the model array in flash, and how large is the tensor arena in RAM, measured on the board?
- Is the model fully int8, including input and output, and what representative data calibrated it?
- How long does one inference take, and how often does it run in normal use?
- Where does the test audio, motion or image data come from, and was it recorded on our hardware?
- How will we update the model after launch, and who re-runs the tests when it changes?
The model, runtime and update path all live in the firmware. For how that work is organized and what source you receive, see firmware development.
Is TinyML the same as on-device AI?
TinyML is the microcontroller end of on-device AI, which also covers larger models on application processors. Google designed LiteRT for Microcontrollers for devices with only a few kilobytes of memory (LiteRT for Microcontrollers, as of November 2026).
Can TinyML run on any microcontroller?
Not with every runtime. Google says LiteRT for Microcontrollers requires a 32-bit platform (source, as of November 2026). Vendor tools such as STM32Cube.AI and ESP-DL target their own chip families.
Can a TinyML model keep learning on the device?
Google lists on-device training as not supported in LiteRT for Microcontrollers, as of November 2026. In the documented workflow the model is a C array built into the firmware, so changing the model usually means a firmware update.
Do I need an NPU to run TinyML?
Not always. Time your model on a CPU-only microcontroller first. If it misses your latency or energy target, an NPU is one option. See what an NPU is in small devices.
How can Duobay help with a TinyML feature?
Duobay develops custom devices with a Shenzhen engineering team that covers electronics, firmware, enclosure, app and test, through samples and small batches. Firmware source is delivered to you. We do not publish prices, and the first call is free. Tell us about your device.
If you want a TinyML feature checked against a real chip and a real power budget, tell us about your device.
Sources. All pages accessed November 6, 2026. Tools and benchmarks are revised often, so confirm current values before you decide.
- Google, LiteRT for Microcontrollers overview (link)
- Google, LiteRT for Microcontrollers get started (link)
- Google, LiteRT post-training quantization (link)
- Google for Developers Blog, TensorFlow Lite is now LiteRT (September 4, 2024) (link)
- TensorFlow, tflite-micro repository README (link)
- Arm, CMSIS-NN documentation (link)
- STMicroelectronics, STM32Cube.AI (link)
- Espressif, ESP-DL introduction (link)
- Espressif, ESP32-S3 product page (link)
- MLCommons, MLPerf Inference: Tiny benchmark (link)
Planning a device like this? Tell us what it needs to do. A Duobay engineer replies within 1 business day. Send a project brief →