Wake Word False Accepts Per Hour: How to Test

Illustration

Vendors publish wake word rates at chosen operating points, such as 1 false alarm per 10 hours, and often omit the hours tested. If you observe zero false accepts in T hours, the 95 percent upper bound is about 3 divided by T per hour, so a target of 1 per 10 hours needs about 30 hours.

If you ship an on-device wake word, two numbers decide how the product feels: how often it wakes when nobody said the keyword, and how often it misses when someone did. Vendors state these differently, and a rate with no stated test duration is hard to trust. This guide shows what the public sources say, how to compute how many hours of audio you need, and what to log on your own board. It contains no Duobay measurement and no Duobay price.

The speech and audio you test with matters as much as the math. Use public corpora or audio from people who agreed to be recorded. This page is an engineering write-up for general consumer audio products, and the recording advice assumes visible indicators and consent.

What do vendors actually publish?

The table lists what each source states, as of October 31, 2026. Where a vendor has not published a number, we say so and quote none.

The Picovoice benchmark is run by the vendor and is vendor-reported. For numbers a vendor has not published, the answer is: vendor did not publish, test on your own board.

How many hours of audio prove a false accept target?

False accepts on background audio are rare events, so model them as a Poisson count. If you play T hours of audio and see k false accepts, the exact (Garwood) one-sided 95 percent upper limit on the rate is the 95th percentile of a chi-square distribution with 2(k + 1) degrees of freedom, divided by 2, divided by T. For k = 0 this is -ln(0.05) / T = 2.996 / T, which rounds to the rule of three: about 3 / T. This is a standard statistics result. We derived and computed the numbers below on this page, and no vendor source states them.

upper_limit_per_hour(k, T) = q(k) / T       (95% one-sided, exact Poisson)
hours_needed(k, target)    = q(k) / target   (target in false accepts per hour)

q(0) = 2.996   q(1) = 4.744   q(2) = 6.296   q(3) = 7.754

Variables: T is hours of non-keyword audio played, in hours. k is the number of false accepts you counted, a count. The target is in events per hour. For a target of 1 per N hours, hours needed = q(k) × N.

Check of the first row: 2.996 × 10 = 29.96, so about 30 hours with zero events. Hours are rounded up. The 1-per-24 and 1-per-100 rows are examples of targets you might set. They are not vendor figures. Your target is an assumption you replace.

Allowing more events means more hours, because the upper limit moves up with k. If you plan to run for T hours and allow k events, check that q(k) / T is below your target before you start.

How wide is a single event?

The Espressif page gives a false triggering rate of once in 12 hours and does not state the hours tested. Suppose, only as an illustration, that it rests on exactly one event in 12 hours. The exact two-sided 95 percent Poisson interval for k = 1 runs from about 0.025 to about 5.6 times the point estimate. That gives a rate between roughly 1 per 474 hours and 1 per 2.2 hours. The width is the lesson. A single event cannot separate a very good engine from a poor one. We do not know the real count behind the vendor figure, so do not treat it as proven.

Computed from the chi-square quantiles: lower factor 0.0253 and upper factor 5.572 for k = 1; 12 / 0.0253 ≈ 474 hours and 12 / 5.572 ≈ 2.2 hours.

How many keyword utterances show a false reject target?

False rejects are a count out of trials, so use the binomial. With zero misses in n utterances, the 95 percent upper bound on the miss rate is about 3 / n. A 5 percent miss rate claim needs about 60 utterances with zero misses (3 / 60 = 0.05). That is the whole test for one condition. Every combination of distance, noise level and speaker you want to claim is its own condition, and each needs its own utterances. For example, 10 combinations at 60 utterances each is 600 utterances. The 5 percent figure here is an example target, and your own target is an assumption you replace.

misses_upper_bound(n) ≈ 3 / n        (zero misses, 95% one-sided)
utterances_needed     ≈ 3 / target_miss_rate
total                 = conditions × utterances_per_condition

Comparing two engines

Play the same audio to both engines for the same hours and count each one's false accepts. Under the assumption that both engines have the same true rate, each event is equally likely to belong to either engine, so the split of events follows a fair coin. If one engine has 0 events and the other has 6, the two-sided exact probability is 2 × (1/2)^6 = 0.031. With 0 and 5 it is 2 × (1/2)^5 = 0.0625. This is a conditional exact test, derived here. It counts events per engine on identical audio. If the same audio event triggers both engines, count it for both and treat the pairing carefully. Few events means a weak comparison, so plan the hours before you start.

Which public corpora can serve as background audio?

OpenSLR lists LibriSpeech (resource 12) as a 1000-hour corpus of read English speech under CC BY 4.0 (OpenSLR 12, as of October 31, 2026). The page does not give the hours of the test-clean subset, so confirm that before you use it to plan a duration. The Picovoice benchmark uses LibriSpeech test-clean as background speech and mixes in DEMAND noise at 10 dB SNR (Picovoice benchmark, as of October 31, 2026). Read speech from audiobooks does not look like household or office conversation, so add audio that matches where your product will sit. Use only audio that you have the right to use and that people agreed to be recorded in.

What traps follow from these sources?

These items come from engineering reasoning about the published numbers. None describes an incident at any company.

• Comparing numbers measured at different operating points. The Picovoice benchmark fixes false alarms at 1 per 10 hours and reports misses. The Espressif page reports a false trigger rate next to detection rates. A miss rate at one false-alarm rate and a detection rate at another cannot be placed on one axis.
• Treating a vendor-run benchmark as your result. The Picovoice benchmark is run by the vendor, and the vendor FAQ itself recommends testing on your own accented dataset. Your microphone, enclosure and room are not in either dataset.
• Trusting a rate with no stated test duration. The Espressif page gives a rate of once in 12 hours but not the hours tested. Without the hours, the number could rest on one event or many. The interval section below shows how wide one event is.
• Reading "false accept" the same way in every document. Neither the Picovoice page nor the Espressif page defines how a false alarm or false trigger is counted. If your firmware merges triggers within a window, or counts only triggers that reach the cloud, your count and theirs may differ.
• Assuming a corpus clip matches your product audio. LibriSpeech is read English speech. The Picovoice setup adds DEMAND noise at one SNR. Household and office audio from your own product placement can differ from both.
• Changing sensitivity without rerunning both sides. The vendor FAQ says a higher sensitivity lowers the miss rate and raises the false alarm rate. A setting tuned on one side moves the other side, so rerun false accepts and false rejects together.
• Moving to a bigger model or a new firmware build and keeping the old result. A published or measured rate belongs to one engine build on one board. Keep the build identifier with every result and rerun after changes. This follows from how the numbers are produced, and describes no incident at any company.

What should you test on your own board?

The public sources do not tell you how an engine behaves with your microphone, enclosure and firmware. We have not measured any engine on your hardware, and no result appears in this article. The pass line is set by you per product requirement, and you record the reason for it.

The pass line is set by you per product requirement, and you record the basis for it. Write it down before you run the test.

What should you think through before you choose an engine?

Take this list to your firmware team and to any vendor you talk to.

• What operating point does each number we are comparing use, and are they the same?
• How many hours of audio sit behind each false accept figure, and where is that stated?
• How does the vendor count one false accept, and does our firmware count the same way?
• Which corpora and noise sources were used, what are their licenses, and do they resemble our product audio?
• Which speakers, accents and distances will our product support, and are they in the test set?
• Who sets the pass line for false accepts and for false rejects, and where is the reason written down?
• Is the audio we record for testing consented, labeled and announced, and where is it stored?
• What happens on the device after a false accept: an indicator, a recording, a network call? What does that cost the user?
• For vendors that withhold numbers, have we asked for the report and under what conditions will they share it?

If you want help planning a wake word test on a real prototype, tell us about your device. You can also read how we approach audio and voice hardware development.

Sources

All pages accessed 2026-10-31. The statistics are standard results that we derived and computed on this page.

• Picovoice, wake word benchmark: https://github.com/Picovoice/wake-word-benchmark (accessed 2026-10-31)
• Picovoice, Porcupine FAQ: https://picovoice.ai/docs/faq/porcupine/ (accessed 2026-10-31)
• Espressif, ESP-SR benchmark: https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/benchmark/README.html (accessed 2026-10-31)
• OpenSLR, LibriSpeech (resource 12): https://www.openslr.org/12/ (accessed 2026-10-31)