What Is a KWS Model? Offline Voice Control with a Small ONNX Model

Voice control does not always need speech-to-text, a cloud API, or a large language model.

If a device only needs to react to a short, known list of phrases—“turn on,” “turn off,” “next,” or “take a photo”—it may only need to detect whether one of those keywords was spoken. That is the job of keyword spotting, usually shortened to KWS.

KWS is best understood as a focused audio classifier. It listens to short audio windows and estimates whether a trained keyword is present. It does not transcribe everything a person says or understand open-ended requests.

Wake words are the common use case, but not the only one

KWS is widely used for wake words. A small detector can stay active and listen for a phrase such as “Hey device.” When it detects the wake word, the product can start a larger speech recognizer or open a voice-assistant session.

But a wake word is only one use for KWS. Some small devices have a limited command set and do not need a follow-up conversation at all. A lamp controller might recognize “lights on” and “lights off.” A presentation remote might listen for “next slide” and “go back.” A simple appliance might support “start,” “pause,” and “stop.”

For a fixed vocabulary of a few commands—or, depending on the model and how similar the phrases are, a few to around ten or more—KWS can map each detected phrase directly to an action. The right limit depends on the vocabulary, acoustic similarity, target hardware, and measured false-trigger rate; there is no universal command-count cutoff.

KWS and ASR answer different questions

Automatic speech recognition (ASR) converts speech into text. That is useful when people can say many different things and the application needs the words they said.

KWS asks a narrower question: did one of these trained phrases occur? Because it solves a smaller problem, a KWS model can be much smaller and cheaper to run than a general-purpose recognizer. A compact model can fit into an offline product workflow where storage, memory, compute, network access, or privacy are constraints.

A small command model is not a miniature chatbot. It recognizes only the phrases it was trained to detect, under the conditions it has been tested for.

What does local ONNX inference mean?

ONNX is a portable format for representing machine-learning models. A compatible runtime can load an ONNX model on a supported device, so inference can happen locally instead of sending microphone audio to a recognition service.

microphone audio
    → resample and prepare short audio frames
    → extract the audio features expected by the model
    → run ONNX inference
    → smooth scores and apply detection rules
    → emit a keyword event
    → perform the app or device action

Once the model and runtime are installed, the inference path can work without an internet connection. That can reduce network dependency and keep recognition audio on the device. Local inference does not by itself guarantee privacy or low power: the application still needs to handle microphone permissions, audio capture, runtime behavior, and device power use appropriately.

ONNX also does not mean that every model runs unchanged on every chip. The target still needs a compatible ONNX runtime and enough memory and compute. Microcontrollers may need a different deployment format or an optimized runtime, such as a TensorFlow Lite Micro path. Always verify the complete model pipeline on the actual hardware.

The model file is only one part of the detector

A model usually produces scores for audio frames; the application decides when those scores count as a command. Triggering on one score crossing a threshold can lead to missed commands or repeated and false activations.

A practical detector may require a score to remain high across several frames, enforce a short cooldown after detection, and use thresholds tested against ordinary speech and background sounds. Commands that sound alike can be difficult to distinguish, so phrase choice matters too. For example, short commands with clearly different sounds are usually easier to separate than several phrases that share the same ending.

Test with the target microphone, speakers, distances, room noise, and accents. Track both missed detections and false activations. A model that performs well on generated samples may behave differently in a real enclosure or a noisy room.

When a small KWS model is a good fit

Consider KWS when the command list is short and known in advance, each phrase maps to a clear action, and offline behavior or keeping audio on-device matters.

Use ASR when people need to dictate text, phrase commands in many different ways, or say things the application did not anticipate. A hybrid design is also common: let KWS listen for a wake phrase, then start ASR only when the user is ready to speak freely.

A practical path to a prototype

Start by writing down the exact phrases and the action for each one. Check that commands are acoustically distinct, then choose a model and runtime that match your target platform. Connect detected keyword events to application actions, and test false activations as carefully as successful commands.

For a custom vocabulary, a KWS model can be trained for the phrases your product needs and exported in a deployment format such as ONNX. The open-source onnx-wakeword runtime provides inference integrations for several platforms. Voicute provides a workflow for creating custom keyword models at voicute.com.