Offline Keyword Spotting on Android with ONNX Runtime

August 2026 · Android 8+ · Java · ONNX Runtime

A voice-controlled app does not always need speech-to-text. If the command list is fixed—“take photo”, “stop”, “next”, “back”—keyword spotting is smaller, private, and easier to connect to UI actions.

Goal: run a multi-command model on the microphone and call Android code when a command is detected. All inference remains on the device.

KWS or ASR?

Use keyword spotting	Use speech recognition
2–10 fixed commands	Open-ended sentences or dictation
Immediate local actions	Need the complete transcript
Offline privacy and predictable vocabulary	Large and changing vocabulary

What the open Android project already includes

AudioCapture.java: microphone capture and a lock-free ring buffer
WakeWordEngine.java: model loading, Mel extraction, and ONNX inference
DetectionLogic.java: consecutive-frame, cooldown, ratio, burst, and energy filters
A runnable Android Studio demo for Android 8.0 / API 26 and newer

Clone and open the demo

git clone https://github.com/voicute/onnx-wakeword.git

Open the repository's android/ directory in Android Studio and let Gradle synchronize.

Add the model assets

Copy the keyword ONNX file and model_info.json into android/app/src/main/assets/. The configuration names the model and lists its commands.

{
  "model_type": "multi_keyword",
  "keywords": ["take photo", "stop", "next", "back"],
  "model_file": "commands.onnx",
  "mel_time": 98,
  "n_mels": 32,
  "cons_frames": 2
}

Request microphone permission

The demo includes audio capture, but a production app must explain why it needs RECORD_AUDIO, request runtime permission, and stop capture when the feature is disabled.

Load and process audio

WakeWordEngine engine = new WakeWordEngine(context);
engine.load("model_info.json", "melspectrogram.onnx");

DetectionResult result = engine.process(audioChunk);
if (result.detected) {
    runOnUiThread(() -> handleCommand(result.word));
}

Map words to safe app actions

private void handleCommand(String word) {
    switch (word) {
        case "take photo": capturePhoto(); break;
        case "next": showNextPage(); break;
        case "back": onBackPressed(); break;
        case "stop": stopCurrentAction(); break;
    }
}

Keep the recognized vocabulary separate from privileged operations. Require confirmation for purchases, deletion, unlocking, or other destructive actions.

Production checklist

Test similar-sounding commands and different speakers.
Stop the microphone in the correct Activity/Service lifecycle state.
Use a foreground service only when the user clearly expects continuous listening.
Start with the L1 consecutive-frame filter; add cooldown or energy filtering only when needed.
Measure battery use on the target device before claiming always-on suitability.

Current boundary: the open Android implementation uses ONNX Runtime. Voicute can export TFLite models, but this repository does not yet provide a complete Android TFLite SDK.

Disclosure: Voicute develops the commercial model generator and the open-source Android inference demo.

Create a multi-keyword model