Teaching my Voice PE to answer to "Hey Snoopy"

I picked up a Home Assistant Voice Preview Edition (the Voice PE: an ESP32-S3 smart speaker that Nabu Casa sells for Home Assistant) and wanted it to wake up to "Hey Snoopy", the name of my AI agent here at the house, instead of "Okay Nabu" or "Hey Jarvis". The device doesn't ship with a way to add your own wake word. Here's how I did it, in three parts: train a model, build firmware that includes it, and flash it over the air.

You'll need:

• a Voice PE, or any ESP32-S3 device running ESPHome's micro_wake_word;
• a Linux machine with an NVIDIA GPU (I used an RTX 4060 Ti with 16 GB), Docker and about 60 GB of SSD space;
• about 7 hours, mostly waiting.

All the scripts are in RickAtSnowcap/microwakeword-custom, and its README has every command. This page explains what they do and what I learned.

Part 1: train the wake word

The Voice PE already runs microWakeWord: a tiny streaming TensorFlow Lite model that runs on the ESP32 itself and listens for its phrase without sending audio anywhere. So the job is to train a new microWakeWord model in the same format as the stock ones.

The community recipe is a Colab notebook that wants a paid A100 runtime. I ran it locally instead, while my speech service kept about 6 GB of the GPU busy. TensorFlow was capped at 4 GB so nothing else fell over. It cost nothing, and none of the data left the house.

You don't have to record yourself

My first instinct was to record "Hey Snoopy" a few hundred times in my own voice. It turns out you can train it with synthetic voices instead. I use Chatterbox for the house's voice because it sounds incredible, but Piper has far more voices for generating short clips and is much easier on the GPU, so I picked it for this.

• Positives: Piper generated 20,000 synthetic "hey snoopy" clips across hundreds of blended voices, at different speeds and pitches. The training pipeline then augments them with room impulse responses (270 real rooms), background audio from AudioSet and the Free Music Archive, EQ, distortion and noise, down to −5 dB signal-to-noise. In effect, it's 20,000 people saying it in 20,000 noisy kitchens.
• Negatives: microWakeWord's published sets of ordinary speech, dinner-party chatter and silence, so the model learns what isn't its name.

source scripts/env.sh
PHRASE="hey snoopy" scripts/01_generate.sh    # 20,000 synthetic clips
scripts/02_download.sh                          # rooms, background audio, negative sets
(cd data && ../venvs/mww/bin/python -I ../scripts/03_features.py)   # augment + spectrograms
VARIANT=v1 scripts/05_train.sh                  # the notebook recipe, about 2 hours

Version 1 looked great, until "Hey Siri"

The first model (v1, the straight notebook recipe) trained in about 2 hours and looked great on paper: it caught about 94% of held-out test clips at its recommended cutoff. Then I tested it against sound-alike phrases. It fired on 100% of "Hey Siri" clips and on 92% of plain "Snoopy". In a house with iPhones and Peanuts on TV, that's the mistake that matters. Oops.

Version 2: teach it what it isn't

So I built v2. It generates another 20,000 clips of 20 sound-alike phrases (hey siri, hey susie, hey stupid, hey soupy, hey scooby, snoopy on its own, hey jarvis, okay nabu and so on), adds them as negatives, and retrains:

scripts/10_generate_adversarial.sh     # phrases come from scripts/adversarial_phrases.txt
(cd data && ../venvs/mww/bin/python -I ../scripts/03_features.py \
  $WW/data/adversarial_samples $WW/data/adversarial_augmented_features 1 2)
VARIANT=v2 scripts/05_train.sh         # about 3 hours

If you train your own phrase, edit adversarial_phrases.txt first: the sound-alikes for "hey jasper" aren't the ones for "hey snoopy".

The results, testing 50 fresh clips of each phrase:

Phrase	v1 wakes on	v2 wakes on
"Hey Siri"	100%	6%
"Snoopy" on its own	92%	14%
"Hey Sophie", "Hey Stacy" (never seen in training)	100%	0%
"Hey Jarvis", "Okay Nabu", "Hey Google"	2–12%	0%
"Hey Snoopy" in a different TTS voice (Chatterbox)	100%	94%

v2 also had just 2 false wakes in 15 hours of dinner-party recordings, at a probability cutoff of 0.85.

The final model is 62 KB with about 26,000 parameters. I even measured the memory it needs on the device (the tensor arena) with the TensorFlow Lite Micro reference interpreter, after checking the method against the official models first.

The whole session took about 7 hours. The bottleneck turned out to be the spinning 8 TB hard drive I'd put it on, because I wasn't sure how much space it would need. Training reads features at random, so the workspace has since moved to NVMe. Only about 60 GB was needed in the end. This was my first real machine learning project, and it was very educational.

Part 2: build the firmware

The Voice PE runs ESPHome. Rather than fork Nabu Casa's firmware, I wrote a small local ESPHome config that pulls in their official package, pinned to the exact release the device was already running. It keeps the same device name, so Home Assistant sees the same device, and adds one entry to micro_wake_word: the new model, alongside the stock ones.

substitutions:
  name: home-assistant-voice-xxxxxx        # your device's existing name
  friendly_name: Home Assistant Voice xxxxxx

packages:
  voice_pe: github://esphome/home-assistant-voice-pe/home-assistant-voice.yaml@26.9.0

esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}

improv_serial:

micro_wake_word:
  models:
    - model: wakewords/hey_snoopy.json
      id: hey_snoopy

That's the whole file. To update later, I bump the @26.9.0 tag and rebuild.

The .json beside the model is a small manifest: the wake word's name, the model file, the cutoff and the tensor arena size. There's an example in the repo's esphome/ folder.

The build runs in the official ESPHome Docker image, so nothing gets installed on the host:

docker run --rm -v "$PWD":/config ghcr.io/esphome/esphome compile voice-pe-custom-wake-word.yaml

One snag: the first compile failed a Wi-Fi validation check. The factory firmware compiles in no Wi-Fi credentials (the device keeps the network you gave it during setup), and my first config had left out the provisioning piece that makes that legal. Adding improv_serial: back in fixed it, and it also gives me a recovery path: if the device ever loses its network, I can set Wi-Fi again over USB.

	RAM	Flash
Stock build	44.6%	35.0%
With Hey Snoopy	44.9% (153 KB)	35.7%, a 2.9 MB OTA image

I'll admit I was nervous about bricking a brand-new device on day one. So before anything got flashed, I confirmed the USB recovery route and saved the stock config as a backup.

Part 3: flash it over the air

This was the anticlimax, in the best way. I found the device's IP in Home Assistant's ESPHome integration and pushed the image straight from the Docker container to the speaker over Wi-Fi:

docker run --rm --network host -v "$PWD":/config ghcr.io/esphome/esphome \
  upload voice-pe-custom-wake-word.yaml --device 192.168.1.50   # your device's IP

• The upload took about 16 seconds.
• The device rebooted and was back on Wi-Fi about 10 seconds later.
• Home Assistant reconnected on its own.
• "Hey Snoopy" appeared in the wake-word picker. I put it in slot 1, with Hey Jarvis as the backup.

I said "Hey Snoopy" from across the room. It worked on the first try and the second, and the lights did what I told them. I've pushed firmware over the air to GPS trackers on cellular networks, so I know how these things can go. This one went without a single problem.

Tuning: one voice it kept missing

A couple of days later, it turned out the speaker sometimes missed one family member's voice. The fix didn't need any retraining: you can override a model's cutoff in the ESPHome config. I lowered it from 0.85 to 0.75 and flashed again:

micro_wake_word:
  models:
    - model: wakewords/hey_snoopy.json
      id: hey_snoopy
      probability_cutoff: 0.75   # trained at 0.85; lower = more sensitive

One gotcha: the firmware's "Wake word sensitivity" setting in Home Assistant only adjusts the stock models. It never touches a custom one, so the YAML is the place to tune it.

---

If you have a Voice PE, or any ESP32-S3 running microWakeWord, the scripts are on GitHub. Your wake word could be anything.