On device wake word on ESP32-S3 is here – Voice: Chapter 6

TL;DR: We’ve added on-device wake phrase detection (microWakeWord)! It’s quicker and extra scalable than processing the wake phrase in Dwelling Assistant. We’ll maintain supporting wake phrase processing in Dwelling Assistant. Additionally new is extra customization for sentence triggers, extra intents for controlling extra units, and higher error messages and debugging instruments.

Watch the total Voice chapter 6 livestream

2023’s Year of the Voice constructed a strong basis for letting customers management Dwelling Assistant by talking in their very own language.

We proceed with enhancements to Assist, together with:

Oh, and “another factor”: on-device, open supply wake phrase detection in ESPHome! 🥳🥳🥳

Take a look at this video of the brand new microWakeWord system working on an ESP32-S3-BOX-3 alongside one doing wake phrase detection inside Dwelling Assistant:

On-device vs. streaming wake phrase

microWakeWord

Thanks to the incredible microWakeWord created by Kevin Ahrendt, ESPHome can now carry out wake phrase detection on units just like the ESP32-S3-BOX-3.
You may install it on your S3-BOX-3 today to strive it out.

Again in Chapter 4, we added wake phrase detection utilizing openWakeWord. Sadly, openWakeWord was too giant to run on low energy units like S3-BOX-3.
So we selected to run wake phrase detection inside Dwelling Assistant as a substitute.

Doing wake phrase detection in HA permits tiny units just like the M5 ATOM Echo Development Kit to easily stream audio and let the entire processing occur elsewhere. That is nice, because it permits low-powered units utilizing a easy ESP32 chip to be reworked right into a voice assistant even when they don’t pack the required energy to detect wake phrases.
The draw back is that including extra voice assistants requires extra CPU utilization in HA in addition to extra community visitors.

Enter microWakeWord. After listening to an interview with Paulus Schoutsen (founding father of Dwelling Assistant) on the Self Hosted podcast, Kevin Ahrendt created a mannequin based mostly on Google’s Inception neural network. As an present contributor to ESPHome, Kevin was capable of get this new mannequin working on the ESP32-S3 chip contained in the S3-BOX-3! (It additionally works on the, now discontinued, S3-BOX and S3-BOX-Lite)

Kevin has skilled three models for the launch of microWakeWord:

• “okay nabu”
• “hey jarvis”
• “alexa”

You may strive these out your self now by following the ESP32-S3-BOX tutorial. Altering the default “okay nabu” wake phrase would require adjusting your ESPHome configuration and recompiling the firmware, which can take a very long time and requires a machine with greater than 2GB of RAM.

We’re grateful to Kevin for growing microWakeWord, and making it part of the open residence!

Sentence trigger responses

Adding custom sentences to Assist is as easy as adding a sentence trigger to an automation. This lets you set off any motion in Dwelling Assistant with no matter sentences you need.

Now with the brand new conversation response motion in HA 2024.2, it’s also possible to customise the response spoken or printed again to you. Utilizing templating, your response can discuss with the present state of your house.

You can even discuss with wildcards in your sentence set off. For instance, the sentence set off:

might have the response:

Taking part in {{ set off.slots.album }} by {{ set off.slots.artist }}

along with calling a media service.

You may experiment now with sentence triggers, and customized dialog responses in our automation editor by clicking right here:

Improved errors and debugging

• The sentence didn’t match any known intent
• The system/space names didn’t match
• There weren’t any units of a selected sort in an space (lights, home windows, and so on.)

Beginning in HA 2024.2, Help gives totally different error messages for every of those instances.

Now if you happen to encounter errors, you’ll know the place to begin trying! The very first thing to test is that your system is exposed to Assist. Some varieties of units, similar to lights, are uncovered by default. Different, like locks, should not and should be manually uncovered.

As soon as your units are uncovered, be sure to’ve added an applicable alias so Help will know precisely the way you’ll be referring to them. Gadgets and areas can have a number of aliases, even in a number of languages, so everybody’s choice might be accommodated.

In case you are nonetheless having issues, the Assist debug tool has additionally been improved. Utilizing the software, you see how Help is deciphering a sentence, together with any lacking items.

Our group language leaders are exhausting at work translating sentences for Help. In case you have recommendations for brand spanking new sentences to be added, please create a problem on the intents repository or drop us a line at [email protected]

Thank you

Thank you to the Home Assistant community for subscribing to Home Assistant Cloud to assist voice and growth of Dwelling Assistant, ESPHome and different initiatives typically.

Because of our language leaders for extending the sentence assist to all the assorted languages.