I set up the perfect voice pipeline in Home Assistant, and here's how I did it

Home Assistant is a powerful tool that you can use to link up countless services in a way to take back control of your smart home and build it the way you like. It's not for everyone, but there's a massive community supporting it that consistently makes it easier to do various different things. It has complete voice assistant support, and I've built my own voice pipeline, for free, so that I control exactly how it runs and how it responds.

Initially, I had used Nabu Casa's Home Assistant Cloud to power my Assistant, as it makes linking up Google Assistant and Alexa to your Home Assistant configuration incredibly easy, too. This comes with speech-to-text and text-to-speech services, with the latter offering a wide range of voices to choose from to customize your assistant and make it sound exactly how you want it to. However, through two self-hosted tools, I've made the perfect voice pipeline in Home Assistant for controlling my home.

I've deployed this on my Home Assistant Voice Preview Edition that I purchased, but you can use this through the Home Assistant app on your smartphone or on the web dashboard, too. I deployed this on Home Assistant OS, but you can deploy it on the Container edition, too; it just requires configuring the two add-ons we'll be talking about manually.

Installing Whisper and Piper

Free add-ons

We're going to be replacing the two cloud-based components of our voice assistant pipeline, which you can find under Settings and Voice Assistants. Whisper is the first and simplest to install, as you can find it in the Home Assistant Add-on repository. It's a simple set-up; install it, enable automatic startup, and go to your devices section in Home Assistant, where it will be automatically detected and should offer you to install the Wyoming Protocol integration. This will then expose Whisper as a speech-to-text model that you can include in your voice pipeline.

Whisper is a fast transcription tool developed by OpenAI, and it's completely open-source and uses very few resources. Out of the box, it's been perfect for me, and I haven't had to make any changes to it to get it to work on any of my devices. For my Voice Preview box, my PC, or my phone, it's been as good as my Google Home Hub (which it replaced) was at understanding requests and what I say. Even before deploying Piper, I was using Whisper for fast transcription as I found it quicker than using Home Assistant Cloud.

However, the next deployment is Piper. If you want to set up a custom voice, you can (it's a bit of work, though), but it also comes with quite a few voices pre-packaged that you can use. Piper synthesizes speech on the fly, just like Home Assistant Cloud does, except it runs on-device. Again, it's a quick install; install it from the Add-on repository, enable automatic startup, and it'll be detected in devices under the Wyoming Protocol integration.

Finally, go to Settings, Voice Assistants, and create a new voice assistant. Set Speech-to-text to "faster-whisper", and Text-to-speech to "piper." Here, you can try out voices and listen to them, so that you can pick exactly how you want the voice to sound. The Conversation agent can be left as Home Assistant if you just want to use the basic built-in controls for your home, or if you want to build your own commands, but you can also use a cloud-based or locally-hosted LLM here.

A voice pipeline you control

No internet required

This setup can take as little as five minutes under Home Assistant OS, and it was absolutely worth it. Whisper and Piper are both very fast, even on my Ryzen 7 5700U-based mini PC that powers this entire pipeline. I've paired it with a local LLM and with ChatGPT's API for testing, and both work well when designated as the conversation agent. You could sit there and have a full, out-loud conversation with your LLM if you'd like using this, all with the benefit of it understanding the complete context of your home and being able to issue commands on the fly.

Even if you don't go that far with it, these local controls enable an entirely custom voice pipeline running on your own network, without any additional fees or even an internet connection. You can configure custom sentences for controls, build your own automations, and more, all without an internet connection. Even before deploying anything else, and still using Home Assistant Cloud's speech synthesis, Whisper was one of the best additions I made and is something that I've had since I set up Home Assistant OS.

Even if you aren't using HAOS, you can still deploy these in containers and hook them up manually instead. I've been loving it, and you almost certainly will too.