TTS is failing when used with VoiceAssistant

[Saur0o0n]

Hi,
I thought that is was somehow related to response time, but now I'm not sure. When used with "Try voice" button (in Voice Assistants configuration) it can be slow, but it does work. Also when tested from Media menu - it's ok.
But when used with Voice Assistant it brakes the tts subroutine somehow - in logs I can only find this:

2025-03-08 17:14:23.985 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error() at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:844 (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 846, in handle_error
    if audio_task.exception():
       ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 786, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message, language, options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 492, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message=message, language=language, options=options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 509, in async_get_tts_audio
    return await self.hass.async_add_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        partial(self.get_tts_audio, message, language, options=options)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
asyncio.exceptions.CancelledError

[shizlkazizl]

Same issue

[SmarterTec]

I was having this issue as well and gave up. I instead switched to Microsoft's TTS which has the same voices plus more, is much more responsive, and is free. Took a little to setup the azure account but got it up and running and am using this integration: https://github.com/hugobloem/wyoming-microsoft-tts

[sfortis]

A major update is published with a lot of changes and fixes. Please check if your issues are fixed!

[Saur0o0n]

Hello, I don't think it change anything in that matter. At least for longer messages it still brakes

Logger: homeassistant
Source: components/tts/__init__.py:509
First occurred: 16:39:58 (1 occurrences)
Last logged: 16:39:58
Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error() at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:844 (None)

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 846, in handle_error
    if audio_task.exception():
       ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 786, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message, language, options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 492, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message=message, language=language, options=options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 509, in async_get_tts_audio
    return await self.hass.async_add_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        partial(self.get_tts_audio, message, language, options=options)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
asyncio.exceptions.CancelledError

it did manage to process shorter answer although. What's strange it looks like it's giving up immediately, not waiting for some timeout.
Also when I press "play audio" in the debug of voice assistant, it right away trows an error (no delay) - but perhaps it just doesn't try to tts it, but try to use cached audio (that doesn't exist), don't know.

Anyway, when try to tts the same longer text in Media sources - it's fine

[sfortis]

That's really strange. I've tested with really long responses and never had an issue. For example, I just asked assist to read a poem and it was reasonably fast (as the TTS still has no streaming capabilities). How LONG text you trying to tts?

Looking also the log, it looks like your home assistant is cancelling this request

Certainly! Here's an expanded version of the poem:

The sun sets low on the horizon's line,
Bathing the world in hues so fine.
Orange and pink, with gentle grace,
Colors dance across the sky's vast space.

In the heart of dusk, beauty displays,
The end of a bustling, vibrant day.
Nature whispers in a soothing tone,
Guiding the weary traveler home.

The breeze carries tales from distant lands,
Through rustling leaves and whispering sands.
The melody of night begins to play,
As daylight slowly fades away.

Stars emerge in the velvet sky,
Twinkling like dreams eager to fly.
Embraced by shadow, wrapped in night,
The world prepares for moonlit flight.

Rest now, gentle soul,

[Saur0o0n]

I've tried now in English - since so far it was all in Polish, but I have the same result:

If you can point how to debug it better - I can do it, but just enabling debug for integration does nothing (logs are the same).

This is "raw" output from conversation debug:

  - type: tts-start
    data:
      engine: tts.openai_tts_ash
      language: en
      voice: null
      tts_input: >-
        Ah, my szanowny panie, the "Litany Against Fear" is indeed a famous
        mantra from Frank Herbert's "Dune." While I cannot quote it verbatim, I
        can summarize its essence: it is a powerful meditative exercise designed
        to help individuals confront and overcome their fears.


        The essence of the litany emphasizes acknowledging fear as an emotion
        but also encourages individuals to move beyond it, seeking clarity and
        courage instead. The repeated phrases serve as a form of mental
        fortification, preparing one to face challenges with resolve.


        If you wish for a more detailed discussion about "Dune" or any specific
        themes and characters, I eagerly await your directives!
    timestamp: "2025-03-14T16:35:08.317691+00:00"
  - type: tts-end
    data:
      tts_output:
        media_id: >-
          media-source://tts/tts.openai_tts_ash?message=Ah,+my+szanowny+panie,+the+%22Litany+Against+Fear%22+is+indeed+a+famous+mantra+from+Frank+Herbert's+%22Dune.%22+While+I+cannot+quote+it+verbatim,+I+can+summarize+its+essence:+it+is+a+powerful+meditative+exercise+designed+to+help+individuals+confront+and+overcome+their+fears.%0A%0AThe+essence+of+the+litany+emphasizes+acknowledging+fear+as+an+emotion+but+also+encourages+individuals+to+move+beyond+it,+seeking+clarity+and+courage+instead.+The+repeated+phrases+serve+as+a+form+of+mental+fortification,+preparing+one+to+face+challenges+with+resolve.%0A%0AIf+you+wish+for+a+more+detailed+discussion+about+%22Dune%22+or+any+specific+themes+and+characters,+I+eagerly+await+your+directives!&language=en&tts_options=%7B%7D
        url: /api/tts_proxy/vIDEFtJ848u1aL-JD2GYjQ.mp3
        mime_type: audio/mpeg
    timestamp: "2025-03-14T16:35:08.318302+00:00"
  - type: run-end
    data: null
    timestamp: "2025-03-14T16:35:08.318341+00:00"

[sfortis]

@Saur0o0n The only time when I was able to reproduce a similar error was after restarting Home Assistant and calling the TTS while the chromecast integration was not ready. Trying again after 5-10 seconds, the TTS worked fine.

A new release has been published with some enhancements in fine-tuning sync/async calls, error catching, and debug logging, which may help.

Im not familiar with SyncWorker so i dont know how to help debugging this error further.

2025-03-15 09:40:16.005 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SpeechManager._async_get_tts_audio.<locals>.handle_error() at /usr/src/homeassistant/homeassistant/components/tts/__init__.py:844 (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 846, in handle_error
    if audio_task.exception():
       ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 786, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message, language, options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 492, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message=message, language=language, options=options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 509, in async_get_tts_audio
    return await self.hass.async_add_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        partial(self.get_tts_audio, message, language, options=options)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
asyncio.exceptions.CancelledError
2025-03-15 09:40:16.066 ERROR (Thread-13) [homeassistant.components.cast.media_player] Failed to cast media http://192.168.1.41:8123/api/tts_proxy/4hw0rYmEk0bdAH60oaKuKg.mp3 from internal_url (http://192.168.1.41:8123). Please make sure the URL is: Reachable from the cast device and either a publicly resolvable hostname or an IP address
2025-03-15 09:40:16.120 ERROR (Thread-18) [homeassistant.components.cast.media_player] Failed to cast media http://192.168.1.41:8123/api/tts_proxy/4hw0rYmEk0bdAH60oaKuKg.mp3 from internal_url (http://192.168.1.41:8123). Please make sure the URL is: Reachable from the cast device and either a publicly resolvable hostname or an IP address

[sfortis]

Having said that, I've started getting errors with an unusually high number of frames on the WAV response from OpenAI (frames=2147483647). Maybe switching back to mp3 is a good solution, but i need to work on the chime.

Im investigating this.

  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 492, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message=message, language=language, options=options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 509, in async_get_tts_audio
    return await self.hass.async_add_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        partial(self.get_tts_audio, message, language, options=options)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
asyncio.exceptions.CancelledError
2025-03-15 10:25:17.240 ERROR (Thread-18) [homeassistant.components.cast.media_player] Failed to cast media http://192.168.1.41:8123/api/tts_proxy/6CWcZ08mfb0b4swI50Wjdw.mp3 from internal_url (http://192.168.1.41:8123). Please make sure the URL is: Reachable from the cast device and either a publicly resolvable hostname or an IP address
2025-03-15 10:25:18.064 DEBUG (SyncWorker_6) [custom_components.openai_tts.tts] Effective chime option: True
2025-03-15 10:25:18.064 DEBUG (SyncWorker_6) [custom_components.openai_tts.tts] Chime option enabled; synthesizing chime and pause.
2025-03-15 10:25:18.064 DEBUG (SyncWorker_6) [custom_components.openai_tts.tts] TTS parameters: sample_rate=24000, channels=1, sampwidth=2, frames=2147483647

[Saur0o0n]

Hi, and thank you for all the updates.
At this point I think now the issue is somewhere in HA TTS component. I've tested it now also with GoogleTranslate TTS and local Piper TTS - both failed to answer with text around 620 characters. But the error path is a bit different for both.
Common part for all 3 tested TTS is here:

  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 786, in get_tts_data
    extension, data = await engine_instance.internal_async_get_tts_audio(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message, language, options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/src/homeassistant/homeassistant/components/tts/__init__.py", line 492, in internal_async_get_tts_audio
    return await self.async_get_tts_audio(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        message=message, language=language, options=options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^

So sadly nothing helpfull.

[sfortis]

Unfortunately, it seems like there's an issue with the core TTS component. There is a stale issue there, but so far no fixes or response on this.

home-assistant/core#108666

I'll try switching to MP3 and probably provide the option in the config flow to select the output format.

[Conclusio]

For reference, have the same problem, opened an issue in January. home-assistant/core#135713