[Assist] Weather forecast not exposed to assist #138828
Assignees
Comments
|
Please provide the corresponding assist debug:
|
|
@mib1185 here you go stage: done
run:
pipeline: 01j71pnavrncqn3bqf2ct0t4ae
language: en
events:
- type: run-start
data:
pipeline: 01j71pnavrncqn3bqf2ct0t4ae
language: en
timestamp: "2025-02-19T16:45:56.481640+00:00"
- type: stt-start
data:
engine: stt.faster_whisper
metadata:
language: en
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
timestamp: "2025-02-19T16:45:56.481988+00:00"
- type: stt-vad-start
data:
timestamp: 1410
timestamp: "2025-02-19T16:45:57.861729+00:00"
- type: stt-vad-end
data:
timestamp: 4990
timestamp: "2025-02-19T16:46:01.438877+00:00"
- type: stt-end
data:
stt_output:
text: " Can you tell me the weather forecast for tomorrow?"
timestamp: "2025-02-19T16:46:02.381802+00:00"
- type: intent-start
data:
engine: conversation.chatgpt
language: en
intent_input: " Can you tell me the weather forecast for tomorrow?"
conversation_id: null
device_id: 25db0e02416b8a878e46110495c57eb6
prefer_local_intents: true
timestamp: "2025-02-19T16:46:02.381888+00:00"
- type: intent-end
data:
processed_locally: false
intent_output:
response:
speech:
plain:
speech: >-
Regrettably, it seems my capabilities do not extend to providing
weather forecasts. Perhaps consulting a weather app or a crystal
ball would yield more fruitful results.
extra_data: null
card: {}
language: en
response_type: action_done
data:
targets: []
success: []
failed: []
conversation_id: 01JMFHM9N5VGJ6PA83HQJQ97MN
timestamp: "2025-02-19T16:46:03.492251+00:00"
- type: tts-start
data:
engine: tts.piper
language: en-us
voice: glados
tts_input: >-
Regrettably, it seems my capabilities do not extend to providing weather
forecasts. Perhaps consulting a weather app or a crystal ball would
yield more fruitful results.
timestamp: "2025-02-19T16:46:03.492702+00:00"
- type: tts-end
data:
tts_output:
media_id: >-
media-source://tts/tts.piper?message=Regrettably,+it+seems+my+capabilities+do+not+extend+to+providing+weather+forecasts.+Perhaps+consulting+a+weather+app+or+a+crystal+ball+would+yield+more+fruitful+results.&language=en-us&tts_options=%7B%22voice%22:%22glados%22,%22preferred_format%22:%22flac%22,%22preferred_sample_rate%22:48000,%22preferred_sample_channels%22:1,%22preferred_sample_bytes%22:2%7D
url: /api/tts_proxy/yf2-nIKvjuo0FxiH8aMCow.flac
mime_type: audio/flac
timestamp: "2025-02-19T16:46:03.495358+00:00"
- type: run-end
data: null
timestamp: "2025-02-19T16:46:03.496075+00:00"
stt:
engine: stt.faster_whisper
metadata:
language: en
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
done: true
stt_output:
text: " Can you tell me the weather forecast for tomorrow?"
intent:
engine: conversation.chatgpt
language: en
intent_input: " Can you tell me the weather forecast for tomorrow?"
conversation_id: null
device_id: 25db0e02416b8a878e46110495c57eb6
prefer_local_intents: true
done: true
processed_locally: false
intent_output:
response:
speech:
plain:
speech: >-
Regrettably, it seems my capabilities do not extend to providing
weather forecasts. Perhaps consulting a weather app or a crystal
ball would yield more fruitful results.
extra_data: null
card: {}
language: en
response_type: action_done
data:
targets: []
success: []
failed: []
conversation_id: 01JMFHM9N5VGJ6PA83HQJQ97MN
tts:
engine: tts.piper
language: en-us
voice: glados
tts_input: >-
Regrettably, it seems my capabilities do not extend to providing weather
forecasts. Perhaps consulting a weather app or a crystal ball would yield
more fruitful results.
done: true
tts_output:
media_id: >-
media-source://tts/tts.piper?message=Regrettably,+it+seems+my+capabilities+do+not+extend+to+providing+weather+forecasts.+Perhaps+consulting+a+weather+app+or+a+crystal+ball+would+yield+more+fruitful+results.&language=en-us&tts_options=%7B%22voice%22:%22glados%22,%22preferred_format%22:%22flac%22,%22preferred_sample_rate%22:48000,%22preferred_sample_channels%22:1,%22preferred_sample_bytes%22:2%7D
url: /api/tts_proxy/yf2-nIKvjuo0FxiH8aMCow.flac
mime_type: audio/flac |
|
Hey there @balloob, mind taking a look at this issue as it has been labeled with an integration ( Code owner commandsCode owners of
(message by CodeOwnersMention) openai_conversation documentation |
|
@mib1185 this is not OpenAI specific. Anthropic is the same: stage: done
run:
pipeline: 01j71pnavrncqn3bqf2ct0t4ae
language: en
events:
- type: run-start
data:
pipeline: 01j71pnavrncqn3bqf2ct0t4ae
language: en
timestamp: "2025-02-19T17:14:17.712336+00:00"
- type: stt-start
data:
engine: stt.faster_whisper
metadata:
language: en
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
timestamp: "2025-02-19T17:14:17.712692+00:00"
- type: stt-vad-start
data:
timestamp: 1050
timestamp: "2025-02-19T17:14:18.745201+00:00"
- type: stt-vad-end
data:
timestamp: 4140
timestamp: "2025-02-19T17:14:21.808405+00:00"
- type: stt-end
data:
stt_output:
text: " Can you tell me the weather forecast for tomorrow?"
timestamp: "2025-02-19T17:14:22.770946+00:00"
- type: intent-start
data:
engine: conversation.claude
language: en
intent_input: " Can you tell me the weather forecast for tomorrow?"
conversation_id: null
device_id: 25db0e02416b8a878e46110495c57eb6
prefer_local_intents: true
timestamp: "2025-02-19T17:14:22.771034+00:00"
- type: intent-end
data:
processed_locally: false
intent_output:
response:
speech:
plain:
speech: >-
I apologize, but I can only retrieve events from calendars, not
weather forecasts. The current weather sensor shows it's a clear
night with 0°C temperature and 45% humidity. For a detailed
forecast, I recommend consulting a meteorological service.
Perhaps you'd like me to check if any interesting events are
scheduled for tomorrow? I could retrieve events from Thomas's or
Phoebe's calendar if you're curious about upcoming activities.
extra_data: null
card: {}
language: en
response_type: action_done
data:
targets: []
success: []
failed: []
conversation_id: 01JMFK865V6N806V1RE6B1BB3Z
timestamp: "2025-02-19T17:14:26.588220+00:00"
- type: tts-start
data:
engine: tts.piper
language: en-us
voice: glados
tts_input: >-
I apologize, but I can only retrieve events from calendars, not weather
forecasts. The current weather sensor shows it's a clear night with 0°C
temperature and 45% humidity. For a detailed forecast, I recommend
consulting a meteorological service.
Perhaps you'd like me to check if any interesting events are scheduled
for tomorrow? I could retrieve events from Thomas's or Phoebe's calendar
if you're curious about upcoming activities.
timestamp: "2025-02-19T17:14:26.588557+00:00"
- type: tts-end
data:
tts_output:
media_id: >-
media-source://tts/tts.piper?message=I+apologize,+but+I+can+only+retrieve+events+from+calendars,+not+weather+forecasts.+The+current+weather+sensor+shows+it's+a+clear+night+with+0%C2%B0C+temperature+and+45%25+humidity.+For+a+detailed+forecast,+I+recommend+consulting+a+meteorological+service.+%0A%0APerhaps+you'd+like+me+to+check+if+any+interesting+events+are+scheduled+for+tomorrow?+I+could+retrieve+events+from+Thomas's+or+Phoebe's+calendar+if+you're+curious+about+upcoming+activities.&language=en-us&tts_options=%7B%22voice%22:%22glados%22,%22preferred_format%22:%22flac%22,%22preferred_sample_rate%22:48000,%22preferred_sample_channels%22:1,%22preferred_sample_bytes%22:2%7D
url: /api/tts_proxy/QVY_KkmcPQ7CYm_uiOcPmA.flac
mime_type: audio/flac
timestamp: "2025-02-19T17:14:26.589080+00:00"
- type: run-end
data: null
timestamp: "2025-02-19T17:14:26.589181+00:00"
stt:
engine: stt.faster_whisper
metadata:
language: en
format: wav
codec: pcm
bit_rate: 16
sample_rate: 16000
channel: 1
done: true
stt_output:
text: " Can you tell me the weather forecast for tomorrow?"
intent:
engine: conversation.claude
language: en
intent_input: " Can you tell me the weather forecast for tomorrow?"
conversation_id: null
device_id: 25db0e02416b8a878e46110495c57eb6
prefer_local_intents: true
done: true
processed_locally: false
intent_output:
response:
speech:
plain:
speech: >-
I apologize, but I can only retrieve events from calendars, not
weather forecasts. The current weather sensor shows it's a clear
night with 0°C temperature and 45% humidity. For a detailed
forecast, I recommend consulting a meteorological service.
Perhaps you'd like me to check if any interesting events are
scheduled for tomorrow? I could retrieve events from Thomas's or
Phoebe's calendar if you're curious about upcoming activities.
extra_data: null
card: {}
language: en
response_type: action_done
data:
targets: []
success: []
failed: []
conversation_id: 01JMFK865V6N806V1RE6B1BB3Z
tts:
engine: tts.piper
language: en-us
voice: glados
tts_input: >-
I apologize, but I can only retrieve events from calendars, not weather
forecasts. The current weather sensor shows it's a clear night with 0°C
temperature and 45% humidity. For a detailed forecast, I recommend
consulting a meteorological service.
Perhaps you'd like me to check if any interesting events are scheduled for
tomorrow? I could retrieve events from Thomas's or Phoebe's calendar if
you're curious about upcoming activities.
done: true
tts_output:
media_id: >-
media-source://tts/tts.piper?message=I+apologize,+but+I+can+only+retrieve+events+from+calendars,+not+weather+forecasts.+The+current+weather+sensor+shows+it's+a+clear+night+with+0%C2%B0C+temperature+and+45%25+humidity.+For+a+detailed+forecast,+I+recommend+consulting+a+meteorological+service.+%0A%0APerhaps+you'd+like+me+to+check+if+any+interesting+events+are+scheduled+for+tomorrow?+I+could+retrieve+events+from+Thomas's+or+Phoebe's+calendar+if+you're+curious+about+upcoming+activities.&language=en-us&tts_options=%7B%22voice%22:%22glados%22,%22preferred_format%22:%22flac%22,%22preferred_sample_rate%22:48000,%22preferred_sample_channels%22:1,%22preferred_sample_bytes%22:2%7D
url: /api/tts_proxy/QVY_KkmcPQ7CYm_uiOcPmA.flac
mime_type: audio/flac |
mib1185
added
integration: conversation
and removed
integration: openai_conversation
labels
|
Hey there @home-assistant/core, @synesthesiam, mind taking a look at this issue as it has been labeled with an integration ( Code owner commandsCode owners of
(message by CodeOwnersMention) conversation documentation |
|
@mib1185 I wanted to debug this in more detail, but I cant figure out how to launch detailed debugging of assist. E.g. I would like to see the whole messages sent to AI provider, including the system prompt and list of tools. And I would like to see the individual tool calls. How to enable this? If I debug OpenAI integration, it does not give me the prompt e.g. But is there some way to debug the voice assistant on some global level? so I see all - tts details, stt and AI calls with all the details. |
|
I think setting the log level to debug for |
|
aha, I see. So the issue is in the conversation.session. It sets this for the weather entity: - names: WeatherStation
domain: weather
state: clear-night
attributes:
temperature: 0.0
temperature_unit: °C
humidity: '47'Is the list of attributes exposed to AI customizable, or this is hardcoded per type? |
|
I don't think we expose forecast information just yet. |
|
@synesthesiam thanks for getting back so quickly. I went over what is exposed and I dont believe this can be the full prompt. E.g. there is nowhere mentioned my calendar (which is exposed), yet when I ask my AI about it, it knows what tool to call (and the name of the calendar). The prompt I see in logs is very long, there are lights, shutters, media players. But not calendar. How does it know about it? |
Not by default - Why Fez has to pump the data to the llm in the scripts he wrote: https://community.home-assistant.io/t/blueprints-for-voice-commands-weather-calendar-music-assistant/838071 Also Op - you can use this as a workaround until it's builtin. Fez' has blueprints for weather and media that work in most scenarios. HIGHLY recommended. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The problem
I have weather entity (I tried several integrations) and on dashboard I see forecast nicely
but even though I exposed it to voice assistant, it complains it cannot read forecast data.
What version of Home Assistant Core has the issue?
core-2025.2.4
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant Container
Integration causing the issue
No response
Link to integration documentation on our website
No response
Diagnostics information
No response
Example YAML snippet
Anything in the logs that might be useful for us?
Additional information
No response
The text was updated successfully, but these errors were encountered: