No history data returned from history component via API (when in fact it should be available). Extending end time works

[adammcdonagh]

The problem

Hi,
This is related to #117793. It's essentially same problem still happening.

Example API call:

Yet if I run the call again straight await, it will return (but in this case I'm still hitting the issue from #117793 and getting an empty array.

Looking at the previous issue, if I add an end_time of today, it successfully returns the data (URL here is: https://homeassistant:8123/api/history/period/2024-12-23?filter_entity_id=sensor.power_load_no_var_loads&end_time=2024-12-25:

Currently it's 25 Dec at 22:21. If I query 23rd it returns the empty array.
Querying 24th, is currently also sometimes returning an empty array, but most of the time it's returning all of the data. e.g.

Querying 25th is returning the data.

Looking at the recorder runs table in the database, there's plenty of entries in there:

And querying the same sensor in HA GUI via History shows all the history is available going back as far as 1st December:

My recorder housekeeping is set as follows:

recorder:
  db_url: "mysql://xxxxx:[email protected]/homeassistant?charset=utf8"
  purge_keep_days: 35

What version of Home Assistant Core has the issue?

2024.12.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

recorder

Link to integration documentation on our website

https://www.home-assistant.io/integrations/recorder/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

This is specifically related to history lookups which are critical for the EMHASS addon to work. For no apparent reason this seems to happen after HA has been restarted (we think?). Not sure whether it's actually related to recorder itself, or if the history API is actually the issue here? It feels like it's more related to the API, since the data is clearly available in the DB, as the History page via the UI is able to display the data

For me, this has been working fine until today. I restarted HA yesterday after making some unrelated changes to configure the rflink integration. From about 4am today it appears that EHMASS has been receiving the blank JSON for any dates prior to 24th December.

After finding the related issue, I upgraded HA to 2024.12.5 hoping it'd fix the issue.

[home-assistant]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (recorder) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of recorder can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign recorder Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

recorder documentation
recorder source
_{^{(message by IssueLinks)}}

[adammcdonagh]

After some digging, I did what @tanelvakker suggested as a workaround in #117793 (comment)_, (using the component source code from the current dev branch) and that still fixes the issue as it seemed to have done before the PR was merged.

Though I just found #133561, so I'm confused as to what's actually meant to be fixed now in 2024.12.5 and what isn't :)

For now I'll stick with the modified version as custom component as I'm not sure what's what to even try and suggest a fix 😀