Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Recently Added shouldn't rely on Update but Create timestamp #2650

Open
3 tasks done
anarcat opened this issue Nov 22, 2023 · 14 comments · May be fixed by #2681
Open
3 tasks done

[Bug]: Recently Added shouldn't rely on Update but Create timestamp #2650

anarcat opened this issue Nov 22, 2023 · 14 comments · May be fixed by #2681
Labels
bug stale triage New bug reports that need to be evaluated

Comments

@anarcat
Copy link

anarcat commented Nov 22, 2023

I confirm that:

  • I have searched the existing open AND closed issues to see if an issue already exists for the bug I've encountered
  • I'm using the latest version (your issue may have been fixed already)

Version

0.50.1

Current Behavior

Modifications to (say) an album cover brings said album back into the top of the "Recently Added" queue, even though it might be an old album.

Expected Behavior

Instead, "Recently Added" should only rely on the creation (or import?) date of the file that's being processed.

Steps To Reproduce

  1. import two albums in Navidrome, with file timestamps far enough apart to make them sortable
  2. modify the old one by, say, adding an album cover

Expected result: the newer album before step 2 should still show up as the newest.

Actual result: modified album bumps the newer album.

Environment

- OS: Debian GNU/Linux bookworm, Docker container
- Browser: Firefox ESR, but irrelevant
- Client: Supersonic or web

How Navidrome is installed?

Docker

Configuration

---
# managed by Puppet, local changes will be lost
version: "3"

volumes:
  navidrome-data:

services:
  navidrome:
    image: deluan/navidrome:latest
    user: 10000:10000  # should be owner of volumes
    ports:
      - "127.0.0.1:4533:4533"
    restart: unless-stopped
    environment:
      ND_SCANSCHEDULE: 1h
      ND_LOGLEVEL: info
      ND_SESSIONTIMEOUT: 24h
      ND_BASEURL: ""
      ND_RECENTLYADDEDBYMODTIME: "true"
      ND_LASTFM_ENABLED: "false"
      ND_AUTOIMPORTPLAYLISTS: "false"
      ND_ENABLESHARING: "true"
    volumes:
      - "navidrome-data:/data"
      - "/srv/mp3:/music/mp3:ro"


### Relevant log output

```shell
I perused the logs and didn't find much interesting, so I do not believe that is relevant.

Anything else?

At first I thought this was because Navidrome only held a single timestamp for albums, but after inspecting the sqlite database (after fear of losing my ratings and play counts, which I think i did?), I noticed there was a created_at AND updated_at column, so that data is actually there, just not used correctly, I believe.

But here's two albums from the database that are sorted, IMHO, backwards:

sqlite> select name,artist,created_at,updated_at from album where name like '%American III: Solitary%';
American III: Solitary Man|Johnny Cash|2021-07-04T00:51:17.930079536Z|2023-03-09T22:58:57.431454258Z
sqlite> select name,artist,created_at,updated_at from album where name like '%Vertiges%';
Vertiges|zouz|2022-06-01T20:11:36.78922778Z|2022-06-01T19:44:39.486551366Z

There, "Johnny Cash" gets sorted first, as more "Recently Added" than "zouz".

Now of course, one fix to this is to run an SQL query updating my entire database to wipe off the differnet update_at timestamps, or at least tweak the ones that are incorrect, but that seems like shoving a problem under the rug...

Besides, maybe we could have a Recently Updated view as well? That would be interesting, listing the albums that have a different created_at and updated_at timestamp! :) Unfortunately, it looks like all those timestamps differ here, sometimes by only a few seconds...

Code of Conduct

  • I agree to follow Navidrome's Code of Conduct
@anarcat anarcat added bug triage New bug reports that need to be evaluated labels Nov 22, 2023
@anarcat
Copy link
Author

anarcat commented Nov 22, 2023

In my case, the workaround was relatively simple: I added a lot (350+) of covers all at once, in a short timeframe, so I could rewrite the update_at fields on those relatively easily with this query:

UPDATE album SET updated_at = created_at WHERE updated_at > '2023-03-06' AND updated_at < '2023-03-10' AND created_at < '2023-03-01';

But of course, if I fix an old cover again, that poor old album is going to come back up the list and try to play with the young kids and everyone is going to look at him sideways and think "ok boomer" and release a barely hidden sigh that will just make everyone really uncomfortable...

@metalheim
Copy link
Contributor

You have set ND_RECENTLYADDEDBYMODTIME to true which is not what you want from what your description reads.

https://www.navidrome.org/docs/usage/configuration-options/#advanced-configuration

The functionality you describe is implemented with #2553 as long as you dont Set ND_RECENTLYADDEDBYMODTIME (default is False)

@anarcat
Copy link
Author

anarcat commented Nov 22, 2023

Hmm... well that setting doesn't do what I thought it did though: I thought that would set the created_at field on import, but from what you seem to be saying here, it influences how the albums are displayed?

the created_at fields are correctly set here, just to be really clear.

@deluan
Copy link
Member

deluan commented Nov 22, 2023

but from what you seem to be saying here, it influences how the albums are displayed?

That's correct. It is used for controlling how to display the Recent Added Albums view.

@anarcat
Copy link
Author

anarcat commented Nov 22, 2023

That's correct. It is used for controlling how to display the Recent Added Albums view.

okay, so couldn't this be made in reverse, like at import time, if that setting is on, use the file mtime as the created_at field instead of updated_at? for me this thing is completely setup backwards, but maybe that's just me. :)

@deluan
Copy link
Member

deluan commented Nov 23, 2023

I think the fields should be used for what they mean: created_at for when the file was created/first imported (see #1350), and updated_at for when was the last time the record was changed in the DB. Inverting these two can cause confusion when maintaining the codebase, and it is just a matter of displaying sorted by one or the other, with no need to change their semantics.

Also, making this distinction at scan time, means that if you want to change it, Navidrome needs to rescan everything again - with a full scan - which can take days sometimes for some large libraries.

@anarcat
Copy link
Author

anarcat commented Nov 23, 2023

i see. so what are my options here?

i guess i could remap the created_at timestamp to updated_at if the latter is lower? would that work?

@deluan
Copy link
Member

deluan commented Nov 23, 2023

Instead, "Recently Added" should only rely on the creation (or import?) date of the file that's being processed.

That should be the behaviour if you remove ND_RECENTLYADDEDBYMODTIME: "true". The default behaviour (false) is to use import date (or birth date as per #1350)

@anarcat
Copy link
Author

anarcat commented Nov 23, 2023

That should be the behaviour if you remove ND_RECENTLYADDEDBYMODTIME: "true". The default behaviour (false) is to use import date (or birth date as per #1350)

okay well maybe that's where my confusion comes from: I thought ND_RECENTLYADDEDBYMODTIME was a feature flag that enabled the "birth date used on import" defined in #1350 - those two are actually unrelated?

@deluan
Copy link
Member

deluan commented Nov 23, 2023

ND_RECENTLYADDEDBYMODTIME was a feature flag that enabled the "birth date used on import" defined in #1350

No it is not!

ND_RECENTLYADDEDBYMODTIME selects whether to use created_at (false, default) or updated_at (true) for sorting the Recently Added albums view. This was implemented by #822 a couple of years ago.

#1350 (fixed by #2553) was released in 0.50.0, which replaces import date with birth date, on systems that support it, as that's a sensible thing to do (use the actual file creation date instead of the import date)

@anarcat
Copy link
Author

anarcat commented Nov 27, 2023

Ookay... I completely misread this! thanks for the clarification...

is there a way to rescan files so that their creation date is retrofitted back into the created_at field?

@deluan
Copy link
Member

deluan commented Nov 27, 2023

Try a full rescan. If that doesn't work, there's no other option except for re-adding all your files, which currently will make you lose your playlists and favourites/ratings/play counts :/

@certuna
Copy link
Contributor

certuna commented Nov 30, 2023

i noticed to that if the song is already in the database, a full rescan will not refresh it with the file creation date.

@certuna certuna linked a pull request Dec 3, 2023 that will close this issue
Copy link

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Navidrome team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

@github-actions github-actions bot added the stale label May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug stale triage New bug reports that need to be evaluated
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants