Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(youtube)!: Use oauth-based api library for better stability #203

Merged
merged 4 commits into from
Oct 31, 2024

Conversation

FoxxMD
Copy link
Owner

@FoxxMD FoxxMD commented Oct 9, 2024

Checklist before requesting a review

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Describe your changes

Migrate from youtube-music-ts-api, which only supports cookie-based auth -- to Youtube.js, which uses oauth auth.

Usage

Start multi-scrobbler with any existing YTM configuration. A log statement will prompt the user to visit a URL with a code (or it can be accessed from the dashboard)

[2024-10-09 15:24:17.358 -0400] INFO   : [App] [Sources] [Ytmusic - MyYTM] ERROR: Sign in with the code 'CLV-KFA-BVKY' using the authentication link on the dashboard or https://www.google.com/device

Visit the URL and enter the code, then finish the auth flow. Afterwards, refresh the dashboard and click Start on the Idle YTM source to start polling.

Issue number and link, if applicable

@FoxxMD FoxxMD added enhancement New feature or request safe to test trusted to build image labels Oct 9, 2024
@FoxxMD FoxxMD self-assigned this Oct 9, 2024
@FoxxMD FoxxMD force-pushed the GH-195/ytoauth branch 4 times, most recently from 86d34bc to 72d79fc Compare October 11, 2024 17:49
@xathon
Copy link

xathon commented Oct 27, 2024

I think something in 671056d broke logging a lot :D it seems that for everything it does, it creates a log file. My logs directory immediately fills up with thousands of mostly empty log files.

Just to give you an idea, these are some of the log files (the ones that aren't empty), note the file names:

./scrobble-2024-10-27.3320.log:
[2024-10-27 02:20:09.619 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] No new tracks discovered
./scrobble-2024-10-27.3321.log:
[2024-10-27 02:20:09.620 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-10-27T02:20:09+01:00 | Next check interval: 10.00s
./scrobble-2024-10-27.9861.log:
[2024-10-27 02:20:17.625 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] Refreshing recently played
./scrobble-2024-10-27.9934.log:
[2024-10-27 02:20:17.713 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] No new tracks discovered
[2024-10-27 02:20:17.715 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] Last activity was at 2024-10-27T02:20:07+01:00 | Next check interval: 10.00s
./scrobble-2024-10-27.11521.log:
[2024-10-27 02:20:19.626 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played
./scrobble-2024-10-27.11981.log:
[2024-10-27 02:20:20.289 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] No new tracks discovered
[2024-10-27 02:20:20.290 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-10-27T02:20:09+01:00 | Next check interval: 10.00s
./scrobble-2024-10-27.18178.log:
[2024-10-27 02:20:27.793 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] Refreshing recently played
./scrobble-2024-10-27.18244.log:
[2024-10-27 02:20:27.884 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] No new tracks discovered
./scrobble-2024-10-27.18246.log:
[2024-10-27 02:20:27.886 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] Last activity was at 2024-10-27T02:20:07+01:00 | Next check interval: 10.00s
./scrobble-2024-10-27.19864.log:
[2024-10-27 02:20:29.794 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played
./scrobble-2024-10-27.20314.log:
[2024-10-27 02:20:30.375 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] No new tracks discovered
[2024-10-27 02:20:30.375 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-10-27T02:20:09+01:00 | Next check interval: 10.00s
./scrobble-2024-10-27.26602.log:
[2024-10-27 02:20:37.890 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] Refreshing recently played
./scrobble-2024-10-27.26661.log:
[2024-10-27 02:20:37.980 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] No new tracks discovered
./scrobble-2024-10-27.26663.log:
[2024-10-27 02:20:37.982 +0100] DEBUG  : [App] [Sources] [Spotify - unnamed] Last activity was at 2024-10-27T02:20:07+01:00 | Next check interval: 10.00s
./scrobble-2024-10-27.28219.log:
[2024-10-27 02:20:39.880 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played
./scrobble-2024-10-27.28643.log:
[2024-10-27 02:20:40.447 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] No new tracks discovered
[2024-10-27 02:20:40.448 +0100] DEBUG  : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-10-27T02:20:09+01:00 | Next check interval: 10.00s

@FoxxMD
Copy link
Owner Author

FoxxMD commented Oct 28, 2024

@xathon the issue is a bug in the underlying log library first found here #214 (comment) . I have a PR opened to fix this that will get merged into this PR

@xathon
Copy link

xathon commented Oct 28, 2024

Ah, I didn't want to raise a separate issue since I had the PR image running and thought it came from that.

But yay, DST magic at it again :D

@FoxxMD
Copy link
Owner Author

FoxxMD commented Oct 28, 2024

@FluffyClaws @xathon other than the DST issue how is this PR working for you? Have you experienced any of the issues reported in #195 (duplicate scrobbles) after using it? I need more feedback from users before I can merge this.

@xathon
Copy link

xathon commented Oct 28, 2024

I think there was something. I'll check on it in the next couple of days and report back.

* Fixes #195 missing (404) history endpoint which only appear when using cookies
* Might fix #158 due to using a different authentication method
Should fix new way YTM orders recent history #195
Copy link
Contributor

📦 A new release has been made for this pull request.

To play around with this PR, pull an image:

  • foxxmd/multi-scrobbler:pr-203

Images are available for x86_64 and ARM64.

Latest commit: 34356c6

@FoxxMD
Copy link
Owner Author

FoxxMD commented Oct 28, 2024

The latest pr-203 docker image has the DST fix incorporated.

@xathon
Copy link

xathon commented Oct 31, 2024

Can confirm that there are no duplicate scrobbles the past few days. Only thing that I see is that the timestamps don't appear in the web log, the lines look like this:

Ad Infinitum - Aftermath --- Abyss @ N/A (Today)

@FoxxMD
Copy link
Owner Author

FoxxMD commented Oct 31, 2024

Thanks for the feedback. The missing timestamps are intentional since MS is showing you what info it received from YTM (it doesn't have timestamps).

@FoxxMD FoxxMD merged commit 1706d5c into master Oct 31, 2024
4 checks passed
@FoxxMD FoxxMD deleted the GH-195/ytoauth branch October 31, 2024 12:30
@xathon
Copy link

xathon commented Nov 2, 2024

Hey, sorry, I have to retract my confirmation. I checked the wrong place, since in the web log of multiscrobbler they show up as single, but in Maloja they're duplicated. Excerpt from the log when this happens:

[2024-11-02 17:26:17.027 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Raw Payload:

[2024-11-02 17:26:17.026 +0100] INFO : [App] [Scrobblers] [Maloja - unnamed-mlj] Scrobbled (New) => (YTMusic) Blackbriar - Moonflower (feat. Marjana Semkina) @ 2024-11-02T17:26:01+01:00 (S)

[2024-11-02 17:26:16.011 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Waiting 955ms to scrobble so time passed since previous scrobble is at least 1000ms

[2024-11-02 17:26:15.998 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Raw Payload:

[2024-11-02 17:26:15.997 +0100] INFO : [App] [Scrobblers] [Maloja - unnamed-mlj] Scrobbled (New) => (YTMusic) Dynazty - Heartless Madness @ 2024-11-02T17:26:01+01:00 (S)

[2024-11-02 17:26:15.947 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Found 40 recent scrobbles

[2024-11-02 17:26:15.927 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Refreshing recent scrobbles

[2024-11-02 17:26:15.925 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] [Upstream Refresh] Should refresh => newest queued scrobble playDate is newer than last refresh

[2024-11-02 17:26:15.200 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-11-02T17:26:01+01:00 | Next check interval: 10.00s

[2024-11-02 17:26:15.200 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Discovered => Dynazty - Heartless Madness @ 2024-11-02T17:26:01+01:00 (S)

[2024-11-02 17:26:14.448 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played

[2024-11-02 17:26:14.447 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-11-02T17:26:01+01:00 | Next check interval: 10.00s

[2024-11-02 17:26:14.442 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Discovered => Blackbriar - Moonflower (feat. Marjana Semkina) @ 2024-11-02T17:26:01+01:00 (S)

[2024-11-02 17:26:05.448 +0100] DEBUG : [App] [Sources] [Spotify - unnamed] Last activity 2024-11-02T17:02:49+01:00 is 18 minutes outside of polling period (last activity + 300s) | Next check interval: (10 + 20)s

[2024-11-02 17:26:05.446 +0100] DEBUG : [App] [Sources] [Spotify - unnamed] No new tracks discovered

[2024-11-02 17:26:05.355 +0100] DEBUG : [App] [Sources] [Spotify - unnamed] Refreshing recently played

[2024-11-02 17:26:04.441 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Potential plays were discovered close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first

[2024-11-02 17:26:03.525 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played

[2024-11-02 17:25:55.917 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Raw Payload:

[2024-11-02 17:25:55.915 +0100] INFO : [App] [Scrobblers] [Maloja - unnamed-mlj] Scrobbled (New) => (YTMusic) Dynazty - Heartless Madness @ 2024-11-02T17:25:01+01:00 (S)

[2024-11-02 17:25:55.862 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Found 40 recent scrobbles

[2024-11-02 17:25:55.840 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Refreshing recent scrobbles

[2024-11-02 17:25:55.838 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] [Upstream Refresh] Should refresh => newest queued scrobble playDate is newer than last refresh

[2024-11-02 17:25:54.017 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-11-02T17:25:01+01:00 | Next check interval: 10.00s

[2024-11-02 17:25:54.016 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Discovered => Dynazty - Heartless Madness @ 2024-11-02T17:25:01+01:00 (S)

[2024-11-02 17:25:53.336 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played




Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request safe to test trusted to build image
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New YTMusic Cookie Format? YouTube Music auth fails after 30 minutes
2 participants