Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Cookies don't seem to work when queuing download via the web frontend? Why are the working on the command line? Are they really passed to yt-dlp? #564

Open
ctschach opened this issue Jan 7, 2025 · 5 comments

Comments

@ctschach
Copy link

ctschach commented Jan 7, 2025

Okay, so I want to download an on demand video from Vimeo. I do have the cookies of my account saved to a txt file and adjusted the YTDL_OPTIONS_FILE file to contain:

"cookiefile": "/cookies/cookies.txt",

But if I queue the download via the web interface, the download fails with "No video formats found!" which is normal when you are not authorised. So here is the log:

metube    | [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
metube    | [debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [65cf46cdd] (pip) API
metube    | [debug] params: {'quiet': True, 'no_color': True, 'paths': {'home': '/inbox', 'temp': '/downloads'}, 'outtmpl': {'default': '%(uploader)s - %(title)s (%(upload_date)s - %(id)s).%(ext)s', 'chapter': '%(title)s - %(section_number)s %(section_title)s.%(ext)s'}, 'format': 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', 'socket_timeout': 30, 'ignore_no_formats_error': True, 'progress_hooks': [<function Download._download.<locals>.put_status at 0x7efeacd5ee80>], 'postprocessor_hooks': [<function Download._download.<locals>.put_status_postprocessor at 0x7efeacd5eca0>], '**cookiefile': '/cookies/cookies.txt',** 'verbose': True, 'updatetime': False, 'write-description': True, 'writethumbnail': False, 'writeinfojson': True, 'nocleaninfojson': False, 'embedthumbnail': True, 'embedmetadata': True, 'embed-thumbnail': True, 'embed-metadata': True, 'embed-chapters': True, 'writesubtitles': True, 'subtitleslangs': ['all', '-live_chat'], 'exec': 'cp -p %(filepath)q %(filepath)q.h264.mp4', 'postprocessors': [{'key': 'Exec', 'exec_cmd': 'sh /savecopy.sh {}', 'when': 'post_process'}, {'key': 'FFmpegEmbedSubtitle', 'already_have_subtitle': False}, {'key': 'FFmpegMetadata', 'add_chapters': True}, {'key': 'FFmpegCopyStream'}], 'postprocessor_args': {'offmerger': ['-metadata:s', 'title=', '-metadata:s', 'handler_name='], 'copystream': ['-vaapi_device', '/dev/dri/renderD128', '-vf', 'format=nv12,hwupload', '-c:v', 'hevc_vaapi', '-crf', '20', '-c:a', 'copy']}, 'color': 'no_color', 'compat_opts': set(), 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-us,en;q=0.5', 'Sec-Fetch-Mode': 'navigate'}}
metube    | [debug] Python 3.11.11 (CPython x86_64 64bit) - Linux-5.15.0-130-generic-x86_64-with (OpenSSL 3.3.2 3 Sep 2024)
metube    | [debug] exe versions: ffmpeg 6.1.2 (setts), ffprobe 6.1.2
metube    | [debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, mutagen-1.47.0, requests-2.32.3, sqlite3-3.47.1, urllib3-2.2.3, websockets-13.1
metube    | [debug] Proxy map: {}
metube    | [debug] Request Handlers: urllib, requests, websockets
metube    | [debug] Loaded 1837 extractors
metube    | [vimeo:ondemand] Extracting URL: https://vimeo.com/ondemand/reise/377045885 
metube    | [vimeo:ondemand] 377045885: Downloading webpage
metube    | WARNING: [vimeo:ondemand] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see  https://github.com/yt-dlp/yt-dlp#impersonation  for information on installing the required dependencies
metube    | [vimeo:ondemand] 377045885: Downloading JSON metadata
metube    | [vimeo:ondemand] 377045885: Loading download config JSON
metube    | [vimeo:ondemand] 377045885: Downloading jwt token
metube    | [vimeo:ondemand] 377045885: Downloading API JSON
metube    | [vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic m3u8 information
metube    | [vimeo:ondemand] 377045885: Downloading fastly_skyfire m3u8 information
metube    | [vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic MPD information
metube    | WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
metube    | [vimeo:ondemand] 377045885: Downloading fastly_skyfire MPD information
metube    | WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
metube    | [debug] Sort order given by extractor: quality, res, fps, hdr:12, source
metube    | [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec, channels, acodec, size, br, asr, proto, vext, aext, hasaud, id
metube    | WARNING: Requested format is not available
metube    | [info] There are no subtitles for the requested languages
metube    | [info] Writing video metadata as JSON to: /inbox/Reise - REISE - Auf  Rädern  (20200129 - 377045885).info.json
**metube    | ERROR: [vimeo:ondemand] 377045885: No video formats found!**
metube    | Traceback (most recent call last):
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1634, in wrapper
metube    |     return func(self, *args, **kwargs)
metube    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1790, in __extract_info
metube    |     return self.process_ie_result(ie_result, download, extra_info)
metube    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1849, in process_ie_result
metube    |     ie_result = self.process_video_result(ie_result, download=download)
metube    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3021, in process_video_result
metube    |     self.process_info(new_info)
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 177, in wrapper
metube    |     return func(self, *args, **kwargs)
metube    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3489, in process_info
metube    |     success, real_download = self.dl(temp_filename, info_dict)
metube    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3177, in dl
metube    |     self.raise_no_formats(info, True)
metube    |   File "/usr/local/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1123, in raise_no_formats
metube    |     raise ExtractorError(msg, video_id=info['id'], ie=info['extractor'],
metube    | yt_dlp.utils.ExtractorError: [vimeo:ondemand] 377045885: No video formats found!
metube    |
metube    | INFO:ytdl:waiting for item to download

I first thought, the cookies are expired. However, I did another test and logged into the container and running yt-dlp from the shell. Which works completely fine and download the video.

root@media:/opt/metube# docker exec -u 1000 -ti metube sh
/app $ cd /inbox/
/inbox $ yt-dlp --cookies /cookies/cookies.txt https://vimeo.com/ondemand/reise/377045885
[vimeo:ondemand] Extracting URL: https://vimeo.com/ondemand/reise/377045885
[vimeo:ondemand] 377045885: Downloading webpage
WARNING: [vimeo:ondemand] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see  https://github.com/yt-dlp/yt-dlp#impersonation  for information on installing the required dependencies
[vimeo:ondemand] 377045885: Downloading JSON metadata
[vimeo:ondemand] 377045885: Loading download config JSON
[vimeo:ondemand] 377045885: Downloading jwt token
[vimeo:ondemand] 377045885: Downloading API JSON
[vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic m3u8 information
[vimeo:ondemand] 377045885: Downloading fastly_skyfire m3u8 information
[vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic MPD information
WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[vimeo:ondemand] 377045885: Downloading fastly_skyfire MPD information
WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[info] 377045885: Downloading 1 format(s): hls-fastly_skyfire-5530+hls-fastly_skyfire-audio-high-Original
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 976
[download] Destination: Reise - REISE - Auf  Rädern  [20200129 - 377045885].fhls-fastly_skyfire-5530.mp4
[download]   1.3% of ~   3.82GiB at    8.36MiB/s ETA 07:30 (frag 13/976)^C
ERROR: Interrupted by user

Soooo, the big question: So why is this working on the command line? How can I make this work? Is the app really passing the cookie file to yt-dlp when downloading a video?

@ctschach ctschach changed the title [Bug] Cookies don't seem to work when queuing download via the web frontend? Are they really passed to yt-dlp? [Bug] Cookies don't seem to work when queuing download via the web frontend? Are they really passed to yt-dlp? Any why are the working on the command line? Jan 7, 2025
@ctschach ctschach changed the title [Bug] Cookies don't seem to work when queuing download via the web frontend? Are they really passed to yt-dlp? Any why are the working on the command line? [Bug] Cookies don't seem to work when queuing download via the web frontend? Why are the working on the command line? Are they really passed to yt-dlp? Jan 7, 2025
@PikuZheng
Copy link
Contributor

PikuZheng commented Jan 8, 2025

debug log
INFO:ytdl:adding https://vimeo.com/ondemand/reise/377045885: quality='best' format='mp4' already=None folder=None custom_name_prefix='' playlist_strict_mode=False playlist_item_limit=0
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp-nightly-builds [0b6b7742c] API
[debug] params: {'quiet': True, 'no_color': True, 'extract_flat': True, 'ignore_no_formats_error': True, 'noplaylist': False, 'paths': {'home': '/downloads', 'temp': '/tmp'}, 'cookiefile': '/downloads/.metube/cookies.txt', 'verbose': True, 'impersonate': ImpersonateTarget(client='chrome', version='110', os='windows', os_version='10'), 'format_sort': ['codec:avc:m4a'], 'overwrites': True, 'outtmpl': {'default': '[%(upload_date)s]%(title).50s %(uploader_id)s(%(vcodec).4s+%(acodec).4s)[%(id)s].%(ext)s', 'chapter': '%(title)s - %(section_number)03d %(section_title)s [%(id)s].%(ext)s'}, 'ignoreerrors': True, 'writethumbnail': True, 'postprocessors': [{'key': 'FFmpegMetadata', 'add_metadata': True, 'add_chapters': True, 'add_infojson': 'if_exists'}, {'key': 'EmbedThumbnail'}, {'key': 'Exec', 'exec_cmd': 'sh /downloads/execafterdownload.sh', 'when': 'after_move'}], 'color': 'no_color', 'compat_opts': set(), 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.54 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-us,en;q=0.5', 'Sec-Fetch-Mode': 'navigate'}}
[debug] Python 3.11.10 (CPython x86_64 64bit) - Linux-6.6.22-1-virt-x86_64-with (OpenSSL 3.3.2 3 Sep 2024)
[debug] exe versions: ffmpeg 7.1 (setts), ffprobe 7.1
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.45.3, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1837 extractors
[vimeo:ondemand] Extracting URL: https://vimeo.com/ondemand/reise/377045885
[vimeo:ondemand] 377045885: Downloading webpage
[vimeo:ondemand] 377045885: Downloading JSON metadata
[vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic m3u8 information
[vimeo:ondemand] 377045885: Downloading fastly_skyfire m3u8 information
[vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic MPD information
WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[vimeo:ondemand] 377045885: Downloading fastly_skyfire MPD information
WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[debug] Sort order given by user: codec:avc:m4a
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source
[debug] Formats sorted by: hasvid, ie_pref, vcodec:avc(7), acodec:m4a(9), quality, res, fps, hdr:12(7), source, lang, channels, size, br, asr, proto, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
DEBUG:ytdl:Processing as a video
INFO:ytdl:downloading Die wundersame Reise der unnützen Dinge
INFO:aiohttp.access:11.20.3.12 [08/Jan/2025:05:03:33 +0000] "POST /ytdl/add HTTP/1.1" 200 267 "http://11.20.3.171:8090/ytdl/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp-nightly-builds [0b6b7742c] API
[debug] params: {'quiet': True, 'no_color': True, 'paths': {'home': '/downloads', 'temp': '/tmp'}, 'outtmpl': {'default': '[%(upload_date)s]%(title).50s %(uploader_id)s(%(vcodec).4s+%(acodec).4s)[%(id)s].%(ext)s', 'chapter': '%(title)s - %(section_number)03d %(section_title)s [%(id)s].%(ext)s'}, 'format': 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]', 'socket_timeout': 30, 'ignore_no_formats_error': True, 'progress_hooks': [<function Download._download.<locals>.put_status at 0x7fa0cc72f880>], 'postprocessor_hooks': [<function Download._download.<locals>.put_status_postprocessor at 0x7fa0cc72fc40>], 'cookiefile': '/downloads/.metube/cookies.txt', 'verbose': True, 'impersonate': ImpersonateTarget(client='chrome', version='110', os='windows', os_version='10'), 'format_sort': ['codec:avc:m4a'], 'overwrites': True, 'ignoreerrors': True, 'writethumbnail': True, 'postprocessors': [{'key': 'FFmpegMetadata', 'add_metadata': True, 'add_chapters': True, 'add_infojson': 'if_exists'}, {'key': 'EmbedThumbnail'}, {'key': 'Exec', 'exec_cmd': 'sh /downloads/execafterdownload.sh', 'when': 'after_move'}], 'color': 'no_color', 'compat_opts': set(), 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.54 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-us,en;q=0.5', 'Sec-Fetch-Mode': 'navigate'}}
[debug] Python 3.11.10 (CPython x86_64 64bit) - Linux-6.6.22-1-virt-x86_64-with (OpenSSL 3.3.2 3 Sep 2024)
[debug] exe versions: ffmpeg 7.1 (setts), ffprobe 7.1
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.45.3, urllib3-2.2.3, websockets-13.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1837 extractors
[vimeo:ondemand] Extracting URL: https://vimeo.com/ondemand/reise/377045885
[vimeo:ondemand] 377045885: Downloading webpage
[vimeo:ondemand] 377045885: Downloading JSON metadata
[vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic m3u8 information
[vimeo:ondemand] 377045885: Downloading fastly_skyfire m3u8 information
[vimeo:ondemand] 377045885: Downloading akfire_interconnect_quic MPD information
WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[vimeo:ondemand] 377045885: Downloading fastly_skyfire MPD information
WARNING: [vimeo:ondemand] 377045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[debug] Sort order given by user: codec:avc:m4a
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source
[debug] Formats sorted by: hasvid, ie_pref, vcodec:avc(7), acodec:m4a(9), quality, res, fps, hdr:12(7), source, lang, channels, size, br, asr, proto, vext, aext, hasaud, id
[info] 137947962: Downloading 1 format(s): http-1080p
[info] There are no video thumbnails to download
[debug] Invoking http downloader on "https://vod-progressive-ak.vimeocdn.com/exp=1736316226~acl=%2Fvimeo-transcode-storage-prod-us-west1-h264-1080p%2F01%2F2589%2F5%2F137947962%2F410208843.mp4~hmac=79c36ee9dcf50d853576569ea5ecc09bafe216e44058f632277724a132907d61/vimeo-transcode-storage-prod-us-west1-h264-1080p/01/2589/5/137947962/410208843.mp4"
[download] Destination: /tmp/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4
[Metadata] Adding metadata to "/tmp/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:/tmp/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4' -map 0 -dn -ignore_unknown -c copy -write_id3v1 1 -metadata 'title=Die wundersame Reise der unnützen Dinge' -metadata date=20150901 -metadata 'description=Ein Film von Katja Schupp und Hartmut Seifert – 90 und 60 Minuten, HD, Redaktion Martin Pieper, ZDF/ARTE
 Der Film thematisiert Konstanten unseres modernen Alltags, die sich gerade im Spiegel der aktuellen wirtschaftlichen Entwicklung als trügerisch erweisen: materieller Wert und dessen Vergänglichkeit, Wohlstand und dessen Relativität – was für den einen Müll ist, ist für den anderen ein (kleiner) Schatz, wobei der Wert eines Gegenstandes immer mehr zunimmt, je weiter er sich von unserer zentristischen Wohlstandsgesellschaft entfernt.
Mit einem Augenzwinkern betrachtet betrachten die Filmemacher alte Klischees der Deutschen gegenüber den Polen. Die sympathischen polnischen Protagonisten, gerade als Müllsammler, sind immer wieder mit diesen Vorurteilen konfrontiert und demaskieren sie mal humorvoll und hintersinnig, mal auf melancholische oder gar tragische Art.' -metadata 'synopsis=Ein Film von Katja Schupp und Hartmut Seifert – 90 und 60 Minuten, HD, Redaktion Martin Pieper, ZDF/ARTE
 Der Film thematisiert Konstanten unseres modernen Alltags, die sich gerade im Spiegel der aktuellen wirtschaftlichen Entwicklung als trügerisch erweisen: materieller Wert und dessen Vergänglichkeit, Wohlstand und dessen Relativität – was für den einen Müll ist, ist für den anderen ein (kleiner) Schatz, wobei der Wert eines Gegenstandes immer mehr zunimmt, je weiter er sich von unserer zentristischen Wohlstandsgesellschaft entfernt.
Mit einem Augenzwinkern betrachtet betrachten die Filmemacher alte Klischees der Deutschen gegenüber den Polen. Die sympathischen polnischen Protagonisten, gerade als Müllsammler, sind immer wieder mit diesen Vorurteilen konfrontiert und demaskieren sie mal humorvoll und hintersinnig, mal auf melancholische oder gar tragische Art.' -metadata purl=https://vimeo.com/ondemand/reise/377045885 -metadata comment=https://vimeo.com/ondemand/reise/377045885 -metadata 'artist=Lichtfilm ' -movflags +faststart 'file:/tmp/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].temp.mp4'
[EmbedThumbnail] There aren't any thumbnails to embed
[MoveFiles] Moving file "/tmp/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4" to "/downloads/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4"
[Exec] Executing command: sh /downloads/execafterdownload.sh '/downloads/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4'
YOU ARE EXECUTING EXECAFTERDOWNLOAD.SH NOW!!!
file downloaded is/downloads/[20150901]Die wundersame Reise der unnützen Dinge user13924219(NA+NA)[137947962].mp4
INFO:ytdl:waiting for item to download

image

I can download this video with webui but cookies. I think it has nothing to do with cookies and should check for network (connectivity, dns, ipv6?) issues

@ctschach
Copy link
Author

ctschach commented Jan 8, 2025

So I See that you have set impersonate in your options file. I don't have this and running

/app # yt-dlp --list-impersonate-targets
[info] Available impersonate targets
Client   OS   Source
---------------------------------------
Chrome   -    curl_cffi (not available)
Edge     -    curl_cffi (not available)
Safari   -    curl_cffi (not available)

inside the container shows me, that no targets are available. So I can't use impersonate targets.

I know both commands above complains:

WARNING: [vimeo:ondemand] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see https://github.com/yt-dlp/yt-dlp#impersonation for information on installing the required dependencies

BUT, this is running with yt-dlp on the command line inside of the container. So the missing impersonate targets should not be an issue. I also don't think there are any networking issues as everything works fine (e.g. download other freely available videos from Vimeo).

If you want to retest it, please use the below URL for download. That was my actual problematic movie, the other one was just from some additional testing (which also didn't work, but which I also didn't purchased):

https://vimeo.com/ondemand/heimreise/388045885

@PikuZheng
Copy link
Contributor

https://vimeo.com/ondemand/heimreise/388045885

can you test it with yt-dlp command line?

@ctschach
Copy link
Author

ctschach commented Jan 8, 2025

Okay, interesting...

If I just use "yt-dlp https://vimeo.com/ondemand/heimreise/388045885" without any additional parameters (so no cookies):

/inbox # yt-dlp https://vimeo.com/ondemand/heimreise/388045885
[vimeo:ondemand] Extracting URL: https://vimeo.com/ondemand/heimreise/388045885
[vimeo:ondemand] 388045885: Downloading webpage
[vimeo:ondemand] 388045885: Downloading JSON metadata
[vimeo:ondemand] 388045885: Downloading akfire_interconnect_quic m3u8 information
[vimeo:ondemand] 388045885: Downloading fastly_skyfire m3u8 information
[vimeo:ondemand] 388045885: Downloading akfire_interconnect_quic MPD information
WARNING: [vimeo:ondemand] 388045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[vimeo:ondemand] 388045885: Downloading fastly_skyfire MPD information
WARNING: [vimeo:ondemand] 388045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[info] 388252447: Downloading 1 format(s): hls-fastly_skyfire-4139+hls-fastly_skyfire-audio-high-Original
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 10
[download] Destination: HEIMREISE - Auf acht Rädern durch Deutschland [388252447].fhls-fastly_skyfire-4139.mp4
[download] 100% of   28.60MiB in 00:00:04 at 5.89MiB/s
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 10
[download] Destination: HEIMREISE - Auf acht Rädern durch Deutschland [388252447].fhls-fastly_skyfire-audio-high-Original.mp4
**[download] 100% of    1.88MiB in 00:00:02 at 684.02KiB/s**
[Merger] Merging formats into "HEIMREISE - Auf acht Rädern durch Deutschland [388252447].mp4"
Deleting original file HEIMREISE - Auf acht Rädern durch Deutschland [388252447].fhls-fastly_skyfire-audio-high-Original.mp4 (pass -k to keep)
Deleting original file HEIMREISE - Auf acht Rädern durch Deutschland [388252447].fhls-fastly_skyfire-4139.mp4 (pass -k to keep)

It is just downloading 1.8MB and the resulting video is the Trailer of the on-demand video.

However, if I provide the cookie file which ontains my login-session - "yt-dlp --cookies /cookies/cookies.txt https://vimeo.com/ondemand/heimreise/388045885". The same link will download the full video (see the downloaded size):

/inbox # yt-dlp --cookies /cookies/cookies.txt https://vimeo.com/ondemand/heimreise/388045885
[vimeo:ondemand] Extracting URL: https://vimeo.com/ondemand/heimreise/388045885
[vimeo:ondemand] 388045885: Downloading webpage
[vimeo:ondemand] 388045885: Downloading JSON metadata
[vimeo:ondemand] 388045885: Loading download config JSON
[vimeo:ondemand] 388045885: Downloading jwt token
[vimeo:ondemand] 388045885: Downloading API JSON
[vimeo:ondemand] 388045885: Downloading akfire_interconnect_quic m3u8 information
[vimeo:ondemand] 388045885: Downloading fastly_skyfire m3u8 information
[vimeo:ondemand] 388045885: Downloading akfire_interconnect_quic MPD information
WARNING: [vimeo:ondemand] 388045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[vimeo:ondemand] 388045885: Downloading fastly_skyfire MPD information
WARNING: [vimeo:ondemand] 388045885: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[info] 388045885: Downloading 1 format(s): hls-fastly_skyfire-5530+hls-fastly_skyfire-audio-high-Original
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 976
[download] Destination: HEIMREISE - Auf acht Rädern durch Deutschland [388045885].fhls-fastly_skyfire-5530.mp4
**[download] 100% of    3.74GiB in 00:17:48 at 3.58MiB/s**
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 976
[download] Destination: HEIMREISE - Auf acht Rädern durch Deutschland [388045885].fhls-fastly_skyfire-audio-high-Original.mp4
[download] 100% of  186.82MiB in 00:03:05 at 1.01MiB/s
[Merger] Merging formats into "HEIMREISE - Auf acht Rädern durch Deutschland [388045885].mp4"
Deleting original file HEIMREISE - Auf acht Rädern durch Deutschland [388045885].fhls-fastly_skyfire-5530.mp4 (pass -k to keep)
Deleting original file HEIMREISE - Auf acht Rädern durch Deutschland [388045885].fhls-fastly_skyfire-audio-high-Original.mp4 (pass -k to keep)

So it definitely makes a difference and take the cookie data which is not happening when I queue the link via the web frontend.

But I overall I spent some more time on investigating the issues. From your log file I could see that you have provided the impersonate option. So while I was able to install the "curl_cffi" on top of the docker container, I'm having trouble to provide the correct impersonate parameters. So could you please let me know how you have defined them in your YTLD_OPTIONS? Maybe this helps to solve the issue.

@ctschach
Copy link
Author

ctschach commented Jan 8, 2025

Okay, I found a quick work-around:

I simply added the Vimeo username and password to my YT_DLP_OPTIONS file.

I know, this is totally insecure, but due to the fact that I am the only one having access to this server, I can deal with it. I still think there is something broken, but as I seems to be the only one having this issue, I don't want to waste anybody's time anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants