Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: support video chapters #388

Merged
merged 1 commit into from
Mar 24, 2025

Conversation

arjitcodes
Copy link
Contributor

Fixes #170

Workflow

scrapper

  • yt-dlp writes a metadata file called video.info.json when downloading a video (without an extra yt-dlp call).
  • Extract chapters from that metadata file.
  • Create a WEBVTT file using the chapters.

zimui

  • Check for chaptersPath in video.json.
  • If found, add a new track called chapter with the WEBVTT file.

Chapter in player

Screenshot from 2025-03-12 07-45-37

Copy link

codecov bot commented Mar 12, 2025

Codecov Report

Attention: Patch coverage is 0% with 46 lines in your changes missing coverage. Please review.

Project coverage is 1.19%. Comparing base (2a823f2) to head (f79c75c).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
scraper/src/youtube2zim/scraper.py 0.00% 40 Missing ⚠️
scraper/src/youtube2zim/schemas.py 0.00% 6 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##            main    #388      +/-   ##
========================================
- Coverage   1.24%   1.19%   -0.05%     
========================================
  Files         11      11              
  Lines       1124    1169      +45     
  Branches     164     172       +8     
========================================
  Hits          14      14              
- Misses      1110    1155      +45     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@arjitcodes arjitcodes force-pushed the feature/support-video-chapters branch from 847e779 to a2d330c Compare March 12, 2025 15:43
@arjitcodes
Copy link
Contributor Author

@kelson42

@benoit74 benoit74 self-requested a review March 13, 2025 08:09
Copy link
Collaborator

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A first glance look very promising, see on minor comment.

Other than that, it looks like we do not have chapters in the timeline below the video like on https://www.nuevodevel.com/nuevo/showcase/chapters ; is it linked to any skin (or something like that) ? I find the chapter button particularly hard to find in general, I like the timeline modification to realize there are chapters.

@arjitcodes
Copy link
Contributor Author

Thanks for your feedback! The chapters you see in the example you shared are part of a paid plugin. I've implemented a similar effect in our code to enhance chapter visibility on the timeline. Let me know if you have any additional suggestions!
Screenshot from 2025-03-15 18-18-57

@arjitcodes arjitcodes requested a review from benoit74 March 18, 2025 02:36
@benoit74
Copy link
Collaborator

Just a small ping to inform you that I did not forgot about this PR but the fact that CI is broken forces me to test this a bit on my own machine, and I would like to test it on a real channel with chapters on some videos, and it takes some time.

Copy link
Collaborator

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not working when the video is in S3 cache, because we do not call yt_dlp in scraper.download_video in such a case.

I assume it would be wiser to add the download of "JSON metadata" in the call to yt_dlp of scraper.download_subtitles ; this call to yt_dlp is always done exactly for the purpose that we want to always do it even when video is in S3 cache.

WDYT?

@arjitcodes
Copy link
Contributor Author

Yes, that makes sense.

Copy link
Collaborator

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a bug when you move from a video with chapters (A) to a video without chapters (B) : on video B (and C without chapter, ...), we still have the chapters of video A.

@arjitcodes
Copy link
Contributor Author

I’ve fixed the issue! There were actually two different bugs—one related to timeline modification and another with the Video.js chapter button:

  1. First issue: The addMarker function was returning early when the chapter list was empty, preventing updates. Removing that check fixed the problem.
  2. Second issue: When switching to a new video, we update the video’s src and other options, but (for some reason—maybe because Video.js only adds text tracks during initialization) the text tracks (chapters, subtitles) weren’t updating with the new options.

I first tried using Video.js’s built-in methods like removeTextTrack and addTextTrack, but they didn’t work in our case. Then, I even attempted manually adding tracks via appendChild, but that also didn’t work. So the only solution left (at least for me) was to reinitialize Video.js when switching videos, and that worked!

I actually think reinitializing is a better approach because when we’re not in a playlist and move from one video to another, Video.js already reinitializes. That’s why this issue only occurred when playing videos in a playlist.

Are there any drawbacks that we should consider? WDYT?

@benoit74
Copy link
Collaborator

Video.JS is indeed a purely JS library, and it is known that these libraries tend to have issues around attributes reloading. The cost of reloading it is moderate and it brings the good advantage that we are sure to free more underlying bugs / weird situations, so I'm fine with this approach.

Copy link
Collaborator

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ; please squash your commits into one commit, we do not need implementation history (you can have two commits or more if it adds value to split this, but I feel like one is enough, might be wrong), and I will then merge

@arjitcodes arjitcodes force-pushed the feature/support-video-chapters branch from c30fbae to f79c75c Compare March 24, 2025 14:21
@arjitcodes arjitcodes requested a review from benoit74 March 24, 2025 14:27
Copy link
Collaborator

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@benoit74 benoit74 merged commit fbf8a91 into openzim:main Mar 24, 2025
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for chapters
2 participants