Video previews often don’t display for large files even when navigating away and returning

[jhdouglas]

Per the test plan, there is a known issue where the preview does not self-update when the video is done processing and users must go back to the list and then open the transcript for it to display. Currently, however, it appears that the preview does not display for larger video files even when navigating away and returning. It remains “Video is being processed” indefinitely.

File sizes where this occurred thus far:
193MB
392MB

1. Open autoEdit 2 app
2. Upload large video file
3. Allow processing to complete
4. Open transcript
5. Observe video preview is still processing
6. Navigate away and return
7. Note preview processes indefinitely

The file in the below example was uploaded on Friday and still shows as processing:

[pietrop]

I think the issue there is that for large files it will just take a very long time to process (eg several hours).

A couple of options

1.There are ways to make webmtranscoding faster

ffmpeg -i input -vcodec libvpx -cpu-used -5 -deadline realtime out.webm

Might also be worth looking into what are the parameters that effect transcoding speed in ffmpeg.

2. give progress update

ffmpeg fluent exampel code for printing out progress, maybe with callback this can be used to make a more meaningful progress bar of transcoding of webm?

 .on('progress', function(progress) {
  //  progress      
 //{"frames":null,"currentFps":null,"currentKbps":256,"targetSize":204871,"timemark":"01:49:15.90"}
 console.log('Processing: ' + progress.timemark + ' done ' + progress.targetSize+' kilobytes');
})

this however does not solves the problem that it will take a long time,and if you close the app it will interrupt the process and not resume when you open it again.

Handbrake vs ffmpeg

Another option is to use handbrake instead of ffmpeg and see if that's faster.

Change to ogg

Change to ogg and see is if it's faster.

Web Chimera / VLC

webchimera uses VLC to play any codecs, and can used with NWJS. Also on npm. Demo repo

wcjs-player uses WebChimera.js to draw the frames of libVLC to a canvas.

A possible solution is use WebChimera to show the original media, while the preview is being processed. Downside is that not linking to the original media allows more flexibility and decoupling. Reminiscent of offline/online traditional video editing workflow.

However this introduces using canvas to play the video. Meaning all the HTML5 video api functionality to keep text and video in sync would need to be re-thought.

---

A few considerations

A couple of constraints to keep in mind when thinking about this:

• The input video file can sometime exceed 60 or 70 gig for certain use cases. The app using NWJS file system API takes in the file path to the media as in input, and uses that to convert it to audio and video preview.
• NWJS despited using chromium v8 engine is out of the box constrained on open source video codecs (ogg, webm etc..) it could be modified to handle other formats such as mp4 but that requires also looking into the licence implications.
• User testing has shown that for the majority of the cases the audio preview is sufficient

[pietrop]

Now that autoEdit has transitioned to electron, could revisit adding mp4 support to see if it improves transcoding speed.

[pietrop]

Changed to faster transcoding using mp4 for preview, needs testing difference

[jondrometa86]

@pietrop - I know it's been a while since this issue has come up. I'm also experiencing the "Video is being processed" error.

Same situation is occurring as the issue described in this ticket.
1. Open autoEdit 2 app
2. Upload large video files
3. Allow processing to complete
4. Open transcript
5. Observe video preview is still processing
6. Navigate away and return
7. Note preview processes indefinitely

Transcriptions were processed through Speechmatics successfully about 3 days ago (Dec. 3rd, 2018)

The difference was my file sizes varied (as listed below):

• 785MB
• 136MB
• 89MB
• 51MB
• 55.6MB
• 18.8MB

I'm assuming the first video in this batch is what triggered the "Video is being processed" for all the rest of the videos in this batch. Maybe.

I've tried to replace the .mp4 file generated in autoEdit's system resources folder, thinking this may allow the app to read the video file, but no luck. (replaced in: /Users/[username]/Library/Application Support/autoEdit2/media)

Any thoughts on how to help autoEdit generate these video previews? Or maybe where to place them manually in the app's resources?

[pietrop]

Hi @jondrometa86 ,
Thanks for flagging this, was wondering if this was still an issue or if by changing to mp4 conversion (used to be webm) the problem had been solved. - happy to re-open if you don't mind doing some digging.

Where to look to improve transcoding speed of video preview

Any thoughts on how to help autoEdit generate these video previews?

The module responsible for the conversion is this one you could try and play around with it in isolation and see if it takes long to convert your file.

You can see an example usage here note that the ffmpeg binary attribute is optional if you have ffmpeg installed on your mac it will pick up on that one.

If you want to use the same ffmpeg used by autoEdit, you can use this module ffmpeg-static-electron - see README for more info.

Manually change file path for a stuck video

Or maybe where to place them manually in the app's resources?

If you open the inspector console in autoEdit and type appTranscription.transcriptionsList.models[yourTranscriptIndexNumber].attributes

Where yourTranscriptIndexNumber is the index of your transcript counting from top down in the the transcript list page in autoEdit. if that makes sense?

You should see something like this if you expand the triangle.

if processedVideo is set to false even changing the media in the system folder won't be picked up by autoEdit.

so the trick here is that once you f you replace the file in /Users/[username]/Library/Application Support/autoEdit2/media with one with the same name, and where the file format is playable by HTML5 media.

then you can do appTranscription.transcriptionsList.models[4].set({processedVideo: true}) to change processedVideo to true, now if you re-start the app, it should pick up the new file.

Let me know if it's not clear or it doesn't work.

Question

Are you able to say a bit more about the file you are ingesting? what file format? file size? is it directly from a camera or is being transcoded etc.. this might shed some light on edge cases

[jondrometa86]

@pietrop - Thanks for the quick reply and potential solutions. I'll dive deep tomorrow and see if I can troubleshoot using your methods.

To your question:

The files I'm ingesting are h.264 proxy file, transcoded using a custom preset in Adobe Media Encoder.

File resolution: a weird 854 x480 (massive compression from the original 4K file).
Codecs used: AAC, h.264
Color profile: SD (6-1-6)
Durations: (vary between 6min - 75min)
Audio Channels: stereo

Perhaps the "854" dimension is the issue? I'll do some more testing on my end and report back.

EDIT: Before this batch, I had tried working with autoEdit2 using similar interviews with this exact transcode preset. The difference was the length of the interviews (much shorter, thus smaller files), which tells me this might be a file size issue.

[pietrop]

What file size do you get from these transcoded files?

[jondrometa86]

Here are transcoded file sizes in the order they were uploaded to Speechmatics via autoEdit2:

• 89MB
• 51MB
• 55.6MB
• 18.8MB
• 136MB
• 785MB

Looking back now, it appears I processed the largest file last. The problem must have begun on the first .mp4 file (89MBs)

[pietrop]

ah, sorry, now I noticed you had already put the file sizes in the previous comment.

Ok, the file size should not be an issue then. 🤞

I am curious tho, if you go into /Users/[username]/Library/Application Support/autoEdit2/media could you check the file size of the corresponding transcoded mp4, for those 6 files and whether they play?

[jondrometa86]

@pietrop,
Update on this issue:
I attempted the 2nd option in your troubleshoot list with a few of the interview files...

replace the file in /Users/[username]/Library/Application Support/autoEdit2/media with one with the same name, and where the file format is playable by HTML5 media.

then you can do appTranscription.transcriptionsList.models[4].set({processedVideo: true}) to change processedVideo to true, now if you re-start the app, it should pick up the new file.

After replacing the clip and then setting the processVideo to true, I would verify by closing and relaunching the app, only to find the video still "being processed"

I would then check on the video's status again by using its proper index appTranscription.transcriptionsList.models[yourTranscriptIndexNumber].attributes, and found it was set to false again.

I attempted replacing the file in ../autoEdit2/media with different resolution sizes and compression formats (all h.264 codecs) but still no luck.

I suppose I'll need to dive in to the processing module you mentioned and see what can be done from there.

Regarding your last question:

I am curious tho, if you go into /Users/[username]/Library/Application Support/autoEdit2/media could you check the file size of the corresponding transcoded mp4, for those 6 files and whether they play?

The file size for each was 0MB, so maybe it was an error during the initial transcoding process? (never occurred, left hanging for eternity.)

For now, I've been using the audio and transcript in autoEdit2 to work with these interviews. I'll export an EDL to Premiere Pro and reconnect them to complete the edit.
But I'd definitely like to figure this out going forward in case anyone else has this issue.