[FEATURE] Support for VSE multithread rendering speedup

[rabits]

Is your feature request related to a problem? Please describe.

Video Sequencer Engine of blender is singlethreaded and it's a pain! But there is a way to actually try to use BlendNet to seedup the process.

requestor: @ErwanMAS

Describe the solution you'd like

For now I have just a simple POC shell script, that actually uses prerendered audio (because otherwise it makes glitches between the parts of the video):

#!/bin/sh -e
# Script to multithread render VSE
# - Start with small number of THREADS, otherwise could be really hungry for resources
# - Render the audio as project.flac manually (menu->Render->Render Audio...)

PROJECT=project.blend
THREADS=10

# Disable the current config to speed-up the process
export BLENDER_USER_CONFIG=/tmp

data=$(blender -b $PROJECT --python-expr 'import bpy; print("DATA CI:", bpy.context.scene.frame_start, bpy.context.scene.frame_end)' | grep '^DATA CI: ')
start=$(echo "${data}" | cut -d" " -f 3)
end=$(echo "${data}" | cut -d" " -f 4)

delta=$((($end-$start+1)/$THREADS))
end_frames=`echo $(seq $delta $delta $end) | rev | cut -d" " -f 2- | rev`

start_frame=$start

echo "Processing $start-$end by $delta"
rm -f render.concat.ffmpeg.lst
outdir=out/render_parts
rm -rf "$outdir"
mkdir -p $outdir

# Render video per with blocks
for end_frame in ${end_frames} ${end}; do
    echo "  run render thread  $start_frame-$end_frame"
    filename="$(printf "%09d" $start_frame)-$(printf "%09d" $end_frame).mp4"
    echo "file '$outdir/$filename'" >> render.concat.ffmpeg.lst
    blender -noaudio -b $PROJECT -s $start_frame -e $end_frame -o "$outdir/$filename" -a &
    start_frame=$(($end_frame+1))
done

# Render the audio separately to not suffer on concat pauses
# bps.ops.sound.mixdown()

wait
echo "Render completed, begin merge"

ffmpeg -f concat -safe 0 -i render.concat.ffmpeg.lst -y -c copy $outdir/out.mp4
# Merge audio
ffmpeg -i $outdir/out.mp4 -i project.flac -c:v copy -c:a aac -b:a 192k -map 0:v:0 -map 1:a:0 release.mp4
echo "Merge completed"

rm -f render.concat.ffmpeg.lst

The issue here is exactly bps.ops.sound.mixdown() - because seems like it could be started only in UI...

requires: #90

[ErwanMAS]

I discover this script :
https://github.com/elmopl/ktba/blob/master/scripts/addons/parallel_render.py

I tested it , this was enough for my small project but not my big project .

The workflow is to split the sequence in 14 parts ( this can be changed ) , must be a least numberoftotalcore x 2 or x3 .

[ErwanMAS]

From https://github.com/elmopl/ktba/wiki/Addons

You might be thinking that if you have N cores and want to utilise all of them then you only want to split your sequence into N chunks (one for each instance of Blender). However having more chunks can have few advantages:
If one section of video is particularly CPU intensive all instances of Blender might finish long time before the one that was unlucky to get the complex part. Having multiple chunks for that sections dynamically allocates new section to idling Blender instances.

[rabits]

Right, that's another issue - the parts with no blur was rendered quite fast...
And usually in VSE we're using huge files (video), means there should be some way to upload it and distribute properly. BlendNet right now uses not so effective (for huge files) way to distribute the resources across the Agents, so this could require to build the complicated system...