Timestamp calculations break when you trim the start of a clip

[latenitefilms]

Apple Feedback Assistant ID: FB12043900

THE PROBLEM:
Currently there’s no way in FxPlug4 to determine the timecode or frame number of a source clip. For example, if you’re making an Effect/FxFilter, there’s no way to get the timecode or frame number of the media asset that the Effect/FxFilter has been applied to.

WHY THIS MATTERS:
Gyroflow Toolbox is a Mac Store Application that allows you to apply gyroscope-based stabilisation to video footage that contains gyroscope metadata. Currently you should only ever apply the Gyroflow Toolbox effect to an entire clip - the clip cannot be trimmed. Due to limitations in Final Cut Pro's FxPlug4 API - we currently can't determine the source start timecode of a clip. Because of this, the Gyroflow Toolbox effect should only be applied to a clip where the start of the clip hasn't been trimmed in the timeline (i.e. the clip you have in the timeline should show the first frame of the source clip). If you need to trim the start of this clip, you can use the full clip within a Compound Clip, then trim the Compound Clip as required.

WHAT NEEDS TO CHANGE:
Ideally, there should be some new methods added to FxTimingAPI_v4 so that we can access the current timecode of the source clip, but also a frame number for connivence. For example, if we have a ProRes File that starts at 01:00:00:00, if we apply our Effect/FxFilter to that ProRes File within Final Cut Pro, for the first frame of the video, we should be able to use FxPlug4 to return the CMTime equivalent of 01:00:00:00, but also frame 0. This would allow Gyroflow Toolbox to be applied like a normal effect, rather than having to contain it within a container clip, like a Compound Clip.

---

ORIGINAL POST:

The timestamp calculations are incorrect. Doh!

Currently it's determining that the first frame of the clip in the timeline is frame 0 - wrongly assuming that the first frame of the clip in the timeline, is actually the first frame of the source clip (i.e. it doesn't take into account the clip may have a trimmed start in the timeline).

We can get the current source timecode in CMTime from the renderTime. For example, I can get the source timecode value:

...however, I'm not really sure how I can convert this back into a frame count for Gyroflow.

My only thought is maybe we just pass the source timecode across to Gyroflow, and do the timestamp maths in Rust-land, as Gyroflow should know the start timecode?

As far as I can tell, there's no way in FxPlug4 to get anything other than:

...which I don't think is helpful.

I got excited about startTimeOfInputToFilter - but that only returns the timecode of the start of the clip as it appears in the timeline, and we really need the start timecode of the source clip itself.

Understanding Time in FxPlug
https://developer.apple.com/documentation/professional_video_applications/fxplug/understanding_time_in_fxplug?language=objc

[latenitefilms]

@AdrianEddy writes on Discord:

FWIW Gyroflow doesn't know anything about timecode, it just uses microseconds from the video start

My reply:

Ah, right. I’m not sure of the fix then off the top of my head. I can only really get the source start timecode of where the clip starts in the timeline, duration of the clip as it appears in the timeline, the source timecode of current frame that’s being rendered of the clip, and all the timeline timecode. I can’t think of a way to use this information to determine microseconds from the video start. Annoyingly, FxPlug4 doesn’t provide any other information about the source clip AFAIK.

Unless I’m missing something obvious, I think I might have to just leave this as a documented limitation - where users will need to apply Gyroflow Toolbox to the entire clip, and create a compound clip if they want to use a section of that clip, or add something to gyroflow_core or my own Rust code to determine the clips start timecode value.

[latenitefilms]

Apple says:

Regarding the timing question, there is no mechanism for a 3rd party FxPlug to retrieve the actual source TC of an FCP clip. If that could be of use to you, and there is no way around your issue using Workflow Extensions, then it probably also deserves an enhancement request.

[JWuStudio]

That sounds like deserving an enhancement request haha!

[latenitefilms]

FWIW - One potential idea is to just add an "start offset" slider to the Final Cut Pro Inspector, so that you can offset the start by a certain number of frames - but I think this would be more annoying for the end user than just compound clipping the clip first.

[NoSpHieL]

What about selecting clip in browser, go to “clip” menu -> “open clip” and applying it there ?

I don’t know much about dev, but I know in FCP, every clip is already a compound 🧐

[latenitefilms]

That's definitely possible in some cases, yes. You just have to remember that each "clip" is it's own instance, so for example, if you apply the Gyroflow Toolbox effect to a clip in the Browser, then you import that clip into your Timeline, if you adjust the Gyroflow Toolbox settings of the clip in the timeline, they won't affect the Gyroflow Toolbox settings in the Browser. So generally speaking... compound clips are easier.

[Jefke81]

Following