reduce the impact of cache layer exports on build time

[andrewnitu]

Description

Hello,

While doing some testing of buildkit using the registry cache backend, I observed that quite a bit of time is spent on exporting the cache to the registry. If doing a completely "cold" build, 30 out of 120seconds is spent on this.

1. Currently the image push and the cache push seem to be sequential. I wonder if it makes sense to parallelize these to speed up build times (and also parallelize the cache layer push, but that's mentioned here allow exporting cache layers in parallel to the remote registry #6123)
2. It could make sense for some use cases to return early with the digest once the image is pushed, and then either continue with the cache push in the background or allow the client to perform it separately (this second option was mentioned here Possible to export cache in a separate command? #1938). Of course this would change the semantics of the build command as the cache is no longer guaranteed to be already pushed at build completion, but I think it could make sense as an explicit option.

Pretty new to working with buildkit but that's just what observed so far. Would love to hear your thoughts :)

[tonistiigi]

Currently the image push and the cache push seem to be sequential. I wonder if it makes sense to parallelize these to speed up build times (and also parallelize the cache layer push, but that's mentioned here

This was updated in latest 2 versions: #6455 #6451

2

This would require configuring cache in daemon level somehow probably. First, the errors could get lost and user needs to have something to monitor what the actual state of upload is (eg. if they want to shut down daemon). Second, cache exporting may require client - obviously "local" type cache backend requires it but also the "registry" may need to update auth token from the client.