Add lazy param to the local cache import #6572
Description
Description
Add a lazy=true option to the local cache importer:
--import-cache type=local,src=/cache,lazy=true
When lazy=true, skip the unlazyProvider wrapping. Layers are imported as blob-only refs and extracted on-demand only when actually needed.
This is safe for ephemeral buildkitd (also called "daemonless") because:
- There is no subsequent build that could reference stale blob-only refs (the original bug from New docker-container builders fail first bake using local cache docker/buildx#1325)
- Each build runs in its own daemon/container and is fully self-contained
- Layers that need to be executed are still extracted — just not eagerly at import time
Related: PR #3493 — introduced unlazyProvider (the root cause for ephemeral setups)
Script that shows the slowdown due to eager "unlazying":
#!/bin/sh
set -eu
CACHE=$(mktemp -d)
CTX=$(mktemp -d)
trap "rm -rf $CACHE $CTX" EXIT
cat > "$CTX/Dockerfile" <<'EOF'
FROM node:20 AS builder
RUN echo "built" > /tmp/out.txt
FROM node:20
COPY --from=builder /tmp/out.txt /tmp/out.txt
COPY . ./
EOF
echo v1 > "$CTX/app.js"
build() {
echo ""
echo "=== $1 ==="
shift
s=$(date +%s)
docker run --rm \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--security-opt systempaths=unconfined \
-v "$CTX":/workspace:ro \
-v "$CACHE":/cache:rw \
--entrypoint buildctl-daemonless.sh \
moby/buildkit:rootless build \
--frontend=dockerfile.v0 --progress=plain \
--local=context=/workspace --local=dockerfile=/workspace \
--export-cache=type=local,dest=/cache,mode=max \
--import-cache=type=local,src=/cache \
"$@" 2>&1
echo "--- $(($(date +%s) - s))s ---"
}
# 1) Populate cache
build "Build 1: cold"
# Simulate a code change — only COPY . ./ is invalidated
echo v2 > "$CTX/app.js"
# 2) Should be fast but isn't — extracts ~350MB of base layers from cache
build "Build 2: code change (expect extracting lines)"
echo "Build 2 should show 'extracting sha256:...' lines for ~350MB of node:20 layers."
The current output is:
#8 [stage-1 2/3] COPY --from=builder /tmp/out.txt /tmp/out.txt
#8 sha256:8578011282ae3fef36307f7e3afb2265a96bada1a57648f07bea9cc1a11e7b7b 48.37MB / 48.37MB 0.1s done
#8 extracting sha256:8578011282ae3fef36307f7e3afb2265a96bada1a57648f07bea9cc1a11e7b7b
#8 extracting sha256:8578011282ae3fef36307f7e3afb2265a96bada1a57648f07bea9cc1a11e7b7b 0.5s done
#8 sha256:443d4217b87aad21c6acb58313c9038eb24571a4e9f81de2463aaf19d403a640 23.60MB / 23.60MB 0.1s done
#8 extracting sha256:443d4217b87aad21c6acb58313c9038eb24571a4e9f81de2463aaf19d403a640
#8 extracting sha256:443d4217b87aad21c6acb58313c9038eb24571a4e9f81de2463aaf19d403a640 0.5s done
#8 sha256:a3d27b842bb73f4af595ce58848c4c53ae713ca5c649636d25b483ca63bccc52 64.48MB / 64.48MB 0.2s
#8 sha256:a3d27b842bb73f4af595ce58848c4c53ae713ca5c649636d25b483ca63bccc52 64.48MB / 64.48MB 0.2s done
#8 extracting sha256:a3d27b842bb73f4af595ce58848c4c53ae713ca5c649636d25b483ca63bccc52
#8 extracting sha256:a3d27b842bb73f4af595ce58848c4c53ae713ca5c649636d25b483ca63bccc52 0.9s done
#8 sha256:869fdd20a7335f51da5a8481862ac636a1e2f1cea7649274e79a5b2a176eaff1 59.77MB / 203.04MB 0.2s
#8 sha256:869fdd20a7335f51da5a8481862ac636a1e2f1cea7649274e79a5b2a176eaff1 121.63MB / 203.04MB 0.3s
#8 sha256:869fdd20a7335f51da5a8481862ac636a1e2f1cea7649274e79a5b2a176eaff1 187.70MB / 203.04MB 0.5s
#8 sha256:869fdd20a7335f51da5a8481862ac636a1e2f1cea7649274e79a5b2a176eaff1 203.04MB / 203.04MB 0.5s done
#8 extracting sha256:869fdd20a7335f51da5a8481862ac636a1e2f1cea7649274e79a5b2a176eaff1
#8 extracting sha256:869fdd20a7335f51da5a8481862ac636a1e2f1cea7649274e79a5b2a176eaff1 2.4s done
#8 sha256:37e99d17a8b8e5b9046298f01713d32ffe79445f4c399b5806d5214dfa7293ea 3.33kB / 3.33kB done
#8 extracting sha256:37e99d17a8b8e5b9046298f01713d32ffe79445f4c399b5806d5214dfa7293ea 0.0s done
#8 sha256:eae2a9ed25334b53f73aeb4626f84e1569b4d1f264e1f2a7c1f19d35fe9c0b0a 48.59MB / 48.59MB 0.1s done
#8 extracting sha256:eae2a9ed25334b53f73aeb4626f84e1569b4d1f264e1f2a7c1f19d35fe9c0b0a
#8 extracting sha256:eae2a9ed25334b53f73aeb4626f84e1569b4d1f264e1f2a7c1f19d35fe9c0b0a 0.8s done
#8 sha256:a1a925eda0879fe64cdd749bfa0bd7e0eb206d4d63dbb6e85f3288042e30a748 1.25MB / 1.25MB done
#8 extracting sha256:a1a925eda0879fe64cdd749bfa0bd7e0eb206d4d63dbb6e85f3288042e30a748 0.0s done
#8 sha256:03607e0e4c940a188cfcafa183bb97b8f97347774f71a34a04bdf8e7c7c2f53e 449B / 449B done
#8 extracting sha256:03607e0e4c940a188cfcafa183bb97b8f97347774f71a34a04bdf8e7c7c2f53e done
#8 sha256:07d07f395b047cb327f0fce502634ae0b3f740e91d08acd1e014af4fc7bd4273 137B / 137B done
#8 extracting sha256:07d07f395b047cb327f0fce502634ae0b3f740e91d08acd1e014af4fc7bd4273 done
#8 CACHED
With lazy=true, it would be:
#8 [stage-1 2/3] COPY --from=builder /tmp/out.txt /tmp/out.txt
#8 CACHED