Docker Buildx Does Not Seem To Respect Output Type Flag

[justinba1010]

Discussed in #2458

^{Originally posted by justinba1010 May 14, 2024}
It seems that the output type flag is ignored during exports. Not sure if I am using this incorrectly. It seems to be documented here:
https://docs.docker.com/build/exporters/oci-docker/

Goal: To build a tarball that has the docker layout instead of OCI, in v24 and below this was default OCI Compliance PR

Steps to recreate:

1. Compile a dockerfile build (used the hello world example)

➜  hello-world git:(master) ✗ docker buildx build --output type=docker -t hello .

2. Save a tarball of image

➜  hello-world git:(master) ✗ docker save hello:latest -o out.tar

It would be expected that this tarball would have a docker image layout but it seems to be exported as an OCI image.

➜  hello-world git:(master) ✗ tar -tvf out.tar | grep oci
-rw-r--r-- 0/0              31 1969-12-31 19:00 oci-layout

There seems to be documentation for other methods to skip the save step, but I have been unable to get those to work as well.

➜  hello-world git:(master) ✗ docker buildx build --output "type=docker,tar=true,dest=out.tar" -t hello .
[+] Building 0.0s (0/0)                                                                    docker:default
ERROR: Docker exporter is not supported for the docker driver.
Switch to a different driver, or turn on the containerd image store, and try again.
Learn more at https://docs.docker.com/go/build-exporters/
➜  hello-world git:(master) ✗ docker buildx build --output "type=docker,tar=true" -t hello . 

➜  hello-world git:(master) ✗ docker version
Client: Docker Engine - Community
 Version:           26.1.2
 API version:       1.45
 Go version:        go1.21.10
 Git commit:        211e74b
 Built:             Wed May  8 13:59:59 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.2
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.10
  Git commit:       ef1912d
  Built:            Wed May  8 13:59:59 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0```</div>

[tonistiigi]

Save a tarball of image

If you are using docker save then it is this command that makes the tarball, not the builder.

Docker tar layout can be made compatible with OCI tar. Exporting Docker tarball does not mean that there can not be oci-layout file, but means that this tarball can be imported to (older) Docker engine. The file that defines this importing logic is in manifest.json that is not part of OCI spec.

ERROR: Docker exporter is not supported for the docker driver.

In order to export OCI tarball or Docker-compatible tarball directly from builder you need to use container/k8s/remote builder instance https://docs.docker.com/reference/cli/docker/buildx/create/ or containerd storage in your docker installation https://docs.docker.com/desktop/containerd/

If you want to control if the objects in the image use OCI or Docker mediatypes then the option for that is oci-mediatypes=true|false

[justinba1010]

I think I possibly have the wrong terminology mixed up here then. I have a workflow that is compatible with the layout for out2.tar. On Docker 24.x this layout seems to be the default. With the buildx and Docker 25.x+ it follows the layout for the out.tar, and I am unable to get the old layout. Basically in the meantime I'm trying to have newer versions of Docker output the out2.tar until we have a migration plan for our workflow. It seems this moby/moby#44598 is the breaking change for us, and possibly we'd like to be able to output a non-OCI compatible tarball in the meantime of upgrading our workflows.

Thanks for the speedy response!

➜  hello-world git:(master) ✗ tar -tvf out.tar                                                                                                                                   <aws:vistarprod> <region:us-east-1>
drwxr-xr-x 0/0               0 1969-12-31 19:00 blobs/
drwxr-xr-x 0/0               0 1969-12-31 19:00 blobs/sha256/
-r--r--r-- 0/0             499 1969-12-31 19:00 blobs/sha256/91c75f89cbeb4ec915b3046093aaa5abdbb915d9545b19d3eba18c28dc688226
-r--r--r-- 0/0            2461 1969-12-31 19:00 blobs/sha256/b39614279ade253c62e97143184b5f8f099f25a9a1614fd7cd94a42fe0e1920b
-r--r--r-- 0/0             596 1969-12-31 19:00 blobs/sha256/fc404f7cf74ec7c7f1dbb1c4a37d8d8153bf1268801b2c3d8e8350d0a63f83ad
-rw-r--r-- 0/0             424 1969-12-31 19:00 index.json
-rw-r--r-- 0/0             211 1969-12-31 19:00 manifest.json
-r--r--r-- 0/0              30 1969-12-31 19:00 oci-layout
➜  hello-world git:(master) ✗ docker -v                                                                                                                                          <aws:vistarprod> <region:us-east-1>
Docker version 24.0.9, build 2936816
➜  hello-world git:(master) ✗ docker buildx build --output "type=docker,tar=true,dest=out.tar" --builder=container -t hello .                                                    <aws:vistarprod> <region:us-east-1>
➜  hello-world git:(master) ✗ docker build -t hello .                                                                                                                            <aws:vistarprod> <region:us-east-1>
[+] Building 0.5s (5/5) FINISHED                                                                                                                                                                      docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                            0.0s
 => => transferring dockerfile: 78B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                                                                                 0.0s
 => [internal] load build context                                                                                                                                                                               0.4s
 => => transferring context: 13.29kB                                                                                                                                                                            0.4s
 => CACHED [1/1] COPY hello /                                                                                                                                                                                   0.0s
 => exporting to image                                                                                                                                                                                          0.0s
 => => exporting layers                                                                                                                                                                                         0.0s
 => => writing image sha256:a26569d153fd86d16013795b0fe2b199a36f8a3fc902779bf554ae7b19dbbd9c                                                                                                                    0.0s
 => => naming to docker.io/library/hello                                                                                                                                                                        0.0s
➜  hello-world git:(master) ✗ docker save hello:latest -o out2.tar                                                                                                               <aws:vistarprod> <region:us-east-1>
➜  hello-world git:(master) ✗ tar -tvf out2.tar                                                                                                                                  <aws:vistarprod> <region:us-east-1>
drwxr-xr-x 0/0               0 2024-05-14 11:27 263fdee695753c7f4aaa51bb806d908a2a3ddc7b35ad76c60e0b7df0a40f8ac8/
-rw-r--r-- 0/0               3 2024-05-14 11:27 263fdee695753c7f4aaa51bb806d908a2a3ddc7b35ad76c60e0b7df0a40f8ac8/VERSION
-rw-r--r-- 0/0             802 2024-05-14 11:27 263fdee695753c7f4aaa51bb806d908a2a3ddc7b35ad76c60e0b7df0a40f8ac8/json
-rw-r--r-- 0/0           14848 2024-05-14 11:27 263fdee695753c7f4aaa51bb806d908a2a3ddc7b35ad76c60e0b7df0a40f8ac8/layer.tar
-rw-r--r-- 0/0             626 2024-05-14 11:27 a26569d153fd86d16013795b0fe2b199a36f8a3fc902779bf554ae7b19dbbd9c.json
-rw-r--r-- 0/0             201 1969-12-31 19:00 manifest.json
-rw-r--r-- 0/0              88 1969-12-31 19:00 repositories
➜  hello-world git:(master) ✗ ls                                                                                                                                                 <aws:vistarprod> <region:us-east-1>
Dockerfile  hello  nanoserver-1809  nanoserver-ltsc2022  out2.tar  out.tar

Regarding the oci-mediatypes flag,

➜  hello-world git:(master) ✗ tar -tvf out3.tar                                                                                                                                  <aws:vistarprod> <region:us-east-1>
drwxr-xr-x 0/0               0 1969-12-31 19:00 blobs/
drwxr-xr-x 0/0               0 1969-12-31 19:00 blobs/sha256/
-r--r--r-- 0/0             499 1969-12-31 19:00 blobs/sha256/91c75f89cbeb4ec915b3046093aaa5abdbb915d9545b19d3eba18c28dc688226
-r--r--r-- 0/0            2461 1969-12-31 19:00 blobs/sha256/b39614279ade253c62e97143184b5f8f099f25a9a1614fd7cd94a42fe0e1920b
-r--r--r-- 0/0             596 1969-12-31 19:00 blobs/sha256/fc404f7cf74ec7c7f1dbb1c4a37d8d8153bf1268801b2c3d8e8350d0a63f83ad
-rw-r--r-- 0/0             424 1969-12-31 19:00 index.json
-rw-r--r-- 0/0             211 1969-12-31 19:00 manifest.json
-r--r--r-- 0/0              30 1969-12-31 19:00 oci-layout
➜  hello-world git:(master) ✗ docker buildx build --output "type=docker,tar=true,dest=out4.tar,oci-mediatypes=true" --builder=container -t hello .                               <aws:vistarprod> <region:us-east-1>
[+] Building 0.0s (5/5) FINISHED                                                                                                                                                          docker-container:container
 => [internal] load build definition from Dockerfile                                                                                                                                                            0.0s
 => => transferring dockerfile: 78B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                                                                                 0.0s
 => [internal] load build context                                                                                                                                                                               0.0s
 => => transferring context: 27B                                                                                                                                                                                0.0s
 => CACHED [1/1] COPY hello /                                                                                                                                                                                   0.0s
 => exporting to docker image format                                                                                                                                                                            0.0s
 => => exporting layers                                                                                                                                                                                         0.0s
 => => exporting manifest sha256:11a3721c62414ce19b36d0fded1d17e21d0fc8c3cd32a0da52c4cbe6ed249553                                                                                                               0.0s
 => => exporting config sha256:fc404f7cf74ec7c7f1dbb1c4a37d8d8153bf1268801b2c3d8e8350d0a63f83ad                                                                                                                 0.0s
 => => sending tarball                                                                                                                                                                                          0.0s
➜  hello-world git:(master) ✗ tar -tvf out4.tar                                                                                                                                  <aws:vistarprod> <region:us-east-1>
drwxr-xr-x 0/0               0 1969-12-31 19:00 blobs/
drwxr-xr-x 0/0               0 1969-12-31 19:00 blobs/sha256/
-r--r--r-- 0/0             477 1969-12-31 19:00 blobs/sha256/11a3721c62414ce19b36d0fded1d17e21d0fc8c3cd32a0da52c4cbe6ed249553
-r--r--r-- 0/0            2461 1969-12-31 19:00 blobs/sha256/b39614279ade253c62e97143184b5f8f099f25a9a1614fd7cd94a42fe0e1920b
-r--r--r-- 0/0             596 1969-12-31 19:00 blobs/sha256/fc404f7cf74ec7c7f1dbb1c4a37d8d8153bf1268801b2c3d8e8350d0a63f83ad
-rw-r--r-- 0/0             414 1969-12-31 19:00 index.json
-rw-r--r-- 0/0             211 1969-12-31 19:00 manifest.json
-r--r--r-- 0/0              30 1969-12-31 19:00 oci-layout

[tonistiigi]

There's no breaking change. If you get file with oci-layout from Docker 25+ docker save then that file can be still imported to old versions of Docker (going back to 1.10 from 2015). If you get your tarball from docker save then with what flags you built it doesn't matter for the tar layout.

[justinba1010]

I understand, but I am explicitly trying to get the unnested layout of the older versions. I am having trouble generating a tarball without the blobs folder on Docker v25+.

[tonistiigi]

I am having trouble generating a tarball without the blobs folder on Docker v25+.

That's not supported anymore. There was never any definition that this tarball needs to or is not allowed to contain directory named "blobs". The layout of the files is defined in manifest.json in Docker tarball format.

[justinba1010]

Thank you @tonistiigi