Docker Diff-Based Layers

A way to work around the fact that subsequent COPY . /app commands re-adds all files in every layer instead of only the files that have changed.

Use these tools to build an image that includes a layer with only the changed files in comparison with an already pushed image/tag.

Caveats

Your images needs to have rsync installed.

Example

Create and build the image for a sample project

docker-compose -f shell.docker-compose.yml run --rm shell ./generate-sample-project.sh 
cd sample-project; docker build -t sample-project:revision-1 .; cd ..

Inspect the layer sizes:

docker history sample-project:revision-1

Output:

IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
d4b30af167f4        3 seconds ago        /bin/sh -c #(nop) COPY dir:68b8f374d8731b8ad8   16.78 MB
c898fe1daa44        About a minute ago   /bin/sh -c apt-get update && apt-get install    10.77 MB
39a8a358844a        4 months ago         /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B
b1dacad9c5c9        4 months ago         /bin/sh -c #(nop) ADD file:5afd8eec1dc1e7666d   125.1 MB

Make some changes to the sample project

docker-compose -f shell.docker-compose.yml run --rm shell ./modify-sample-project.sh 

Verify that basing the project images on the revision 1 image tag contents does not lead to desired outcome

Verify that subsequent COPY . /app commands re-adds all files in every layer instead of only the files that have changed.

docker-compose -f shell.docker-compose.yml run --rm shell ./base-sample-project-images-on-revision-1.sh 
cd sample-project; docker build -f Dockerfile.based-on-revision-1 -t sample-project:revision-2-based-on-revision-1 .; cd ..
docker history sample-project:revision-2-based-on-revision-1

Output:

IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
4a3115eaf267        3 seconds ago       /bin/sh -c #(nop) COPY dir:61d102421e6692b677   16.78 MB
d4b30af167f4        25 seconds ago      /bin/sh -c #(nop) COPY dir:68b8f374d8731b8ad8   16.78 MB
c898fe1daa44        2 minutes ago       /bin/sh -c apt-get update && apt-get install    10.77 MB
39a8a358844a        4 months ago        /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B
b1dacad9c5c9        4 months ago        /bin/sh -c #(nop) ADD file:5afd8eec1dc1e7666d   125.1 MB

Even though we added/changed only a few bytes, all files are re-added and 16.78 MB is added to the total image size.

Also, the file(s) that we removed did not get removed.

Create an image with an optimized layer

cd sample-project; docker build -t sample-project:revision-2 .; cd ..
export RESTRICT_DIFF_TO_PATH=/app
export OLD_IMAGE=sample-project:revision-1
export NEW_IMAGE=sample-project:revision-2
docker-compose -f rsync-image-diff.docker-compose.yml up
docker-compose -f shell.docker-compose.yml -f process-image-diff.docker-compose.yml run --rm shell ./generate-dockerfile.sh
cd output; docker build -t sample-project:revision-2-processed .; cd ..

Verify that the processed new image has smaller sized layers with the changes:

docker history sample-project:revision-2-processed

Output:

IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
1920e750d362        24 seconds ago      /bin/sh -c if [ -s /.files-to-remove.list ];    0 B
1267bf926729        2 minutes ago       /bin/sh -c #(nop) ADD file:5021c627243e841a45   19 B
d04a2181b62a        2 minutes ago       /bin/sh -c #(nop) ADD file:14780990c926e673f2   264 B
d4b30af167f4        7 minutes ago       /bin/sh -c #(nop) COPY dir:68b8f374d8731b8ad8   16.78 MB
c898fe1daa44        9 minutes ago       /bin/sh -c apt-get update && apt-get install    10.77 MB
39a8a358844a        4 months ago        /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B
b1dacad9c5c9        4 months ago        /bin/sh -c #(nop) ADD file:5afd8eec1dc1e7666d   125.1 MB

Verify that the processed new image contains the same contents as the original:

export RESTRICT_DIFF_TO_PATH=/app
export OLD_IMAGE=sample-project:revision-2
export NEW_IMAGE=sample-project:revision-2-processed
docker-compose -f rsync-image-diff.docker-compose.yml up

The output should indicate that there are no differences between the images/tags. Thus, the sample-project:revision-2-processed tag can now be pushed and deployed, leading to the same end result but without having to push an unnecessary 16.78M over the wire, leading to faster deploy cycles.