Publishing to colima Docker context from kaniko running in Kubernetes

[daecabhir]

I'm trying to noodle out how to make this basic workflow happen in colima, with Kaniko running inside of a Kubernetes pod:

1. Pull a base image from the local Docker context
2. Build an image on top of that with other resources
3. Publish the new image back to Docker to the local context, so it is available to Kubernetes

with the goal of starting a new pod in Kubernetes that references the image that was just built, and having Kubernetes launch the pod with that new image.

We're already kind of doing this in a full-blown Kubernetes cluster where Kubelet is configured to pull from an external registry via an alias of cluster.local so instead of pulling from / publishing to Docker we're talking to that registry. The Dockerfile would resemble:

ARG REGISTRY_HOST=external.registry.fqdn
FROM ${REGISTRY_HOST}/path/to/my-base-image:1.0.0

COPY ...

RUN ...

...etc...

and we publish to external.registry.fqdn/path/to/my-new-image:2.0.0. But the image name spec for the pod would be cluster.local/path/to/my-new-image:2.0.0. So in colima we'd need to be able to configure the build argument REGISTRY_HOST so that it points to the colima Docker context (or registry - I think).

This is primarily for being able to test the process locally, instead of having to push helm charts, etc. up to a cluster to test changes. Any suggestions on how one might go about doing something like this, perhaps with a local registry outside of Kubernetes and tweaks to kubelet to have it work like the full cluster configuration we're using would be greatly appreciated.