neonvm-controller: Add image mapping feature

[hlinnaka]

The image map allows quickly overriding image names specified in the VirtualMachine spec. This is for local developmnet, there's currently no intention of using this in production.

I will use this to implement fast reloading of compute images, when you run the console and control plane under Tilt.

[github-actions]

No changes to the coverage.

HTML Report

Click to open

[mikhail-sakhnov]

I am a bit concerned that we already have fields to control images in VM:

- spec.guest.rootDisk.image
- spec.guest.kernelImage
- spec.runnerImage

and now introducing something that overlaps it.

Should it be better instead to introduce a new field like spec.image_live_reload: bool and handle those fields in a controller? What do you think?

For example, in the controller loop we could have a logic (pseudocode)

if spec.image_live_reload == true && anyOfTheImageFieldsHasChanged(vm_from_req, current_vm) { 
  recreatePod()
}

func anyOfTheImageFieldsHasChanged(vm_from_req, current_vm) bool {
// if any of the images in vm_from_req is different from current_vm we trigger pod recreation
}

[hlinnaka]

I am a bit concerned that we already have fields to control images in VM:

- spec.guest.rootDisk.image
- spec.guest.kernelImage
- spec.runnerImage

and now introducing something that overlaps it.

Wouldn't say "overlaps". Rather, it provides an extra layer of indirection to it. :-)

Should it be better instead to introduce a new field like spec.image_live_reload: bool and handle those fields in a controller? What do you think?

For example, in the controller loop we could have a logic (pseudocode)

if spec.image_live_reload == true && anyOfTheImageFieldsHasChanged(vm_from_req, current_vm) { 
  recreatePod()
}

func anyOfTheImageFieldsHasChanged(vm_from_req, current_vm) bool {
// if any of the images in vm_from_req is different from current_vm we trigger pod recreation
}

The problem I'm trying to solve with this is that when I hook up Tilt to automatically build the compute image from sources, it uses a different image tag on every build. For example, k3d-neon-local-registry:5000/vm-compute-node-v17:tilt-bce03df2bbb7ad9d. We need to somehow get that image tag to the pods that are created.

What you're proposing with image_live_reload doesn't address that problem. We might want to have that too, i.e. automatically recreate the pod when the image changes, but it's a different issue.

One option would be to modify the the default image tag in the control plane configmap, and restart the control plane. Another would be to update the settings in the control plane db; that's how we deploy images in staging and production. But it's a bit tedious to write a script that would do that as part of a tilt reload.

This PR is a third alternative: Use the original image tag, e.g. neondatabase/vm-compute-node-v17:latest in the control plane, but have this extra mapping that tells neonvm-controller that "when neondatabase/vm-compute-node-v17:latest is requested, actually use k3d-neon-local-registry:5000/vm-compute-node-v17:tilt-bce03df2bbb7ad9d instead". It seemed like the simplest way to accomplish this.

Yet another approach might be to always import the generated k3d-neon-local-registry:5000/vm-compute-node-v17:tilt-bce03df2bbb7ad9d image to the k3d cluster with the same tag, neondatabase/vm-compute-node-v17:latest, overriding the previous image. I think I tried to do that at some point, but it was also a bit ugly: you need to invoke k3d image import from the tiltfile, and it adds another image-copying step which makes it slower.

[sharnoff]

Would it work to re-tag each new image to some consistent tag? e.g. k3d-neon-local-registry:5000/vm-compute-node-v17:tilt-bce03df2bbb7ad9d -> k3d-neon-local-registry:5000/vm-compute-node-v17:latest in the actual local registry (so we're still only copying on-demand, but we don't need substantial changes elsewhere)

We'd also need to set ImagePullPolicy: Always in the VM spec, IIRC

[hlinnaka]

Would it work to re-tag each new image to some consistent tag? e.g. k3d-neon-local-registry:5000/vm-compute-node-v17:tilt-bce03df2bbb7ad9d -> k3d-neon-local-registry:5000/vm-compute-node-v17:latest in the actual local registry (so we're still only copying on-demand, but we don't need substantial changes elsewhere)

Hmm, I don't know how to do that. Would need a command that connects to the k3d-neon-local-registry and creates a new tag for the image. I'll google around a bit, but let me know if you have ideas..

[hlinnaka]

Ok, I was able to cajole Tilt to use a constant tag, instead of generating a new one every time the image is built. With that, I don't need this PR anymore.