machine-specific machineconfigs

[cgwalters]

Ironically today MachineConfig objects target a pool - not a specific machine.

Managing machine-specific configuration today

Follow up to this comment. Today if you want to manage per-machine configuration such as e.g. statically set hostname, the best way to do this is to take the worker.ign file generated by openshift-installer and create e.g .worker-foo.ign and worker-bar.ign copies, then modify them to include configuration specific to the machine, and provide that Ignition to the node.

For the "provide that Ignition to the node" phase, if you're using MachineSets via MachineAPI, then one would need to edit the machineset object to point to a new user data secret. Note this will eventually conflict with having the MCO manage userdata but that enhancement was reverted. But when we get there, we can teach the MCO to retain any additional config it finds in the machineset perhaps?

The OpenShift 4.8 documentation will describe how to use https://github.com/coreos/butane which is a high level tool for managing Ignition, but it isn't yet ergonomic to "backconvert" that pointer Ignition to butane, then output a new Ignition config.

Using the Live ISO

Additionally, one can use the Live ISO which can be programmed to have its own Ignition configuration that e.g. pulls a container which performs hardware inspection, and then dynamically generates a configuration which is passed to coreos-installer.

Background information

If we had a way to provide "machine specific machineconfigs" then the admins could provide those MCs as additional manifests and it'd all Just Work.

But...this gets into the "node identity" problem. We'd need to define a way for the MCS to identify the node it's serving the config to.

Perhaps the simplest thing is to assume anyone who wants this is statically assigning hostnames, and we change Ignition to include a header with the node's hostname.

(I guess we could do something like a reverse lookup of the requester's IP address too)

One messy aspect of this too is that we can't include these bits configs in the main rendered-$pool configs which means the MCS needs to generate it internally. (I guess we could create a separate rendered-node-$x config too?)

[michaelgugino]

Perhaps the simplest thing is to assume anyone who wants this is statically assigning hostnames, and we change Ignition to include a header with the node's hostname.

Definitely we should do this with or without machine-specific machineconfigs.

(I guess we could do something like a reverse lookup of the requester's IP address too)

This is not guaranteed to work in all environments, and I think we discovered during troubleshooting 4.4 release that the IPs that show up in the MCS during first boot are VIPs, not instance IPs.

Including the hostname and IP on every request to the MCS would greatly aid in determining when a machine failed to boot/request an ignition vs failing to join cluster after getting ignition file.

[cgwalters]

One way to do this today is - for each machine one wants to have a custom config for, manually set things up to pass it a custom derived pointer config.

For example in a PXE install scenario, take the Ignition output from openshift-install create ignition-configs and edit that, rather than trying to customize the full Ignition config returned from the MCS.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[dhellmann]

/remove-lifecycle stale

[dhellmann]

/lifecycle frozen

[cgwalters]

Perhaps one simple approach here is:

• user injects /etc/hostname in the pointer Ignition config
• Ignition detects this and adds it as a header when doing requests for merged configs (like the MCO)
• MCO reads this header and adds in to the rendered Ignition (from all machineconfig objects with a label for that hostname)

However, we could simplify things significantly if we said the MCD wouldn't be aware of this - we wouldn't support "day 2" reconfiguration for machine-specific configs. IOW if e.g. you want to change the static IP address for a node, you fully reprovision it.

Given that I think most of this per-node configuration wouldn't change often, that could be a useful balance.