Custom Provider destructive update behavior

[samidbb]

We are currently building a Crossplane provider for provisioning resources on Confluent Cloud. We have run into a very general discussion (we think) about how to implement the update functionality for managing a very stateful resource. For resource provisioning some of the initial inputs would lead to a destructive update procedure (delete and then create) if changed for the reconsiler to apply the changes to the resource.
We can think of these solutions to the problem:

1. Destructive updates shouldn't be implemented in the provider, so that the Kubernetes API server would accept the changes, but the resource would never sync.
2. Destructive updates are just implemented in the provider, but if wanted/needed validation for immutable fields are handle by validating admission webhooks, since they are not supported for CRD's.

[jbw976]

Good question! IMHO, I think option 2 with the validating admission webhook is the better user experience, because it provides the immediate feedback upon applying the edits. Option 1 would probably result in more confusion if the updates are accepted by the API server but then nothing happens and the user needs to go searching around for an explanation of what is happening under the covers. A validating webhook could immediately provide the feedback that the update is not valid or accepted, so that UX is probably best.

This is another good example of how a pluggable webhook framework in crossplane could make developing and deploying functionality like this more streamlined. Feel free to add a thumbs-up on this issue: #1152

[stevendborrelli]

There are some Crossplane docs on immutable properties. In general, I've seen #1 used more often.

Also see Crossplane issue 727.

[wcarlsen]

@stevendborrelli interesting link. Could one interpret it as okay to taste on the DeletionPolicy setting for the resource in the update method if the update turns out to be destructive and if not Orphan destructive updates could be allowed?

[negz]

My thinking here is:

• Fields that are immutable should be marked as such, and some machinery (either native API server support if it ever comes, or an admission control webhook) should prevent updating them. Sadly this does not exist today and thus as you've noticed, we silently accept what are effectively no-op update operations.
• Destructive operations shouldn't be implicit; we shouldn't delete and recreate something when someone appeared to ask for an update. Instead I think we should require folks to delete and recreate the managed resource. This is arguably a little imperative rather than declarative, but I worry about potentially losing state without a "hey I'm actually going to delete and recreate this" prompt.

I would consider it acceptable for a provider (and its managed resource APIs) to reach beta quality (e.g. v1beta1) without addressing this shortcoming; we're planning to start adding the plumbing to tackle at least the ability to reject updates to immutable fields this quarter per #2719.

[wcarlsen]

In the case of using validating webhooks and implementing destructive updates one could easily think of a situation where one imports a resource but hasn't gotten the configuration exactly right, so your import do a "latent create" and then a destructive update because the validating webhook is unaware of resource you are trying to import.