Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offline validation for config passed to proxy wasm #35

Open
jcchavezs opened this issue Nov 17, 2022 · 5 comments
Open

Offline validation for config passed to proxy wasm #35

jcchavezs opened this issue Nov 17, 2022 · 5 comments

Comments

@jcchavezs
Copy link

jcchavezs commented Nov 17, 2022

proxy-wasm extensions allows people to extend proxy functionalities in the request/response lifecycle and are deployed along with proxies but have their own lifecycle.

One big issue right now is that the config passed to those extensions are mainly validated in runtime, meaning that user can't have feedback about incorrect config before deployment have been attempted for example:

  • in envoy, the failure in config crashes the proxy.
  • in istio wasm plugins are eventually deployed and we have no clue if a plugin failed due to config unless we look at the logs or describe the pod.

A new proxy_validate_config method could be added to the ABI:

proxy_validate_config
params:
i32 (uint32_t) root_context_id
i32 (size_t) plugin_configuration_size
returns:
i32 (proxy_result_t) call_result

the result could be a list of errors e.g. [LINE 1] Unexpected token {or going forward a flatmap with the location e.g. .rules[0] but that assumes all configs are structured which I am not 100% sure.

Ping @mathetake @PiotrSikora

Related: corazawaf/coraza-proxy-wasm#88 (comment)

@mathetake
Copy link
Contributor

even if this function is added, Envoy/Istio has no way to report back to the users. Plus this seems essentially the same as the return value from the existing proxy_on_configure ABI. I suggest instead we just instantiate plugins at the control plane layer, and see the return value from proxy_on_configure

@jcchavezs
Copy link
Author

jcchavezs commented Nov 21, 2022 via email

@PiotrSikora
Copy link
Member

PiotrSikora commented Nov 21, 2022

The original ABI had a function exactly like that, and Proxy-Wasm C++ SDK still has it (see: proxy_validate_configuration, RootContext::validateConfiguration), but I'm not aware of any plugins using it.

Note there are a few problems with such approach:

  1. proxy_validate_configuration might be only slightly different from proxy_on_configure(configuration), which can result in code duplication and both functions could easily get out-of-sync over time,
  2. some plugins require access to runtime variables (e.g. environment variables or host metadata), which control plane might not know about,
  3. some plugins fetch parts of configuration from remote HTTP/gRPC endpoints at startup.

Basically, proxy_validate_configuration might catch some errors (e.g. parsing), but it cannot catch all of them, and it could introduce false negatives, so it's not a replacement for the proper solution (i.e. Envoy rejecting the configuration, and Istio propagating this information back to the operator).

@PiotrSikora
Copy link
Member

Another consideration here is that we access to those content types that
are being inspected and only buffer when inspection.

Could you elaborate? I don't understand how this is related to control plane performing config validation.

@jcchavezs
Copy link
Author

proxy_validate_configuration might be only slightly different from proxy_on_configure(configuration), which can result in code duplication and both functions could easily get out-of-sync over time

Agree and I think the main difference between the two is the output but functionality wise they are quite similar.

some plugins require access to runtime variables (e.g. environment variables or host metadata), which control plane might not know about,some plugins fetch parts of configuration from remote HTTP/gRPC endpoints at startup.

Agree, this is mainly a best effort to allow users to validate the config offline.

So providing the method in the wasm extension isn't a perfect solution it just guarantees cohesion meaning that the same wasm that is running in prod is the one validating the config as opposed to provide another wasm (to be able to load it in different languages) or per language libraries (mostly limited to the wasm extension language)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants