Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to unset "retry_host_predicate" using EnvoyFilter? #50756

Closed
ktalg opened this issue Apr 30, 2024 · 4 comments · Fixed by #50759
Closed

How to unset "retry_host_predicate" using EnvoyFilter? #50756

ktalg opened this issue Apr 30, 2024 · 4 comments · Fixed by #50759

Comments

@ktalg
Copy link

ktalg commented Apr 30, 2024

Requests routed using the consistency hash lb are retried and the target host is affected by envoy's 'retry_host_predicate', and the target host is different from the initial request, which is not what I expected, so I would like to use EnvoyFilter to unset this configuration, but I found that EnvoyFilter can hardly merge fields of this array type when configuring

original config_dump：

{
  "name": "httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000",
  "domains": [
    "httpbin.infra-al-cn-gdsz-1.svc.cluster.local"
  ],
  "routes": [
    {
      "match": {
        "prefix": "/"
      },
      "route": {
        "cluster": "outbound|8000||httpbin.infra-al-cn-gdsz-1.svc.cluster.local",
        "timeout": "0s",
        "retry_policy": {
          "retry_on": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
          "num_retries": 2,
          "retry_host_predicate": [
            {
              "name": "envoy.retry_host_predicates.previous_hosts"
            }
          ],
          "host_selection_retry_max_attempts": "5",
          "retriable_status_codes": [
            503
          ]
        },
        "hash_policy": [
          {
            "header": {
              "header_name": "xxy"
            }
          }
        ],
        "max_stream_duration": {
          "max_stream_duration": "0s",
          "grpc_timeout_header_max": "0s"
        }
      },
      "decorator": {
        "operation": "httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000/*"
      },
      "name": "default"
    }
  ],
  "include_request_attempt_count": true
}

envoyfilter:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: set-retry-predicate
  namespace: infra-al-cn-gdsz-1
spec:
  workloadSelector:
    labels:
      app: grpc-c
  configPatches:
    - applyTo: HTTP_ROUTE
      match:
        context: ANY
        routeConfiguration:
          vhost:
            name: httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000
            route:
              action: ROUTE
              name: default
          name: "8000"
      patch:
        operation: MERGE
        value:
          route:
            retry_policy:
              retry_host_predicate: [ ] # NOT work !
              host_selection_retry_max_attempts: 1 # work !

new config_dump

{
  "name": "httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000",
  "domains": [
    "httpbin.infra-al-cn-gdsz-1.svc.cluster.local"
  ],
  "routes": [
    {
      "match": {
        "prefix": "/"
      },
      "route": {
        "cluster": "outbound|8000||httpbin.infra-al-cn-gdsz-1.svc.cluster.local",
        "timeout": "0s",
        "retry_policy": {
          "retry_on": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
          "num_retries": 2,
          "retry_host_predicate": [
            {
              "name": "envoy.retry_host_predicates.previous_hosts"
            }
          ],
          "host_selection_retry_max_attempts": "1",
          "retriable_status_codes": [
            503
          ]
        },
        "hash_policy": [
          {
            "header": {
              "header_name": "xxy"
            }
          }
        ],
        "max_stream_duration": {
          "max_stream_duration": "0s",
          "grpc_timeout_header_max": "0s"
        }
      },
      "decorator": {
        "operation": "httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000/*"
      },
      "name": "default"
    }
  ]
}

You can see that the string type host_selection_retry_max_attempts was successfully set to 1, while the array type retry_host_predicate was not. How should I set it correctly? Or is it some kind of bug?
@ramaraochavali
Copy link
Contributor

Are you sure, with consistent hash, the retry predicate picks another host that does hash based on the hashing config? If yes, it seems like a bug in Envoy

@ktalg
Copy link
Author

ktalg commented Apr 30, 2024 •
edited

Are you sure, with consistent hash, the retry predicate picks another host that does hash based on the hashing config? If yes, it seems like a bug in Envoy

I can confirm that Envoy behaves exactly as I describe, but I can't confirm if this is a bug.

there're simply 3 httpbin pods

when I call curl httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000/status/200 -v -H "xxy: 1"(setting the hash header'xxy'), the istio-proxy container for only one of them prints access log

but when I call curl httpbin.infra-al-cn-gdsz-1.svc.cluster.local:8000/status/503 -v -H "xxy: 1"(returns 503 and triggers retry), two of them print logs

istio ver: 1.11
envoy ver: 1.19

@ramaraochavali ramaraochavali mentioned this issue Apr 30, 2024
Merged
12 tasks
@ktalg
Copy link
Author

ktalg commented May 6, 2024

@ramaraochavali Thank you very much for your MR, but before merging, is there any way to solve the "array merge" problem in encoyFilter?

@ramaraochavali
Copy link
Contributor

AFAIK, It is not possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

do not add previous hosts predicate when consistent hashing is enabled ramaraochavali/istio
2 participants
@ramaraochavali @ktalg