Clarifications on KEDA scaler behavior with multiple triggers and with/withoutfallback options

[SomaSharathSurya]

I have a couple of questions regarding the behavior of the KEDA scaler in the following scenarios:

Scenario 1: ScaledObject with azure-log-analytics and CPU Triggers, with fallbackReplicas configured

1. What happens when one of the triggers fails to retrieve the metric? Does it fallback to the fallbackReplicas?
2. If the answer to question 1 is yes, is there an option to fallback to the CPU trigger instead of directly falling back to the fallbackReplicas?

log analytics queries are sometimes timing out in our case and wanted to understand whats the best way to control the behaviour in this instance

Scenario 2: ScaledObject with azure-log-analytics and CPU Triggers, without fallbackReplicas configured

1. What happens when one of the triggers fails to retrieve the metric? Does it fallback to the CPU-based scaler, or does it continue running with the current number of replicas?
2. What will be the behavior if both triggers fail?

Couldn't find any detailed documentation on how the behaviour will be in different scenarios, so thought of reaching out for help here. Thanks

[rickbrouwer]

Scenario 1:

1. Yes, it does fallback to the fallbackReplicas
2. No, there is no option for this use case

Scenario 2:

1. It continue running with the current number of replicas
2. Also it continue running with the current number of replicas

Maybe worth mentioning is that in the upcoming release (2.17) we will get an extra option 'behavior' at fallback:

https://github.com/kedacore/keda-docs/pull/1519/files

[SomaSharathSurya]

Thanks @rickbrouwer for sharing the details.

In our environment, we configured maxReplicas to equal fallbackReplicas to ensure that there is no impact on consumers in the event of a failure. However, we have observed that the LAW scaler often fails to retrieve the metric due to query timeouts or network issues, resulting in unnecessary scaling. We have tried adjusting failureThreshold, pollingInterval, and enableCache settings, but the issue still happens sometimes

I am wondering what's the best way to address these types of problems. Any thoughts / suggestions from your experience ?

[zroubalik]

@rickbrouwer the timeout in rabbitmq had been introduced before we added the global timeout. I think it could be added, I was thinking about introducing this on a trigger level (not in metadat) but on the same level like metricType or metricName, but http timeout is not applicable for scaler, so we can probably add it to the metadata.

Though I don't like the naming used in rabbit (timeout), it's too generic. Maybe httpTimeout? I think I can add this feature. @SomaSharathSurya does this apply only to the azure log analytics scaler?

[SomaSharathSurya]

@zroubalik Yes, for log analytics. It would be greatly appreciated if this could also be done for Datadog and Prometheus scalers.

[zroubalik]

Azure Log Analytics and Prometheus added here: #6607

Datadog is on hold, we need to merge this refactoring first: #6552

[SomaSharathSurya]

@zroubalik That's great, thank you. Appreciate the prompt response and actioning on this.

Just wondering will this be included in v2.17 that is schedule soon, or this will be part of a later release ?

[zroubalik]

There's a high chance it will be in 2.17, hopefully :)