Error checking metric providers: prometheus in metric template success-rate.fakenamespace not avaiable

[felipesbs]

Describe the bug

running query failed: error response: ","canary":"fakedeploy.fakenamespace │
│ ","stacktrace":"github.com/fluxcd/flagger/pkg/controller.(*Controller).recordEventErrorf\n\t/workspace/pkg/controller/events.go:39\ngithub.com/fluxcd/flagger/pkg/controller.(*Controller).advanceCanary\n\t/workspace/pkg/controller/scheduler.go:207\ngithub │
│ .com/fluxcd/flagger/pkg/controller.CanaryJob.Start.func1\n\t/workspace/pkg/controller/job.go:39"}

To Reproduce

Create MetricTemplate Resource with this specs.

apiVersion: flagger.app/v1beta1
kind: MetricTemplate
metadata:
  name: success-rate
  namespace: fakenamespace
spec:
  provider:
    type: prometheus
    address: http://prometheus.linkerd-viz:9090
  query: |
    sum(
      rate(
        response_total{
          namespace="{{ namespace }}",
          deployment=~"{{ target }}",
          classification!="failure",
          direction="{{ variables.direction }}"
        }[{{ interval }}]
      )
    ) 
    / 
    sum(
      rate(
        response_total{
          namespace="{{ namespace }}",
          deployment=~"{{ target }}",
          direction="{{ variables.direction }}"
        }[{{ interval }}]
      )
    ) 
    * 100

Expected behavior

Check Metrics for Canary Deploy

Additional context

• Flagger version: 1.34.0
• Kubernetes version: GKE 1.27.7-gke.1121000
• Service Mesh provider: Linkerd
• Ingress provider: Nginx

[LiZhenCheng9527]

Could you provide the flagger logs? Was there a log like this when the flagger installation succeeded:
{"level":"info","ts":"2024-01-26T07:44:04.473Z","caller":"flagger/main.go:206","msg":"Connected to metrics server http://prometheus:9090"}