feat(quarantine): add configurable warning/critical thresholds to flaky tests report

[dhiller]

Summary

• Adds visual highlighting (warning/critical) for flaky test impacts in the most-flaky-tests report
• Configurable thresholds via CLI flags for both 3-day and 14-day time ranges
• Warning (yellow/🟡) and critical (red/🔴) styling based on failure percentage relative to quarantine criteria

Flags

Flag	Default	Description
--three-day-warning-threshold	10%	Minimum 3-day failure % for warning
--three-day-critical-threshold	20%	Minimum 3-day failure % for critical
--fourteen-day-warning-threshold	3%	Minimum 14-day failure % for warning
--fourteen-day-critical-threshold	5%	Minimum 14-day failure % for critical

Critical thresholds align with quarantine criteria.

Fixes #4906

🤖 Generated with Claude Code

[kubevirt-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tiraboschi for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dhiller]

FYI @cuiyingd

[kubevirt-bot]

Keywords which can automatically close issues and hashtag(#) mentions are not allowed in commit messages.

The list of commits with invalid commit messages:

• 2d27a00 feat(quarantine): add configurable warning/critical thresholds to flaky tests report

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• The thresholds map uses a bare [2]float64 index in aggregateMostFlakyTestsBySIG without an ok check, which will panic if a new TimeRange is ever added without a corresponding entry; consider either validating the map up-front or handling the missing-key case explicitly.
• Using a plain [2]float64 for thresholds makes the code harder to read and error-prone (indexes 0/1 are not self-describing); consider introducing a small struct (e.g. { Warning, Critical float64 }) to make the thresholds usage clearer in both the map and the template helpers.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The thresholds map uses a bare `[2]float64` index in `aggregateMostFlakyTestsBySIG` without an `ok` check, which will panic if a new `TimeRange` is ever added without a corresponding entry; consider either validating the map up-front or handling the missing-key case explicitly.
- Using a plain `[2]float64` for thresholds makes the code harder to read and error-prone (indexes 0/1 are not self-describing); consider introducing a small struct (e.g. `{ Warning, Critical float64 }`) to make the thresholds usage clearer in both the map and the template helpers.

## Individual Comments

### Comment 1
<location path="pkg/searchci/searchci.go" line_range="174-179" />
<code_context>
+
+// ScrapeImpactsWithMinPercent works like ScrapeImpacts but uses a configurable minimum percentage threshold
+// instead of the default thresholds for each time range.
+func ScrapeImpactsWithMinPercent(testNameSubstring string, timeRange TimeRange, minPercent float64) ([]Impact, error) {
+	scrapeResultURL := NewScrapeURL(testNameSubstring, timeRange)
+	logger.Debugf("scraping search.ci for test %q with URL %q", testNameSubstring, scrapeResultURL)
+	resp, err := http.Get(scrapeResultURL)
+	if err != nil {
+		return nil, fmt.Errorf("failed to get search.ci results from %s: %w", scrapeResultURL, err)
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Consider checking HTTP status codes before reading the body

Non-2xx responses (e.g. 404, 500, 503) will currently be treated as successful and passed into `ScrapeImpact`, which can lead to misleading parsing errors or empty results. Consider checking `resp.StatusCode` and returning a clear error for non-2xx responses so callers can distinguish genuine scrape failures from legitimate “no impacts” outcomes.

```suggestion
	resp, err := http.Get(scrapeResultURL)
	if err != nil {
		return nil, fmt.Errorf("failed to get search.ci results from %s: %w", scrapeResultURL, err)
	}
	defer resp.Body.Close()

	if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
		return nil, fmt.Errorf("unexpected status code %d from search.ci at %s", resp.StatusCode, scrapeResultURL)
	}

	body, err := io.ReadAll(resp.Body)
```
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion (bug_risk): Consider checking HTTP status codes before reading the body

Non-2xx responses (e.g. 404, 500, 503) will currently be treated as successful and passed into ScrapeImpact, which can lead to misleading parsing errors or empty results. Consider checking resp.StatusCode and returning a clear error for non-2xx responses so callers can distinguish genuine scrape failures from legitimate “no impacts” outcomes.

Suggested change

	resp, err := http.Get(scrapeResultURL)
	if err != nil {
	return nil, fmt.Errorf("failed to get search.ci results from %s: %w", scrapeResultURL, err)
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(resp.Body)
	resp, err := http.Get(scrapeResultURL)
	if err != nil {
	return nil, fmt.Errorf("failed to get search.ci results from %s: %w", scrapeResultURL, err)
	}
	defer resp.Body.Close()
	if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
	return nil, fmt.Errorf("unexpected status code %d from search.ci at %s", resp.StatusCode, scrapeResultURL)
	}
	body, err := io.ReadAll(resp.Body)