[CI] RolloverActionIT testILMRolloverRetriesOnReadOnlyBlock failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #12685 / amazonlinux-2023_platform-support-aws

Reproduction Line:

./gradlew ":x-pack:plugin:ilm:qa:multi-node:javaRestTest" --tests "org.elasticsearch.xpack.ilm.actions.RolloverActionIT.testILMRolloverRetriesOnReadOnlyBlock" -Dtests.seed=9727581EDB18DA1 -Dtests.locale=sa -Dtests.timezone=Pacific/Yap -Druntime.java=26

Applicable branches:
8.19

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: <{"phase":"hot","action":"complete","name":"complete"}>
     but: was <{"phase":"hot","action":"rollover","name":"set-indexing-complete"}>

Issue Reasons:

• [8.19] 1 consecutive failures in test testILMRolloverRetriesOnReadOnlyBlock
• [8.19] 1 consecutive failures in step amazonlinux-2023_platform-support-aws
• [8.19] 1 consecutive failures in pipeline elasticsearch-periodic-platform-support

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-core-infra.

[elasticsearchmachine]

This has been muted on branch 8.19

Mute Reasons:

• [8.19] 1 consecutive failures in test testILMRolloverRetriesOnReadOnlyBlock
• [8.19] 1 consecutive failures in step amazonlinux-2023_platform-support-aws
• [8.19] 1 consecutive failures in pipeline elasticsearch-periodic-platform-support

Build Scans:

• elasticsearch-periodic-platform-support #12685 / amazonlinux-2023_platform-support-aws

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

Test Failure Analysis

Summary

• testILMRolloverRetriesOnReadOnlyBlock likely failed due to an overly short default assertBusy timeout (10s) after removing the read-only block. ILM had already progressed to the set-indexing-complete rollover step but did not reach hot/complete before the assertion window expired. This matches an intermittent CI timing flake rather than a stuck production ILM path.

Release risk: low-risk

The failure was a single consecutive miss on amazonlinux-2023 periodic platform-support; reproduction on the same commit, seed, locale, timezone, and Java 26 passed on AlmaLinux 9. The observed step (set-indexing-complete) is a normal late stage of rollover, not an error loop. A production ILM bug would require evidence of indices stuck at set-indexing-complete after read-only is cleared — none seen here. Would raise to medium-risk only if amazonlinux-only reproduction shows ILM genuinely stuck (not merely slow).

Impact

• Affected branches: 8.19
• Affected OS: amazonlinux-2023 (original CI); not reproduced on almalinux-9 (GCP substitute)
• Affected pipelines: elasticsearch-periodic-platform-support (amazonlinux-2023_platform-support-aws)
• Trend: first consecutive failure on 2026-08-27; test auto-muted on 8.19

Reproduction

• Command: ./gradlew ":x-pack:plugin:ilm:qa:multi-node:javaRestTest" --tests "org.elasticsearch.xpack.ilm.actions.RolloverActionIT.testILMRolloverRetriesOnReadOnlyBlock" -Dtests.seed=9727581EDB18DA1 -Dtests.locale=sa -Dtests.timezone=Pacific/Yap -Druntime.java=26
• Commit: 4fe44c255c3d0da06779b921e132cdc555ed9aff
• Remote job: https://buildkite.com/elastic/elasticsearch-agentic-workflow/builds/838#01a0413f-56c8-4052-b146-3f89771b0015 — almalinux-9 (amazonlinux-2023 unavailable in reproduction generator)
• Result: not reproduced — test passed in ~3m on AlmaLinux 9 with identical filters and commit
• Build scan: https://gradle-enterprise.elastic.co/s/6nqkkgplkbw4i

Mitigation Proposal

• Reproduction verdict: Passing reproduction on the failing commit lowers confidence in a platform-agnostic production defect; supports intermittent / platform-timing flake.
• Codebase investigation: RolloverActionIT.testILMRolloverRetriesOnReadOnlyBlock — final completion assertBusy uses the 10s default while most sibling ILM waits in the same file use 30s. Failure message showed step hot/rollover/set-indexing-complete instead of expected hot/complete.
• Recent git history: No recent ILM rollover/read-only changes on 8.19; test was auto-muted in e9d633ff after this failure.
• Proposed fix: Add 30, TimeUnit.SECONDS to both assertBusy calls in testILMRolloverRetriesOnReadOnlyBlock (retry-count wait and final completion wait), matching other rollover ITs; unmute after merge.
• Effort estimate: trivial — two-line timeout change plus mute removal
• Confidence: 0.82 (high) — failure step progression + timeout mismatch + non-reproduction on same commit are consistent with flake; amazonlinux-only timing not directly verified

---

Generated by test-analysis · archimedes v0.50.3 · cursor/composer-2.5