Skip to content

[LLK AI codegen] Best-so-far retention + max-iter cap on score-driven regen loop #44155

@mvlahovicTT

Description

@mvlahovicTT

Two known caveats of the score-driven regen loop, called out in README §"Score-driven regen loop":

  1. No max-iteration cap. If a doc consistently scores below TIER1_THRESHOLD = 0.85 (e.g. source is genuinely malformed or has unparseable generate blocks), the loop runs indefinitely. Fix: hard cap at ~5 iterations, log+park.
  2. No best-so-far retention. Each regen overwrites the previous doc on disk. If round-2 produces a worse doc than round-1, the better one is lost. Fix: keep the highest-scoring version per element across all regen rounds and only commit-to-disk when the new score beats the stored max.

Both are documented but not fixed yet. Low priority — current corpus converges in 2-4 iterations in practice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions