Skill to construct configuration JSON files powering the lemonade router - #105

[sdevinenamd]

Adds a new skill that converts plain-English routing intent into a valid Lemonade collection.router policy JSON, ready for the user to register via POST /api/v1/pull.

Router reference: https://lemonade-server.ai/docs/dev/router-policy/

What the skill does

• Supports all three authoring levels: deterministic rules (keywords, regex, length, tools, images, metadata), semantic similarity classifiers (L2), and LLM-as-router (L0a)
• Generates and offline-validates the JSON before showing it
• Fills all unspecified fields with safe defaults (mode selection, ids, thresholds, on_error, model_name derivation)
• Hands the user copy-paste curl commands to register and trace-test on their own

Files

• SKILL.md: Generation steps 1–8, mode selection logic, condition reference table
• reference.m: Full schema contract, classifier type matrix, validation checklist
• examples.md: 7 worked NL→JSON pairs (examples 5–7 are production configs from an enterprise HR/Benefits/Finance deployment)
• scripts/validate.py: Offline structural validator

Example Prompts:

• Any message containing a credit card number pattern (four groups of four digits, optionally separated by spaces or dashes) must stay on Qwen3.5-9B-GGUF. Everything else goes to Qwen3.6-35B-A3B.
• Requests tagged with either "legal" or "compliance" or "audit" must stay on the local model Qwen3.5-9B-GGUF. Everything else goes to Qwen3.6-35B-A3B.
• Use an embedding model to detect whether a request is about cooking. High confidence cooking questions (above 0.8) go to Qwen3-1.7B-GGUF. Medium confidence ones (between 0.5 and 0.8) go to Qwen3.5-9B-GGUF but tag them as uncertain. Anything below 0.5 falls through to Qwen3.6-35B-A3B.
• I want image requests to stay on Qwen3.5-9B-GGUF since it supports vision. Requests without any images go to Qwen3-1.7B-GGUF. Default to the vision model.

[ramkrishna2910]

Some comments:

1. Example 7 embeds the exact mistake the skill warns against. The complexity llm classifier's prompt ends with "Reply with exactly one label: COMPLEX or SIMPLE." The engine's {"model", "rationale"} contract applies to all llm classifiers, not just routing.router — it's the same LlmClassifier class either way. A weaker model obeying that authored line replies bare COMPLEX, the strict parser rejects it, the score comes back empty, and the rule silently never fires. That's the identical silent-fall-through failure described in checklist item 12 — but the skill scopes the warning (and validate.py's BAD_ROUTER_PROMPT_RE) to router.prompt only. Fix: delete that sentence from example 7, and extend item 12 + the validator regex to cover llm classifier prompts too.

2. The reference never documents that hand-authored llm classifiers speak the router's wire contract — that their score is binary (chosen label = 1.0, everything else absent). Authors will naturally write "reply with the label" prompts unless told; this is the doc gap that produced problem 1.

3. Minor gap in validate.py: the server rejects regexes with nested unbounded quantifiers (e.g. (X+)+) at policy load as a ReDoS safeguard (documented in route_policy.schema.json); the validator's Python re.compile check can't catch that, so a policy can pass validation and still fail /pull. A cheap pattern check would close it.

4. Nits: the decision object also carries version: "1" (skill omits it); metadata comparisons are case-sensitive with comma-decoded token sets for any (unmentioned); and example 3's keywords_all: ["apparel", "clothes"] demands both words for a user who said "clothing or apparel" — should be keywords_any.

[sdevinenamd]

@ramkrishna2910 Thanks for the feedback! Addressed them in the latest commit