Skip to content

Latest commit

 

History

History
228 lines (160 loc) · 10.9 KB

File metadata and controls

228 lines (160 loc) · 10.9 KB

11 — Edge cases and failure modes

Grouped by stage. Each entry: what happens → how it presents → what to do.


A. Model / tokenizer

A1. Vocabulary mismatch between teacher and student. Logit KD compares unrelated indices. Loss decreases anyway; quality plateaus mysteriously. → Assert get_vocab() equality at startup. If they differ, use sequence-level KD only.

A2. vocab_size ≠ embedding rows (151,643 vs 151,936 for Qwen2.5). 293 unused columns receive probability mass; student can emit undecodable ids. → Slice to min(V_t, V_s); optionally mask reserved ids to -inf before softmax.

A3. Wrong EOS. Qwen2.5-Instruct ends turns with <|im_end|> (151645), not <|endoftext|>. Generation never stops; every summary runs to max_new_tokens. → generation_config.eos_token_id = [151645, 151643]; verify the training targets end with 151645.

A4. No pad token. Crash, or silent use of EOS as pad. Since pad == EOS, masking by token id also masks the real EOS → model never learns to stop. → Mask by position/length, never by id.

A5. Left vs right padding. Right-padded generation continues from pad tokens (garbage); left-padded training misaligns labels. → Right for training, left for generation. Set tokenizer.padding_side explicitly in both paths.

A6. Tied embeddings on 0.5B/1.5B. LoRA on lm_head also changes input embeddings; separate quantization of lm_head corrupts embeddings. → Exclude lm_head/embed_tokens from LoRA targets and from per-module quantization.

A7. Chat template drift between transformers versions. Your stored logits were computed under one template, training uses another; every position is shifted. → Pin transformers, store template_hash = sha256(tokenizer.chat_template) in the logits meta, and assert it matches at train time.

A8. add_special_tokens=True after apply_chat_template. Double special tokens. → Always add_special_tokens=False when the template already ran.

A9. BPE merge across the prompt/completion boundary. Concatenating separately-tokenized ids produces a sequence the model never sees at inference. → Tokenize the full string; assert the prompt ids are a prefix.


B. Data

B1. Truncated document, untruncated summary. Teacher summarized content the student can't see. You are training hallucination. Presents as: model invents plausible facts, worst on long docs. → Drop, or truncate then regenerate the summary.

B2. Teacher refusals in the training set. "I cannot summarize this content." Student learns to refuse ordinary documents. → Filter refusal patterns; check the rate — a spike means a prompt or safety-filter change.

B3. Teacher degeneration (repetition loops) at high sampling temperature. → 4-gram repetition filter; cap gen_temperature ≤ 1.0.

B4. Duplicate documents across train and val. Val metrics are inflated; you ship a worse model believing it's better. → Hash-based split assignment + cross-split dedup.

B5. Class/domain imbalance after filtering. Filters drop harder on some domains (legal fails format checks more), silently skewing the mixture. → Report drop rate per domain; rebalance after filtering, not before.

B6. Length distribution shift between train and production. Train p95 is 2k tokens, users paste 20k. Model degrades off a cliff beyond training length. → Include long examples, or add an explicit chunk-and-merge path and evaluate it separately.

B7. PII memorized into weights. Not deletable, not GDPR-erasable. → Scrub before the teacher sees it.

B8. Contaminated eval. Test documents are in the teacher's pretraining set; teacher recites. → Post-cutoff OOD test set.

B9. All-masked example. A record where the summary is empty → |M| = 0 → NaN. → denom.clamp_min(1) plus drop empty targets at filter time.

B10. Non-UTF8 / mojibake / OCR noise. Tokenizes into byte-fallback soup, consuming 3–4x the tokens and teaching the student nothing. → Normalize (NFC), drop records above a byte-fallback-token ratio threshold.

B11. Mixed-script and Indic text. Hindi consumes ~2–3x more tokens per character than English in most BPEs. Your max_seq_len budget is silently smaller for Hindi documents. → Compute length stats per language; consider a per-language max_seq_len.


C. Loss and training

C1. F.kl_div with reduction='mean'. Divides by B·T·V; KD term is ~150,000x too small. Presents as "KD makes no difference." → Use reduction='batchmean' or reduce manually.

C2. kl_div argument order. input must be log-probabilities, target probabilities. Reversed gives a wrong, sometimes negative, loss. → Assert loss >= 0.

C3. Off-by-one in the shift. Student predicts the current token. Loss drops near zero fast; model useless. → Shift student logits, teacher logits, and labels identically. Suspect this if L_CE < 0.05 early.

C4. Temperature applied only to the teacher. Target is permanently flatter than anything the student can output; KD loss floors at a large value. → Apply τ to both, and multiply the term by τ².

C5. Missing τ² factor. Raising τ silently reduces the KD gradient ~τ²-fold; you think τ doesn't matter. → Always include it.

C6. Top-K renormalization bias. Renormalizing over top-K trains the student to be sharper than the teacher → over-confidence → more hallucination. → Use the residual bucket.

C7. Reverse-KL NaN. log(q/p) with p ≈ 0 outside top-K. → p.clamp_min(1e-8), or residual bucket, or use JSD instead.

C8. Teacher not in eval() / not under no_grad(). Dropout active → noisy targets; grads retained → 2x memory. → teacher.eval(); teacher.requires_grad_(False); wrap forward in torch.no_grad().

C9. Forgetting loss / grad_accum. Effective LR is grad_accum× too high → spikes/NaN.

C10. Equal-weighting micro-batches with unequal token counts. Short sequences over-weighted. → Accumulate sum_loss and sum_tokens, divide once.

C11. Loss spike from one pathological batch. Usually a maximally long sequence or a degenerate target. → Gradient clipping (always on) + skip batches whose loss exceeds running mean, and log the id.

C12. fp16 logit overflow. exp(z) overflows above ~65k. → Cast logits to fp32 before softmax; prefer bf16 training.

C13. Catastrophic forgetting. Student becomes a summarizer and loses everything else. → 5–15% general-instruction replay data in the mixture. Only matters if the product needs it.

C14. Mode collapse under reverse KL. All summaries become near-identical templates. → Watch diag/student_entropy; raise jsd_beta, mix in forward KL, or lower the KD weight.

C15. use_cache=True with gradient checkpointing. Wasted memory + warning. → model.config.use_cache = False during training, restore before generation.

C16. Resuming without dataloader state. The first N batches get trained on repeatedly after each restart, over-weighting them. → Save/restore the sampler state and epoch.

C17. Changing max_steps on resume with a cosine schedule. The LR curve silently changes shape.

C18. Multi-GPU changes the effective batch size. Same config on 4 GPUs = 4x the batch, so your LR is now effectively too small.

C19. Student and teacher disagree on attn_implementation. Small numeric differences in stored vs live logits. Minor, but makes debugging comparisons fail. → Record and pin it.

C20. Distillation from a teacher that is bad at your task. The ceiling is the teacher. If the teacher only wins 60% vs a human reference, no student will exceed that. → Measure the teacher first. If it's weak, improve the prompt or pick a better teacher before touching training code.


D. Quantization (details in docs/12)

D1. Calibration set from the wrong distribution (e.g. C4 for a Hindi legal summarizer). Scales are tuned for the wrong activations; quality drops far more than expected. → Calibrate on ~256–512 of your own held-out documents.

D2. Activation outliers. A few channels with 100x magnitude dominate the quantization range, crushing everything else. This is the reason naive INT8 fails on LLMs. → AWQ (per-channel scaling) or SmoothQuant (migrate outliers weight-ward).

D3. Quantizing lm_head on a tied-embedding model. Corrupts input embeddings too. → Keep lm_head in fp16; it's a small fraction of a 0.5B model anyway.

D4. group_size not dividing the hidden dimension. Kernel errors or silent padding. → 128 divides 896, 1536, 3584. Verify for any custom size.

D5. Long-context degradation after quantization. Quality is fine at 512 tokens, degrades at 8k, because errors accumulate over the sequence and RoPE-sensitive channels are quantization-sensitive. → Evaluate quantized models at your production context length, not a short benchmark.

D6. KV-cache quantization. Separate from weight quantization, much more damaging per bit. FP8 KV is usually safe; INT4 KV noticeably degrades long-context.

D7. Perplexity looks fine, generation is worse. PPL is a weak proxy — it's teacher-forced, so it never exercises error accumulation. → Always run the generation-based eval on the quantized model.

D8. Quantized model is slower. At batch size 1 with a small model you may be latency-bound on dequantization kernels rather than memory-bound. → Benchmark. INT4's win is largest for large models and memory-bound decode.

D9. Reordering (desc_act=True) in GPTQ. Better accuracy, but slower kernels and incompatible with some serving stacks. → Test end-to-end on the actual serving runtime before committing.

D10. Double quantization loss with QLoRA-trained adapters. Training a LoRA on a 4-bit base then merging into fp16 then re-quantizing compounds error. → Merge into fp16 and re-quantize once, then evaluate; or train on the fp16 base.


E. Product and operations

E1. The student is worse only for one customer segment. Aggregate metrics pass. → Per-slice gates.

E2. Style regression. Teacher used to write "• " bullets, student writes "- ". Nobody noticed until a customer's parser broke. → Format-compliance metric + a golden-set diff you actually read.

E3. Nondeterminism in production. Same document, different summary, user files a bug. → Greedy decoding + fixed seed for the customer-facing path if determinism is a requirement. Note that batching itself can change results slightly on GPU.

E4. Teacher upgraded, student didn't. Model drift; the student now underperforms a teacher it was never distilled from. → Version the pair together; re-distill on teacher upgrades.

E5. Prompt changed in production but not in training data. Silent quality drop. → prompt_version in every record, asserted at serving time.

E6. No rollback path. Keep the previous quantized artifact and a one-command rollback.

Next: 12 — quantization