Diagnose poor results

Start with the symptom. Is the output still noise, repetitive, copied from data, off-topic, or grammatically plausible but incoherent?

Then check the evidence:

• If loss never improves, check data loading, tokenization, and the run configuration.
• If training improves but validation worsens, stop near the best validation checkpoint or change capacity and data.
• If loss improves but samples remain weak, compare prompts and sampling settings before retraining.
• If output copies long passages, inspect duplication and train/validation separation.

Add two checks before spending more compute:

• Decode a batch of training tokens. If the text is broken, training longer will not repair the input pipeline.
• Ask the model to overfit a tiny batch in a disposable run. If loss cannot fall there, suspect the objective, shapes, learning rate, or optimizer path.

Use this evidence table:

Symptom	First measurement	One possible next test
Loss is NaN	First bad step and batch	Lower learning rate or inspect invalid values
Both losses stay flat	Gradient and learning-rate behavior	Tiny-batch overfit test
Validation rises	Best validation checkpoint	Stop earlier with the same run
Repetition only in samples	Token probability trace	Change one sampling control
Memorized passages	Nearest training matches	Deduplicate and rebuild the split

Choose one main change: cleaner data, more appropriate data, a different run length, or a revised blueprint. Give the new run a new identifier.

Keep the failed run. It is the comparison that explains why the next decision exists.

Match the fix to the bottleneck. More parameters can worsen overfitting. More data can waste compute when tokenization is broken. More steps can amplify memorization after validation stopped improving.

State the expected result before running the change. “Lower the learning rate to prevent the early loss spike” is testable. “Try different settings” is not.