[ Machine Learning ]

Discover insights, tutorials, and thoughts on technology, homelab, and development.

Dec 18, 2025 5 min read

I Was Wrong About Self-Improving Models: Here's What I Actually Found

A follow-up to my retracted post on self-improving models. The fidelity metric improved 18% but actual performance dropped 11%. Here's what went wrong, what I learned about Goodhart's Law, and why reproducibility filtering might be the answer.

self-improvement machine-learning goodharts-law

Dec 18, 2025 7 min read

When Models Talk the Talk but Don't Walk the Walk: A Journey into LLM Behavioral Consistency

We fine-tuned a security agent to 100% skill differentiation in probing tests, but it collapsed to a single behavior in deployment. This gap led us to develop a trust diagnostic framework.

llm fine-tuning behavioral-consistency