Member-only story
CLAUDE.md Is Failing Your Agent. State Of The Art Recommendations.
A new study shows context files can actually impair coding agents. The results should change how you work.
Lo Zarantonello4 min read·Just now--
Running /init was a good practice. Now forget about it.
A new study from ETH Zurich is questioning the usefulness of context files like AGENTS.md and CLAUDE.md.
They tested three conditions:
- no context file at all,
- an LLM-generated file (the kind you get from
/init), - a human-written file crafted by the actual developers of the repo.
The results weren’t what anyone expected.
TLDR: Context files impair your agent
The Numbers
According to the study, LLM-generated context files like CLAUDE.md and AGENTS.md decreases task success rates by ~3% compared to providing no file at all.
And on top of that, they increased inference cost by over 20%!
Congrats, we just automated paying more to get less.
Humans-written context files
Human-written context files did just slightly better.
