The 66% Problem

The text says that AI assistants generate a new kind of problem: code that looks correct, passes tests, and still introduces subtle errors. The author treats this as a greater risk than obvious failures, because the cost appears later, when it is necessary to debug what the AI assumed without asking for confirmation.

Reading notes

#

• The author says he spent three hours chasing a bug that, in the end, did not even exist; the real problem was a function added by Claude for pagination, although that had not been requested.
• He uses this to define the “66% problem”, linked to the frustration of “almost right” code and the extra work of debugging code generated by AI.
• Totally wrong code fails clearly, while almost right code can pass tests, go to staging, and cause subtle failures later.
• For the author, AI-generated bugs are harder because, when he finds a problem, he needs to discover what the AI did without noticing.
• The text cites a Microsoft Research study with nine models and the SWE-bench Lite benchmark; the best model solved 48.4% of the tasks, less than half.
• The author distinguishes generating code from debugging code: generating is completing patterns, but debugging requires hypothesis, understanding what the system should do, and why it differs.
• AI does not know old business rules or architectural particulars that shape the system, so it tends to only recognize patterns.
• The text cites a randomized trial by METR, from July 2025, in which experienced developers were 19% slower with AI, although they thought they were 24% faster.
• The author says the perceived productivity gain can hide accumulated debugging debt.
• He does not argue for abandoning these tools, but for reading each suggestion more cautiously, as if it came from a confident junior.
• The conclusion is that the tools are good enough to be dangerous, and that continuing to use AI requires maintaining strong testing and debugging habits.