trampolim.net
PT EN

Trampolim · Technology weekly

The Week in Tech

Issue 05Week of May 25–31, 202616 stories
AI

Coding numbers rise five points between versions

The gain from 64.3% to 69.2% describes the category's current pace.

Coding numbers rise five points between versions
AI · May 25–31, 2026

The difference between coding model versions launched months apart, from 64.3% to 69.2% on a real repository benchmark, describes the category's pace well.

Five points sounds small and isn't. In that range of the benchmark, the remaining problems are the hardest: code with scattered context, tests depending on external state, and fixes requiring understanding why something was done a particular way.

That's why marginal gains get more expensive with each version. The easy cases are solved, and every additional point requires capability that also makes execution costlier.

The usual caveat applies to numbers published by whoever launches, particularly when accompanied by a lower result attributed to a competitor on the same benchmark.

For anyone deciding a purchase, what changes the choice isn't the difference between 64 and 69 on a public benchmark. It's how much each model resolves of the operation's specific task, counting total spend until the result is finished.

Book a call