trampolim.net
PT EN

Trampolim · Technology weekly

The Week in Tech

Issue 01Week of April 20–26, 202616 stories
AI

Benchmark numbers come with comparisons chosen by whoever launches

The launch material listed its own results beside lower numbers for competitors.

Benchmark numbers come with comparisons chosen by whoever launches
AI · April 20–26, 2026

The GPT-5.5 launch material carried its own benchmark results, including 82.7% on a terminal task test and 51.7% and 35.4% on distinct tiers of a mathematics test, alongside lower numbers attributed to competing models on the same tests.

A comparison published by whoever launches is useful and partial at once. It's useful because it gives a verifiable reference; it's partial because the choice of which test to show belongs to the party with an interest in the result.

There's also the configuration question. A model can be run with different parameters, with more or fewer reasoning steps, and the same benchmark yields different results depending on the setting, which is rarely detailed.

That doesn't mean the numbers are wrong, only that they answer a specific question: how does this model perform on this test, in this configuration, chosen by the publisher.

The question that matters to buyers is different and nobody answers it from outside: how does this model perform on the task my operation runs, with my data, counting total spend until the result is finished.

Book a call