Verified: what changes when a third party measures

On 13 October, Gemini 4 Argon reached general availability on the Gemini API, with a model card published the same day: a sparse mixture-of-experts transformer, a new model and not a fine-tune of another, taking text, image, audio and video in and returning text, with a one-million-token window and knowledge up to March 2026. The price follows what was announced in September — US$ 2 and US$ 10 until 30 November, double from December —, with no free tier.^1,2,3

The same day, the ARC Prize published its independent verification of the model on the semi-private set. On ARC-AGI-3, Argon scored 97.5% with the Provider Adapter — the harness that keeps reasoning state across requests and, in its case, uses context management designed by Google — and 49.8% with the standard harness. On ARC-AGI-1, 98.5%; on ARC-AGI-2, 91.2%.⁴ Figure 1 places it beside GPT-6 Astra, verified in September.

Two readings. First: on the harness each lab optimizes for its own model, both are at the ceiling, and the 2.4-point gap says almost nothing. Second: on the common harness, Argon sits 13 points below Astra — but with runs that did not finish, which the ARC Prize warns probably underestimates the model. Independent verification did not settle the harness question; it put it in plain sight, with the numbers and the caveats side by side. That is what one can ask of a third party.

On 19 October, Vals AI published its Terminal-Bench 4.0 leaderboard, with a single agent for every model (mini-SWE-agent), three runs per task and up to eight hours per task. Opus 5.5 leads at 65.15%, followed by Sonnet 5.5 (64.14%), GPT-6 Astra (59.60%), Fable 5.1 (58.08%) and Gemini 4 Argon (57.58%).⁶ Artificial Analysis, which also runs everything on its own harness, shows a different order on its evaluation page: Sonnet 5.5 at 63.6%, Opus 5.5 at 59.6%.⁷ The official leaderboard received no new entries in the week.⁸ Figure 2 brings the four sources together.

The practical question — which of the two to use in a terminal agent — has a different answer by source: Opus on the official leaderboard and at Vals, Sonnet at Anthropic and at Artificial Analysis. The gap between the two never exceeds six points, and in the two third-party measurements it is one and four. What the figure shows most clearly is each model's spread: Opus 5.5 runs from 59.6% to 65.15% depending on who measures; Sonnet 5.5, from 61.8% to 70.6%. Harness differences — the agent, the time per task, network access, the fallback policy — weigh as much as the difference between models.

For the panel, the ceiling stays at Sonnet 5.5's self-reported 70.6%, by method. The week's highest independent number is Vals's, 65.15%, and it goes alongside. Seven editions in, this indicator's lesson is that an agent-benchmark ceiling without the name of who measured it is a number without a unit.

On 17 October, DeepSeek launched V4.1-Flash-1017, with weights on Hugging Face, a one-million-token window, output of up to 384 thousand and vision. It began answering to the name deepseek-flash on the website, the app and the API, and the old names — deepseek-v4-flash and deepseek-v4-flash-vision-exp — are served by it too. According to the company, the calling method and the price did not change.¹⁰ The weights' license was not confirmed at closing.

In no. 2, the problem was a fixed name starting to point at a different model. This time the swap is for the better: Artificial Analysis measured 1017 at 45 on the index at maximum effort, against 39 for September's V4.1-Flash — six points more, at the same price, with nobody changing a line.¹¹ It is the same mechanism with the sign flipped, and no. 2's advice holds just the same: whoever runs their own evaluation against the name they use in production saw the change on the 17th; whoever does not finds out from the agent's behavior. 1017 sits one point behind MiMo-V2.6-Pro among open models — and, with no confirmed license, off the panel.

The same notice brings a new line on the bill: from 31 October, at 16:00 UTC, the API's search and page-fetch tools will cost US$ 0.75 per thousand calls.¹⁰ It looks small next to the token price. Code 1 shows when it is not.

# deepseek-flash a partir de 31/10: quanto pesa a busca na conta de uma tarefa de agente (US$, horário de pico)
entrada_cheia, entrada_cache, saida = 0.30, 0.006, 1.20   # US$ por milhão de tokens
busca = 0.75 / 1_000                                       # US$ por chamada de Search/Fetch

def tarefa(tokens_in=200_000, tokens_out=8_000, acerto=0.0, buscas=10):
    tokens = (tokens_in * ((1 - acerto) * entrada_cheia + acerto * entrada_cache) + tokens_out * saida) / 1e6
    return tokens, buscas * busca

print(f"{'cache':>6}{'buscas':>8}{'tokens':>9}{'busca':>8}{'busca/total':>13}")
for acerto in (0.0, 0.9):
    for buscas in (5, 10, 20):
        t, b = tarefa(acerto=acerto, buscas=buscas)
        print(f"{acerto:>6.0%}{buscas:>8}{t:>9.4f}{b:>8.4f}{b / (t + b):>13.0%}")

 cache  buscas   tokens   busca  busca/total
    0%       5   0.0696  0.0037           5%
    0%      10   0.0696  0.0075          10%
    0%      20   0.0696  0.0150          18%
   90%       5   0.0167  0.0037          18%
   90%      10   0.0167  0.0075          31%
   90%      20   0.0167  0.0150          47%

Without cache, ten searches are 10% of the task's bill. With a 90% cache hit rate — what no. 3 recommended —, the token share falls to a quarter, and the same ten searches become 31%; twenty searches, almost half. The cheaper the token gets, the more what is not a token shows on the invoice. A research agent that runs many short searches needs to count tool calls as carefully as it counts tokens.

On 15 October, Anthropic took Opus 5.5's fast mode out of preview: there is no longer any need to request access or send the beta header, and it has a default rate limit. It costs US$ 8 input and US$ 40 output, twice the normal price. Fast mode for Opus 5 and Opus 4.8 remains in preview.¹³ On 14 October, xAI put Grok Voice 3.0 on its voice API at US$ 0.08 per minute, the previous version's price; it is the model that now leads Artificial Analysis's voice index.¹⁴ And on the edition's day, OpenAI began offering cache writes valid for twelve hours, at US$ 5 per million tokens on 6.1 Sol, against US$ 2.50 for thirty minutes.^15,16 Table 1 sums up what each change does to the bill.

The twelve-hour cache is the most useful change for agents. With a thirty-minute validity, an agent that sits idle over lunch loses the prefix and pays full input on returning; with twelve hours, it writes the prefix once in the morning and reads it at 5% of the price until the end of the day. Writing costs twice as much, and only pays off if the prefix is read many times — which, in an agent, is the rule. It is OpenAI's answer to the problem no. 3 described: the cache decides the bill, and a cache that expires too early is money lost without anything in the code looking wrong.

Days after the second, ANPD published the third partial report of its AI regulatory sandbox, on test cycle 3, run with USP's Ciaam.^17,18 For the first time there are test results, not just documentation. Table 2 sums up the three companies.

Prevvine's row is the one that matters outside the sandbox. Thirty simulated clinical cases, spread across patient groups, found the failures model benchmarks do not measure: invented answers, citations of articles that do not exist, links that do not open. And the agency did not accept the vignette spreadsheet as proof of clinical performance or of absence of bias, because the results were not consolidated.¹⁷ It is a domain yardstick, built by a Brazilian regulator with a Brazilian university, and it is more demanding than any leaderboard cited in this series. Whoever builds AI for health in Brazil has here the best public model of what will be asked of them.

The rest of the week confirmed the deadlines. PL 2338/2023 reached seven weeks without movement in the Chamber; the last action is from 2 September.¹⁹ The Macaíba supercomputer session remains set for 4 November, at 10:00.²⁰ The RHAE IA result is scheduled for December. The PBIA indicator closes the seventh edition at R$ 1.009 billion — the same value since August.