Skip to content
Spellwright News

Benchmarks

AutomationBench

AutomationBench

Other · higher is better · 1 model
  1. Gemini 4 Argon51.3%*

Every result

  • Gemini 4 Argon51.3%Reported by the model's maker (Google), Sep 30, 2026

    Google reported 51.3% for Gemini 4 Argon on AutomationBench in its September 30 introduction. Evaluation details are incomplete.

    Limits: Google reports the result; it has not been independently reproduced here. The report predates the October public API launch. The evaluated checkpoint and its equivalence to the currently served model are unknown. The task subset, sample size, harness, prompts, tools, reasoning/token budgets, run date, aggregation and confidence interval are unspecified. Model context capacity is not an evaluation budget. Evaluation safeguards and routing are unspecified. Settings reported for other models or later releases cannot fill those gaps. These are reported results, not a controlled cross-model ranking or proof of real-world reliability, speed, cost or statistically meaningful differences. The benchmark owner is Zapier; the inspected Google report does not establish that Zapier ran or verified this evaluation.

    Google

No published AutomationBench score checked yet for: Claude Sonnet 5.5, Claude Opus 5.5, Claude Opus 5, Claude Sonnet 5, Gemini 2.5 Pro Experimental (March 2025), Gemini 2.0 Flash (December 2024 experimental), Qwen2.5-7B, Qwen2.5-7B-Instruct, GPT-4o mini (July 2024 release), GPT-4o (May 2024 release), Claude 3 Haiku, Claude 3 Opus.