DeepSWE 1.1

Every result

• Gemini 4 Argon77.9%Reported by the model's maker (Google), Sep 30, 2026

  Google reported 77.9% for Gemini 4 Argon on DeepSWE 1.1 in its September 30 introduction. Evaluation details are incomplete.

  Limits: Google reports the result; it has not been independently reproduced here. The report predates the October public API launch. The evaluated checkpoint and its equivalence to the currently served model are unknown. The task subset, sample size, harness, prompts, tools, reasoning/token budgets, run date, aggregation and confidence interval are unspecified. Model context capacity is not an evaluation budget. Evaluation safeguards and routing are unspecified. Settings reported for other models or later releases cannot fill those gaps. These are reported results, not a controlled cross-model ranking or proof of real-world reliability, speed, cost or statistically meaningful differences.

  Google

• Gemini 3.8 Flash73.7%Reported by the model's maker (Google DeepMind)

  Google DeepMind reports 73.7% on DeepSWE v1.1 for Gemini 3.8 Flash, with results described as of September 2026. The exact evaluated build and full configuration remain unknown.

  Limits: Provider-reported own-model result; the actual execution runner and scorer are not identified. No independent replication, overall ranking or matched-budget comparison is established. The source identifies the nominal Gemini 3.8 Flash release, not an exact evaluated checkpoint. Equivalence to the currently served API build and its routing or fallback configuration is unknown. Task subset, sample size, trial count, harness, prompting, tool mode and budget, reasoning effort, token budget, safeguards, aggregation, pass-k and uncertainty are not established by the inspected HTML. Medium is the API product default; it is not established as the effort used for this result. Settings from other benchmarks and models are not transferred. The source page is dated September 2, 2026 and describes results as of September 2026. The exact evaluation/run date and exact date on which this scalar was added are unknown. A shared benchmark name and percentage unit with another provider’s report do not establish matched methods or justify score-gap claims.

  Google DeepMind · Google