Gemini 3.5 Flash performs better than Gemini 3.1 Pro on coding and agentic tasks.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2026-05-20 using Google/DeepMind's published benchmark tables (model card or launch post) and independent leaderboards. True: on the coding benchmarks (e.g. SWE-bench Verified, LiveCodeBench, Terminal-Bench) and agentic benchmarks (e.g. tau-bench, tool-use/agent evaluations) reported at launch, Gemini 3.5 Flash scores higher than Gemini 3.1 Pro on all or nearly all of them. Mostly true: it leads on clearly most of them, trailing slightly on one or two. Mixed: it leads on roughly half. Mostly false: it leads on only a few, or only in Google's own chosen benchmarks while independent evaluations show it behind. False: it scores at or below 3.1 Pro on most coding and agentic benchmarks, or no Gemini 3.5 Flash model was released and evaluated.
Set before the research began.
Evidence (3)
-
E1 Gemini 3.5: frontier intelligence with action Google (blog.google) · 19 May 2026 Own record
“It’s our strongest agentic and coding model yet, outperforming Gemini 3.1 Pro on challenging coding and agentic benchmarks like Terminal-Bench 2.1 (76.2%), GDPval-AA (1656 Elo) and MCP Atlas (83.6%), and leading in multimodal understanding (84.2% on CharXiv Reasoning).”
-
E2 I/O 2026: Welcome to the agentic Gemini era Google (blog.google) · 19 May 2026 Own record
“When compared to 3.1 Pro, 3.5 Flash is better across almost all benchmarks. It’s made huge progress in coding — and look at the extraordinary jump in GDPVal.”
-
E3 Gemini 3.5 Flash: LayerLens Evaluation Data Reveals Where Google's Fastest Model Actually Wins LayerLens · 21 May 2026
“The Pro model still wins on Big Bench Hard by 4.33 points and LiveCodeBench by 3.34 points, which makes a real difference for teams with hard reasoning and coding requirements.”
Reasoning
Google's launch post (E1) says Gemini 3.5 Flash outperforms Gemini 3.1 Pro on coding and agentic benchmarks, naming Terminal-Bench 2.1, GDPval-AA and MCP Atlas. Sundar Pichai's I/O post (E2) says it is better than 3.1 Pro on almost all benchmarks. LayerLens's independent evaluation (E3) finds that 3.1 Pro still leads on LiveCodeBench by 3.34 points, so 3.5 Flash does not lead on every coding benchmark. Together, the evidence shows 3.5 Flash ahead on most of the reported coding and agentic benchmarks and slightly behind on at least one.
Cite this claim
Demis Hassabis, “Gemini 3.5 Flash performs better than Gemini 3.1 Pro on coding and agentic tasks.” (said 20 May 2026). Verdict: mostly true. Reliability API, claim c_165a390330c8, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_165a390330c8
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Demis Hassabis
-
Mostly true
“It can take photos, video & audio and build entirely new scenes.”
Gemini Omni can take photos, video and audio and build entirely new scenes.
-
Mostly true
“I didn’t feel like that was proper AI because that system, let’s take Deep Blue, okay, it’s world champion level at chess, at chess, but it can’t do anything else.”
Deep Blue was world-champion level at chess but could not do anything else.
-
Resolved true
“And it uses AI to actually digitally watermark any generated image.”
-
Mostly true
“They still can't really yet go beyond asking a new novel question or a new novel conjecture or coming up with a new hypothesis that has not been thought of before.”
-
Mostly true
“they certainly have modeled enough of the dynamics, put it that way, that they can pretty accurately generate whatever it is, eight seconds of consistent video”
Verdicts are made automatically; how verdicts are made.