On this page · 5 sections ▾
AI labs measure cybersecurity skill by running a model through standardized tests, each covering one stage of handling a software flaw: finding it, exploiting it, or patching it. Google's new Gemini 4 Carbon was announced with a score for each stage. The numbers allow rough comparisons between models, but they describe test performance, not how a model behaves in a real security team.
What did Google actually report?
As 9to5Google reported, Google cited three external benchmarks for Carbon: 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching. The Verge describes SEC-bench Pro as a test of a model's ability to discover real-world bugs, and ExploitGym as a measure of how well a model turns a vulnerability into an exploit.
Those three tasks mirror a security workflow. A defender needs to locate a weakness, understand whether it can be abused, and then repair it without breaking the software.
How do these benchmarks work?
A benchmark is a fixed set of problems with known answers or checkable outcomes. The model gets each problem, produces a result, and the test scores how often it succeeds. Because every model faces the same problems, the percentages can be compared side by side.
In general, finding a flaw means reading code and spotting where it can fail. Exploitation tests whether the flaw can actually be used to cause harm. Patching tests whether the model can write a fix. A weakness class label such as CWE, a common way to categorize software flaws, helps organize the patching problems.
The three scores are not on one scale. A 54.8% on exploitation and an 88.2% on finding bugs measure different, differently hard tasks, so they should not be compared with each other.
Why are the numbers hard to read on their own?
Most of these figures come from Google itself. Ars Technica points out that every frontier model scores in the same neighborhood on these tests, and that the gap between Carbon and its competitors is only a few points. It also notes that Carbon beats its predecessor Argon by slim margins in every test Google gave it.
Digital Trends adds that Carbon still trails rivals on a couple of specific coding benchmarks. A leaderboard lead is therefore not the same as a win everywhere.
