TL;DR
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Google says Gemini 4 Carbon scored highly on three cybersecurity benchmarks covering vulnerability discovery, exploitation and repair. The results are not directly comparable across labs, and only the repair score was produced by an outside benchmark operator; no independent public evaluation of Carbon is available.
Google says Gemini 4 Carbon scored 88.2% on a vulnerability-finding test, 54.8% on an exploit-generation test and 71.0% on a software-repair test. The results put the model forward as capable across three stages of security work, while Google says access is initially limited to vetted cyber defenders because the same capabilities could be used to attack systems.
The three scores measure different tasks. SEC-bench Pro tests whether a model can trigger known bugs in browser engines and, in its May 2026 version, the Linux kernel. Google reports Carbon’s 88.2% result across 183 tasks, averaged over three attempts using its Antigravity agent tool. OpenAI reported 85.4% for GPT-6 Astra on that version, while Gemini 4 Argon scored 83.4%. The benchmark gives models clues about the bug’s location and type, so it does not test discovery of an entirely unknown vulnerability.
ExploitGym starts with a crash and tests whether a model can turn it into an exploit that runs code and retrieves a hidden flag. Google’s headline result is 54.8% with a 10-hour limit. Its account also gives a 53.5% result under a six-hour limit. Anthropic reported 34.5% for Claude Opus 5.5 with six hours. These figures come from different labs’ reports, and the source material says the tools, graders and time limits are not consistent across comparisons. Google ran the tests with standard protections such as memory randomization disabled.
CWE-bench asks a model to inspect a codebase and repair flaws without being told where they are. A repair counts only if it blocks the attack and existing tests still pass. Collinear AI, which created the benchmark, ran the test on a private set of 120 tasks. It reported Carbon at 71%, three percentage points ahead of GPT-6 Astra, Gemini 4 Argon and Grok 4.7 at 68%, and Claude Opus 5.5 at 67%. The supplied account says five tasks defeated every model tested and that even the top result left nearly three in ten tasks unsolved on the first attempt.
Why the Scores Matter to Defenders
Carbon’s reported results describe a model that may assist with finding and validating software flaws, as well as preparing fixes. That could help security teams triage known weaknesses and test patches. The exploit result also shows why access controls matter: a system capable of turning a crash into working code could help defenders reproduce attacks, but the same capability could be misused.
The repair score has practical limits. A 71% benchmark result does not mean a company can safely accept most AI-generated patches without review. The benchmark has its own task set and pass conditions, and the supplied account says some tasks remained unsolved by all tested models. Organizations considering such tools would still need engineers to review changes and confirm that repairs work in their own environments.
For ordinary users, the announcement does not change immediate security steps. Automatic software updates remain the direct way to receive fixes from product makers. Google’s limited rollout also means the reported performance is not evidence that Carbon is generally available for consumers or businesses.
Products Worth Considering
LG UltraGear 27-Inch QHD OLED Gaming Monitor, 240Hz, 0.03ms, G-SYNC Compatible, FreeSync Premium Pro, HDMI 2.1
Samsung 27 Odyssey OLED G5 (G50SF) QHD QD-OLED Gaming Monitor
LG UltraGear 32GX850A-B 32-Inch 4K UHD Glossy OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Three Tests, Three Different Tasks
The benchmark results should be read according to what each test supplies to the model. SEC-bench Pro provides source code and information about a known bug, but withholds the input needed to trigger it. ExploitGym supplies a crashing input and tests whether the model can develop it into a working attack. CWE-bench gives a codebase and asks for an audit and repair, without identifying the flaw.
The tests also have different evaluators. Google ran Carbon’s SEC-bench Pro and ExploitGym evaluations, according to the source account. Collinear AI ran CWE-bench on its private test set. That means two of the three headline scores are Google-run results; the third comes from the benchmark’s creator. The account says there is no public independent evaluation of Carbon’s security performance.
Other labs have introduced restricted access programs for advanced cyber capabilities. The source material names Google’s Fairwind, Anthropic’s Cyber Verification Program and OpenAI’s Trusted Access for Cyber. These programs reflect a shared approach of giving certain capabilities to selected security users first. Google also says Carbon is part of voluntary pre-release testing by the U.S. government.
“Newer Claude results “are not available due to safeguard restrictions”.”
— The SEC-bench Pro public leaderboard
Limits of the Public Comparisons
No independent public test of Carbon’s security skills is available in the supplied material. Google produced two sets of scores, while Collinear AI produced the repair result. Comparisons with rival models also rely on figures reported by the rival labs, and the source account says methods, tools, graders and time limits differ. The scores therefore do not establish a like-for-like ranking across all three tasks.
The results also do not show how Carbon performs on unknown, real-world vulnerabilities or within a company’s live security workflow. The source describes tests based on known bugs, supplied crashes or a private benchmark set. It does not provide details about the size of Google’s vetted access group, its selection criteria, or when broader access might begin.
Access and Independent Testing
Google’s next stated step is to keep Carbon in a limited program for vetted cyber defenders, alongside its reported participation in U.S. government pre-release testing. The supplied source gives no public timetable for wider availability or for an independent evaluation. Further comparisons will be easier to assess if labs publish results using shared versions, time limits, tools and grading rules.
Until then, security teams weighing the model’s use can treat the reported scores as benchmark results with stated limits, rather than a guarantee of performance in production. Any use of generated fixes still calls for review and testing against the affected software.
Key Questions
What did Google report about Gemini 4 Carbon?
Google reported scores of 88.2% on SEC-bench Pro, 54.8% on ExploitGym using a 10-hour limit, and 71.0% on CWE-bench. The tests cover vulnerability triggering, exploit development and software repair.
Are the scores independently verified?
The supplied source says there is no public independent test of Carbon’s security skills. Google ran the SEC-bench Pro and ExploitGym evaluations; Collinear AI ran CWE-bench.
Why is access to Carbon limited?
The same capabilities that can help defenders identify and reproduce software vulnerabilities could also be misused. Google says it is initially providing access to a small group of vetted cyber defenders.
Does the 71% repair score mean Carbon can patch software safely?
No. It is a result on Collinear AI’s private benchmark, where a repair must stop the attack and preserve existing tests. The source account says the top model still failed nearly three in ten tasks on its first try, so real-world changes need review and testing.
Source: Thorsten Meyer AI
