AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: OpenAI Is Training Agents Inside Software. What Should You Read In Ironclad’s Terms? on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on monitors, keyboards and dev gear

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

TL;DR

OpenAI’s Oct. 6 post describes training a frontier model in hosted copies of Ironclad’s contract-management software, using 11 legal, commercial and procurement tasks. GPT-6 Astra met an average 55% of task criteria, while estimated completion times were simulated; the results do not show that the agent is ready to handle contract workflows without human review.

OpenAI said on Oct. 6 that it trained its GPT-6 Astra model using tasks inside hosted copies of Ironclad’s contract-management software, a test of whether AI agents can follow business rules in specialized applications. Across 11 legal, commercial and procurement tasks, Astra met an average 55% of the evaluation criteria; OpenAI says the reported time estimates are simulated, not measured customer savings.

Ironclad staff and OpenAI employees who use the product selected 11 tasks, including setting up nondisclosure agreements, creating procurement approval processes and updating reusable contract clauses based on a requester’s jurisdiction. OpenAI estimated an experienced user would take 30 to 40 minutes on each task. Evaluators scored each task against between 8 and 50 criteria, depending on complexity.

OpenAI reported that GPT-6 Astra met an average 55.0% of criteria, compared with 41.6% for GPT-5.6 Sol at its high setting. An internal model used during Astra’s development reached 63.7%. Astra met about 94% of the criteria on one showcased task, but that result does not represent its average performance across the test.

OpenAI said Ironclad provided hosted product environments for the model to practise in. It says training tasks were generated synthetically from publicly filed contracts in the SEC’s EDGAR database, with personal information filtered out. OpenAI also says it did not use its customers’ data, its internal contracts or non-public Ironclad customer data. Those statements describe the data used in this project, not a general policy covering every model or partnership.

At a glance
reportWhen: Published Oct. 6; further software-comp…
The developmentOpenAI described a partnership with Ironclad to train and evaluate a frontier model on workflows inside contract-management software.
OpenAI × Ironclad — Insights
AI Dispatch · Insights · 7 October 2026

OpenAI is training agents inside your software. Read the fine print on Ironclad.

Several AI trackers guessed “Ironclad” was a hardened agent framework. It’s a contract-management software company — and the post describes OpenAI training its frontier model inside a vendor’s real product, then inviting other vendors to do the same.

What they did
Tasks
11

legal, commercial & procurement — e.g. NDAs, approval flows, jurisdiction clauses

Human time
30–40m

per task, experienced user (OpenAI estimate)

Grading
8–50

criteria per task — a rubric, not pass/fail

Training data
EDGAR

public SEC filings; no customer or non-public Ironclad data

The results — and what the footnotes say
GPT-5.6 Sol (high) · criteria met41.6%
GPT-6 Astra (max) · criteria met55.0%
Internal model · criteria met63.7%
What “55%” means

The average share of rubric criteria met — not tasks completed. In contracting, partial credit isn’t partial value: a workflow that skips one required approval is the exact failure the system exists to prevent.

The time numbers are simulated

37.0 → 19.2 minutes are “simulated estimates … not measured customer time savings,” per OpenAI’s own footnote. Credit to OpenAI for saying so plainly.

~20 simulated minutes, ~half the criteria, and a human checks every requirement — vs 30–40 minutes for an expert done right. For now, the human is still the faster route to a correct workflow. The trend is the story.
The bigger story: software vendors as training grounds
Upside for the vendor

Its hardest customer problems get built into the next frontier model; agents that work well in its product make the product more valuable.

Risk for the vendor

Every improvement makes the model better at operating the vendor’s interface. Taken far enough, the agent becomes the interface.

The post frames it as showing why “a full contracting platform remains essential.” Winners will be vendors whose value is in rules, records and controls — not the screens an agent learns to click.
Five questions before letting agents into your systems of record
Which criteria failed?

Averages hide missed approvals.

What permissions?

Narrowest access; no self-escalation.

Tamper-proof logs?

METR found agents spoofing tool-call records.

Who checks, how long?

Measure the whole loop.

Whose training data?

Public filings, not your contracts.

The take

Modest numbers, significant method. A frontier lab is moving from general computer use to training inside specialised business software, with the vendor’s help — agents learning their trade the way people do. Today: just over half of a contracting workflow’s requirements, in simulated time, on 11 research tasks.Software vendors are becoming training grounds for the agents that may one day operate their products for them.

Source: OpenAI, “Advancing computer use with Ironclad” (6 Oct 2026) — tasks, criteria, EDGAR training data, 55.0% vs 41.6%, 19.2 vs 37.0 simulated minutes, 63.7% internal model, simulation footnote, collaboration invitation. Mischaracterisations of “Ironclad” in automated AI-news trackers (7 Oct 2026). METR investigation as covered here. Analysis is the author’s.
thorstenmeyerai.com

Why Partial Scores Matter in Contracts

The results point to a potential way to teach agents how to operate professional software, but the score needs careful interpretation. 55% is the average share of criteria met, not the proportion of tasks completed successfully. OpenAI’s evaluation does not show that Astra can reliably finish a contract workflow from beginning to end.

That distinction matters when a workflow contains approvals and safeguards. A procurement process might require Finance approval above a spending threshold, Security review for certain requests and Legal review for nonstandard terms. Missing one requirement could let a purchase bypass a control. In that setting, a partially correct workflow can still create a consequential failure. OpenAI’s post acknowledges that losing track of a business rule limits what an agent can safely be asked to do, and says human oversight remains important.

The project also has implications for software companies. Working with model developers could help them test agents on difficult, product-specific tasks. But as agents become better at using an application, customers may interact with the agent rather than the software’s screens. A vendor’s lasting value may depend more on its business rules, records, audit trails and controls than on its interface. That is an implication of the development, not a demonstrated outcome of this test.

Products Worth Considering

EcoFlow Delta 3 Max Portable Power Station with 220W Solar Panel, 2048Wh

EcoFlow Delta 3 Max Portable Power Station with 220W Solar Panel, 2048Wh

Jackery Explorer 300 Portable Power Station with 40W Solar Panel

Jackery Explorer 300 Portable Power Station with 40W Solar Panel

Anker SOLIX C300 DC Portable Power Station with 60W Solar Panel, 288Wh, 300W

Anker SOLIX C300 DC Portable Power Station with 60W Solar Panel, 288Wh, 300W

As an affiliate, we earn on qualifying purchases.

How Ironclad Became the Test Environment

The OpenAI post, titled “Advancing computer use with Ironclad,” describes a collaboration with a contract-management software company. The project is not, as some trackers inferred from the name, the launch of a new “Ironclad” agent framework. Its focus is training and evaluating a model within a vendor’s product and workflows.

OpenAI says the tasks were selected by people familiar with Ironclad and the work its users perform. The evaluation used criteria tailored to each task, rather than a single pass-or-fail measure. That provides a way to inspect whether requirements were met, but the reported average can conceal which particular rules were missed. The post presents one task with a score near 94% alongside the lower overall average, so the showcase should not be treated as representative performance.

The post also reports estimated time per attempt: 19.2 minutes for Astra compared with 37 minutes for GPT-5.6 Sol. OpenAI’s footnote says those figures are simulated estimates based on assumed processing and generation speeds. They are not observed time savings for Ironclad customers, and the 11 research tasks do not establish performance across Ironclad workflows generally.

What the Evaluation Does Not Establish

The post does not establish how Astra would perform across the full range of Ironclad customer workflows, or whether the model can consistently meet every requirement in production. The 55% average criteria score does not identify, in the source material provided, which requirements were most often missed or how severe those misses were. A single task near 94% also cannot resolve that uncertainty.

It is also unclear whether the simulated time estimates would translate into faster work once people review outputs, correct errors and verify that approvals and other controls were followed. OpenAI says human oversight remains necessary. No customer deployment results, independently assessed findings or measured return on investment are provided in the source material.

OpenAI says it used no non-public Ironclad customer data for this research. The post’s description does not settle what data arrangements, safeguards or evaluation standards would apply in any future partnership, or how broadly the approach will be adopted.

The Next Test Is Controlled Deployment

OpenAI says it is inviting a small number of software companies to work on tasks current agents cannot reliably complete. It asks prospective partners to bring a concrete example of failure, people with detailed knowledge of the work, a secure test environment and data that can safely be used for research. The post does not name additional partners or give a schedule for further evaluations.

For companies considering agent access to contract, finance or customer systems, the next useful evidence would be task-level results: which criteria were missed, what the consequences could be, how often human intervention is needed and whether performance holds across varied cases. Until those details and measured results are available, the Ironclad project is best read as a research demonstration of a training approach—not proof that agents can independently run high-consequence workflows.

Key Questions

What is Ironclad in this OpenAI announcement?

Ironclad is a contract-management software company. OpenAI’s post describes training and evaluating a model on tasks inside hosted copies of its product, not the launch of a new agent framework called Ironclad.

What does Astra’s 55% score mean?

It is the average share of evaluation criteria met across 11 tasks. It is not the share of tasks completed successfully, and it does not mean the agent can safely handle 55% of contract work without review.

Did Astra save customers time?

The source does not report measured customer savings. OpenAI labels the 19.2-minute estimate as simulated, based on assumed processing and generation speeds; the estimates cover the research tasks.

What data did OpenAI say it used?

OpenAI says it created synthetic tasks from publicly filed contracts in the SEC’s EDGAR database and filtered out personal information. It says the project did not use OpenAI customer data, OpenAI internal contracts or non-public Ironclad customer data.

Can companies let agents run contract workflows now?

The reported results do not establish that Astra is ready to run such workflows independently. OpenAI says human oversight remains important, and the source provides no customer deployment results showing reliable, unattended performance.

Source: ThorstenMeyerAI.com

You May Also Like

How to Achieve 3x Higher Sign-Up Completion with Multi-Step Forms

Discover how breaking forms into multiple steps triples completion rates. Learn practical tips to design engaging, high-converting multi-step forms today.

17 Organizations Using AI To Accelerate Scientific Breakthroughs

Google.org named 17 organizations for $36 million in AI-for-science grants, supporting projects in climate, energy, health and biology.

Is AI The Key To Faster Product Launches? Stampli’s 68% Time Cut With ChatGPT

OpenAI reports Stampli reduced launch hours by 68% with ChatGPT, though details about the measurement and scope remain undisclosed.

QuantLase Tests Photonic Computing Platform On Financial Market Data

QuantLase says it tested its PIPU photonic computing platform on seven financial market datasets, with no numerical results yet showing an advantage.