📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems now demonstrate near-complete automation of core engineering tasks in AI R&D, with some research aspects still requiring human creativity. This shift could accelerate AI progress but raises questions about the future role of human researchers.
Recent developments in AI capabilities demonstrate that AI systems can now automate the majority of core engineering tasks involved in AI research and development. While research activities that require creative insight remain less automatable, the progress in engineering automation is significant enough to potentially transform the AI research landscape. This shift is based on empirical benchmarks and recent industry demonstrations, marking a pivotal moment in AI’s evolution.
According to Thorsten Meyer’s analysis of Jack Clark’s recent essay, six key benchmarks measuring AI’s ability in core AI R&D skills have shown rapid progress, with several nearing or reaching saturation levels. For example, the CORE-Bench, which evaluates research reproduction, has improved from 21.5% in September 2024 to 95.5% in December 2025, with the benchmark’s author stating it is ‘solved.’ Similarly, the MLE-Bench, assessing Kaggle competition performance, moved from 16.9% to 64.4% in roughly 16 months, indicating AI systems can now perform at a mid-tier human level in competitive machine learning tasks. Industry demonstrations also include advances in automated GPU kernel design, with models generating optimized code for production infrastructure, signaling that engineering tasks are increasingly handled by AI. Clark’s conclusion emphasizes that AI can automate a vast portion of engineering work, but the residual challenge remains in automating research activities that involve creative problem-solving and hypothesis generation. The key insight is that engineering is now largely automated, while research—particularly the innovative and conceptual aspects—remains less so, though this gap may close faster than previously thought.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
AI Workflow Automation for Bloggers: Build a Simple Content System to Research, Write, Optimize, and Repurpose Posts Faster with AI and No-Code Tools (AI Toolkit for Bloggers 2026 Book 8)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
automated GPU code generator
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
AI Engineering: Building Applications with Foundation Models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
Writing With AI: A Practical Guide to Tools, Workflows, and Writing in Your Own Voice (AI For Creative Professionals)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Why Automated Engineering Shifts AI Development Speed
This trend toward automation in AI engineering could dramatically accelerate the pace of AI development, reducing costs and timeframes for deploying new models. It also raises questions about the role of human researchers, as many routine and even complex engineering tasks become AI-managed. However, the residual research component—requiring creativity, hypothesis testing, and theory development—still depends on human insight. The potential for AI to automate both engineering and research at scale might lead to a paradigm shift, where the bottleneck moves from technical capability to strategic and conceptual innovation.
Recent Empirical Evidence of AI Progress in R&D Skills
Over the past year, multiple benchmarks and industry demonstrations have shown rapid advancements in AI capabilities relevant to research and engineering. The CORE-Bench, which measures research reproduction ability, has seen a 4.4× improvement, and the MLE-Bench, evaluating Kaggle competition performance, has improved nearly threefold. Industry examples include models generating GPU kernels and converting code between frameworks, signaling that AI is reaching production-grade levels in engineering tasks. These developments follow a consistent pattern of rapid progress across different skill domains, suggesting that the automation of engineering tasks is nearing completion. Meanwhile, the question of whether AI can fully automate research activities remains open, with some experts arguing that creative problem-solving may be less automatable but could also be impacted by ongoing AI advancements.
“The pattern across multiple benchmarks indicates that AI is approaching saturation in core engineering skills, making research the remaining frontier.”
— Thorsten Meyer
Unresolved Questions About AI’s Research Capabilities
While engineering tasks are increasingly automated, it remains unclear how much of the research process—particularly hypothesis generation, creative insight, and conceptual breakthroughs—AI can automate. The structural question Clark leaves open is whether research is fundamentally distinct from engineering or if it can be subsumed under engineering at scale. The pace at which AI might close this residual gap is uncertain, and some experts suggest that creative and strategic elements may always require human input, at least for the foreseeable future.
Next Steps in Monitoring AI Automation Progress
Researchers and industry leaders will likely focus on further benchmarking to measure AI’s capabilities in more complex, creative research tasks. Expect ongoing demonstrations of AI handling increasingly sophisticated research activities, alongside efforts to understand the limits of automation in hypothesis formulation and scientific discovery. Policy and institutional responses may include re-evaluating the role of human researchers and adjusting R&D workflows to incorporate AI more deeply. The next 32 months are expected to reveal whether the residual research tasks can be automated or if human insight remains indispensable.
Key Questions
What specific engineering tasks can AI now automate?
AI can automate tasks such as reproducing research experiments, generating optimized GPU kernels, converting code between frameworks, and managing complex dependencies, effectively handling many routine and technical aspects of AI development.
Does this mean human researchers are no longer needed?
While AI automates many engineering tasks, research activities involving creative hypothesis generation, strategic planning, and conceptual breakthroughs still require human insight, though this may change in the coming years.
How reliable are these benchmarks and demonstrations?
The benchmarks are based on recent empirical data and industry demonstrations, with some reaching or nearing saturation levels. However, the full scope of AI’s capabilities in research remains to be tested across more complex and less structured tasks.
What are the implications for AI research institutions?
Institutions may need to adapt workflows to leverage AI automation effectively, potentially reducing the need for routine engineering work and focusing more on strategic, innovative research areas that AI cannot yet handle.
Source: ThorstenMeyerAI.com
