📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems are rapidly automating core engineering tasks, reaching near-saturation in benchmarks. Research, however, remains only partially automated, with some aspects still requiring human insight. This shift could reshape AI development timelines and institutional strategies.
Recent research and benchmarking data confirm that AI systems can now automate the majority of engineering tasks involved in AI development, with some experts suggesting that the residual research component remains only partially automated.
Six key benchmarks measuring AI capabilities in core AI R&D tasks show rapid progress, with three reaching or approaching saturation. For example, the CORE-Bench, which tests research reproduction, has improved from 21.5% in September 2024 to 95.5% in December 2025, with one of its authors declaring it ‘solved.’ Similarly, the MLE-Bench, evaluating Kaggle competition performance, has advanced from 16.9% in October 2024 to 64.4% in February 2026, nearing competitive levels with mid-tier human practitioners.
These benchmarks indicate that AI can now reliably handle complex, friction-laden research reproduction tasks and achieve performance comparable to skilled human practitioners in competitive environments. Meanwhile, progress in kernel design—another critical component of AI R&D—continues through published research and practical applications, such as automated GPU kernel generation and optimization by companies like Meta and Huawei.
According to Thorsten Meyer, these developments suggest that the engineering aspect of AI research has been largely automated, with the remaining residual being the research itself, which may involve creative, hypothesis-driven work less amenable to automation.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
AI Tools for Finance and Accounting Professionals: Automate Tasks, Save Hours, Work Smarter
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
GPU-Accelerated Computing with Python 3 and CUDA: From low-level kernels to real-world applications in scientific computing and machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
Evals for AI Engineers: Systematically Measuring and Improving AI Applications
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
Smart WordPress Engineering With Claude Code: Create Responsive Business Platforms Through Automated AI-Driven Development Frameworks (Intelligent Programming and Systems Architecture)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Near-Complete Engineering Automation
The rapid automation of engineering tasks in AI development could significantly shorten research cycles, reduce costs, and shift institutional focus toward higher-level research questions. As engineering becomes an automated process, the bottleneck may shift to the creative and theoretical aspects of research, which remain less automatable. This transition could influence how AI labs allocate resources and prioritize innovation, potentially accelerating the pace of AI advancement.
Background on AI Progress in R&D Tasks
Over the past two years, AI capabilities have shown consistent improvement across multiple benchmarks relevant to AI research and engineering. The CORE-Bench, MLE-Bench, and kernel design research demonstrate a pattern of rapid progress, with all indicators pointing toward saturation or measurement limits. This trend aligns with broader observations that AI systems are increasingly capable of performing complex, multi-step research and engineering tasks with minimal human intervention.
Thorsten Meyer notes that these developments challenge traditional notions of research as a purely creative, human-driven activity, suggesting a structural shift where research may itself become an engineering process at scale.
“AI can today automate vast swatches, perhaps the entirety, of AI engineering. It is not yet clear how much of AI research it can automate, given that some aspects of research may be distinct from the engineering skills.”
— Thorsten Meyer
Unresolved Questions About Research Automation
It remains unclear how much of the research process—particularly hypothesis generation, creative problem-solving, and theoretical innovation—can be automated. While engineering tasks are approaching full automation, the residual research activities may still require human insight, and the timeline for their automation is uncertain.
Next Steps in AI R&D Capabilities Development
Over the next 32 months, experts anticipate continued progress in automating research tasks, potentially leading to fully automated research pipelines. Institutions may need to adapt strategies, focusing more on high-level research questions and less on engineering execution. Monitoring benchmark developments and practical applications will be essential to gauge the pace of this transition.
Key Questions
What does automation of engineering tasks mean for AI research?
It suggests that much of the technical, repetitive, and implementation-heavy work involved in AI development can now be handled by AI systems, reducing costs and accelerating project timelines.
Are there aspects of AI research that still require human input?
Yes, creative problem formulation, hypothesis generation, and theoretical innovation are still less automatable and may remain human-led for the foreseeable future.
How reliable are the current benchmarks in measuring AI progress?
The benchmarks have shown rapid improvement and are approaching saturation, but they may not fully capture all aspects of research and engineering complexity.
What are the potential risks of automating AI research and engineering?
Risks include over-reliance on automation, potential loss of human oversight, and challenges in ensuring AI-generated research aligns with ethical standards and safety protocols.
Source: ThorstenMeyerAI.com
