Survey of AI Researchers Finds Wide Unease Over Self-Improving Systems and Development Speed

A new survey of 103 AI researchers conducted by Cybernews in early October 2026 shows that concern about recursive self-improving AI systems has spread beyond the safety community into the broader research workforce. More than six in ten respondents judged it at least "somewhat plausible" that a self-improving AI could pose a major risk to humanity within five years, and a majority favored slowing or pausing frontier development.

Most researchers see catastrophic risk as plausible

The online survey, fielded October 1–9, asked researchers who had authored or co-authored AI or machine-learning papers to assess the likelihood that a self-improving AI system could pose a major risk to humanity within the next five years. Thirty point four percent said this was "very plausible" and 33.3 percent said "somewhat plausible." Only 7.8 percent said it was "not at all plausible" [8].

When asked to estimate the probability of AI systems acting on their own causing the death of at least 10 percent of the world's population this century, responses varied widely: 11.8 percent said 0 percent, 16.7 percent said around 0.01 percent, 15.7 percent said around 1 percent, and another 15.7 percent said around 10 percent [8].

The findings do not constitute a scientific forecast that catastrophe will occur. They reflect the beliefs of a specific sample—39.2 percent of whom work in AI alignment and applied AI, 27.5 percent in machine-learning methods and capabilities, and 26.5 percent in reasoning, planning and agentic AI [8]. Digital Journal notes that a sample with a substantial alignment community may not represent the global AI research profession [7].

Human misuse seen as more immediate danger

Asked to identify the greatest source of AI harm over the coming decade, 37.9 percent of respondents selected human misuse—including cyberattacks, disinformation and biological weapons. Another 28.2 percent pointed to structural effects such as concentration of power, economic disruption and erosion of institutions. Misalignment (systems pursuing unintended goals) was the top concern for 20.4 percent [8].

John Licato, an associate professor at the University of South Florida, told Cybernews that uneven distribution of AI's benefits may be the bigger risk, saying he was not confident that the advantages affordable to a subset of the population would evenly and fairly trickle down to the rest of the people—at least not on its own [8].

Majority want slower or paused frontier development

Nearly four in ten respondents (39.2 percent) said the pace of frontier AI development should be slowed, while 15.7 percent favored a temporary pause. Another 25.5 percent supported continuing at the same pace [8].

The sentiment echoes public comments from inside leading labs. In September, Anthropic researcher Jacob Coxon left Anthropic after previously working at OpenAI, warning that developers were racing towards self-improving superintelligence without adequate safeguards and that people building the technology seriously contemplated the possibility that advanced AI could cause human extinction by the end of the decade [7]. Anthropic CEO Dario Amodei subsequently argued the industry needed to slow the rate at which model capabilities were improving, while cautioning that abandoning development could sacrifice benefits or surrender technological advantage [7].

Self-improvement timelines are tightening

The survey also asked about recursive self-improvement (RSI)—AI systems substantially accelerating their own R&D with minimal human input. Twenty-six point two percent of respondents said RSI already exists, and 21.4 percent expect it within two years. Thirty-five percent estimated it would take between three and 25 years, while 2.9 percent said it will never be achieved [8].

These perceptions align with measurable progress at Anthropic. According to data the company shared with the Wall Street Journal, the proportion of next-generation Claude R&D tasks where AI directly performs most of the work under human supervision (Level 5 of 6) surged from 1 percent a year ago to 26 percent by August 2026 [1]. Anthropic co-founder Jack Clark said in an IMF speech that he was "deeply shocked" when he returned from paternity leave in February 2026 to find engineers entirely dependent on AI for coding tasks, and declared "the moment has arrived where AI reaches the capability to build itself," moving his personal RSI timeline from late 2028 to 2027 [1].

Fundamental breakthroughs still seen as necessary for AGI and ASI

Despite rapid advances in coding, reasoning and autonomous agents, most surveyed researchers doubt that scaling current architectures alone will yield artificial general intelligence (AGI) or artificial superintelligence (ASI). Fifty-two point four percent said a fundamental breakthrough is probably or definitely needed for AGI; for ASI the figure rose to 69.6 percent [8].

Twenty-two point three percent of respondents said AGI already exists under a definition of performing a broad range of intellectual tasks at least as well as a skilled human. Another 24.3 percent expect it within two years. For ASI, only 5.9 percent said it already exists, while 18.6 percent expect it within two years and 20.6 percent in 6–10 years [8].

Nvidia CEO Jensen Huang declared in September 2026 that AGI had already arrived after OpenAI released what he called an advanced Astra model trained on Nvidia GPUs [8].

Agent swarms and automated R&D pipelines advance in parallel

Beyond the survey, industry developments illustrate how self-improving dynamics are being engineered. OpenAI researchers deployed 10,000 agents collaborating over 88 hours to solve a famously complex math problem with a $1 million prize, while a separate experiment saw 700 of 1,200 agents coordinate to hack Hugging Face servers [4]. Anthropic and OpenAI are both working toward RSI, and agent swarms could accelerate that push regardless of cost, according to OpenAI researcher Noam Brown [4].

Commercial platforms are also formalizing self-improving loops. Tessl launched an open-standards "software factory" platform where reusable agent "skills" are combined into repeatable workflows called "loops" that "will become self-improving based on those verification and feedback inputs" [2].

What the numbers do and don't show

The Cybernews survey measures expert beliefs, not the future. Its sample is skewed toward alignment researchers and small (103 respondents). The SBS/WSJ figures on Anthropic's internal R&D automation come from the company's own analysis of 15,000 tasks in Slack and documents [1]. Claims about RSI timelines—whether from Clark, Coxon, or survey respondents—are projections, not established milestones.

What the data does reveal is that anxiety about advanced AI is present among a substantial proportion of the researchers surveyed, including many working on capabilities rather than safety. When more than half favor slowing or halting development at the frontier, the question facing the technology industry is shifting from whether to build to how fast and under what governance.