IBL News | New York
Leading researchers believe AI systems will eventually be powerful enough to improve themselves with little or no help from human developers. This idea that AI can learn and train itself, creating exponential progress, is called recursive self-improvement, or R.S.I., by computer scientists.
Some Silicon Valley start-ups valued at billions, and top labs like OpenAI and Anthropic, are hoping to accelerate the development of AI that discovers drugs, creates new materials, speeds up other forms of scientific discovery, and, one day, surpasses human intelligence in practically every way.
This way, AI is already accelerating the development of new AI systems, as they analyze vast amounts of digital data and learn an increasingly impressive array of skills.
“Now is the time to take these ideas, which we have been incubating in the lab for decades, and start to really scale them up,” said to The New York Times Jeff Clune, a veteran of OpenAI, Google and other top labs who helped found a start-up called Recursive Superintelligence late last year. “We have all the pieces of the puzzle.”
In a blog post this spring called “When AI Builds Itself,” Anthropic said its push toward R.S.I. could increase the risks of humans losing control over A.I. systems. Last week, the company’s CEO cited these efforts as a primary reason for slowing the development of A.I.
This belief is one reason that some people, including some employees of leading AI labs, are loudly predicting that artificial intelligence could destroy humanity.
“If models can self-improve quickly via architectural improvements, it is quite possible a single model can disable all rivals while it acquires more and more power,” Jason Abaluck, a Yale University economics professor, said on social media as the discussion turned toward doomsday scenarios.
If this happens, AI could rapidly become so powerful as to dominate the world, for good or ill.
At the moment, most companies design neural networks using an architecture called a transformer, which a team of Google researchers developed in 2017. But some startups are creating systems that can spawn entirely new architectures and methods.