Those who fear that AI poses an existential risk to humanity fear that new systems will be developed before we understand the existing systems.
In any case, the release of GPT-5 is expected to be the AI event of 2024.
Computer software development typically involves tweaking previous versions to make small improvements.
The development of new AI systems – so-called large language models – is often a new beginning. An unprecedentedly large amount of data is being thrown at an unprecedentedly powerful system of next-generation microchips, resulting in a model that is many times more powerful than what came before.
GPT-1, the master model created in 2018, was trained using 117 million data points, so-called parameters. GPT-3 required more than a thousand-fold at 175 billion, and GPT-4 was a further tenfold increase at 1.7 trillion.
The need for computing has also increased significantly. GPT-4 reportedly required 16,000 high-end Nvidia A100 chips, up from 1,024 for the previous generation. Little is known about the next wave of models, but they will certainly be trained on Nvidia's new H100 chips, a much more powerful successor that is the first designed specifically for training AI models.
“The history of computer science and AI shows that larger scale leads to significant improvements,” says Oren Etzioni, former executive director of the Allen Institute for AI.
“The rise from GPT-3 to GPT-4 was so dramatic that it would be an idiot not to try again.”
Google, which unveiled its new Gemini model in December, is preparing to release the more powerful Gemini Ultra in the new year. Anthropic, the Amazon-backed AI lab, could also launch a new system.
However, scientists do not agree on what exactly “more powerful” means. Today's large language models are approaching their upper limits on certain tasks. Google's Gemini already outperforms humans on a widely used language comprehension test and computer programming exams.
Comments are closed.