Tim Hardwick
Leading AI companies including OpenAI, Google, and Anthropic are facing “diminishing returns” on their expensive efforts to build new AI models, according to a new report from Bloomberg. The stumbling blocks appear to be growing as Apple continues to roll out its own AI features in stages through Apple Intelligence.
OpenAI’s latest model, known internally as Orion, reportedly failed to meet the company’s performance expectations, particularly when handling coding tasks. The model is said to have no significant improvement over existing systems, compared to GPT-4's advances over its predecessor.
Google is also reportedly facing similar hurdles with its upcoming Gemini software, while Anthropic has delayed the release of its anticipated Claude 3.5 Opus model. Industry experts who spoke to Bloomberg attributed the issues to the growing difficulty of finding “new, untapped sources of high-quality human-generated training data” and the huge costs associated with developing and operating new models alongside existing ones.
The report says Silicon Valley believes that more computing power, data, and larger models will inevitably lead to better performance, and that the Holy Grail of artificial general intelligence (AGI) may ultimately be based on false assumptions. As a result, companies are now exploring alternative approaches, including further post-training (incorporating human feedback to improve responses and refine tone) and developing AI tools called agents that can perform targeted tasks, such as booking flights or sending emails on behalf of a user.
“The AGI bubble is bursting a little bit,” said Margaret Mitchell, chief ethical scientist at AI startup Hugging Face. She told Bloomberg that “different approaches to training” may be needed to make AI models work really well on different tasks. Other experts who spoke to the publication echoed Mitchell’s sentiments.
It’s unclear how much these concerns will impact Apple’s approach, though Apple Intelligence is more focused by comparison, and the company uses privacy-focused large language models (LLMs) internally. Apple’s AI services mostly run on-device, while the company’s encrypted Private Cloud Compute servers are only pinged for tasks that require more advanced computing power.
Apple integrates AI capabilities into existing products and services, including writing tools, Siri enhancements, and image generation features, so it’s not exactly competing directly in the LLM space. However, Apple has partnered with OpenAI to allow Siri to optionally pass more open queries to ChatGPT. Apple has also reportedly held talks with other LLM companies about similar outsourcing partnerships.
It’s possible that the challenges faced by major AI companies pursuing breakthrough general-purpose AI models may ultimately vindicate Apple’s more conservative strategy of developing specific AI features that improve the user experience. In that sense, its privacy policy may not be as straitjacketed as it first seemed. Apple plans to expand Apple Intelligence capabilities next month with the release of iOS 18.2, and then through further updates through 2025.
Tags: Apple Intelligence Guide, ChatGPT[ 170 comments ]
