AI's Freelance Fail: Even Top Agents Struggle to Earn a Living
Artificial Intelligence
AI
Freelance Work
Scale AI
Center for AI Safety
Automation
Job Displacement
8
Reality Check
Media Hype
6/10
Real Impact
8/10
What is the Viqus Verdict?
We evaluate each news story based on its real impact versus its media hype to offer a clear and objective perspective.
AI Analysis:
While AI hype surrounding job displacement remains, this benchmark provides a critical dose of reality, significantly reducing the short-term impact score due to the demonstrable limitations of current models.
Article Summary
Researchers at Scale AI and the Center for AI Safety have developed the Remote Labor Index, a novel benchmark designed to assess the economic viability of frontier AI agents. Their experiment tested leading AI agents—including Manus, Grok, Claude, ChatGPT, and Gemini—across a range of simulated freelance tasks sourced from verified Upwork workers. The results were sobering: the best agents only managed to complete less than 3% of the work, generating a paltry $1,810 from a potential $143,991. The agents’ struggles stem from their inability to handle complex, multi-step tasks, lack of long-term memory, and failure to continually learn like human freelancers. This challenges previous optimistic predictions about widespread AI job displacement, particularly following OpenAI’s GDPval benchmark, which had suggested AI models were nearing human-level performance. The Remote Labor Index highlights a critical gap – AI’s current capabilities are far from ready to replace a significant portion of the freelance workforce, and raises questions about the speed of AI’s evolution.Key Points
- Even the most advanced AI agents perform poorly in economically valuable freelance tasks, achieving less than 3% completion.
- AI’s primary limitation is its inability to handle complex, multi-step tasks and lacks the ‘long-term memory’ and continual learning capabilities of human freelancers.
- The Remote Labor Index offers a crucial counterpoint to previous optimistic predictions about AI’s immediate impact on the workforce, particularly in light of the GDPval benchmark.