Skip to content
ViqusViqus
Navigate
Company
Blog
About Us
Contact
System Status
Enter Viqus Hub

OpenAI Proposes Perplexity as a Shared Metric for Tracking AI Progress and Safety Limits

Perplexity Cross-Entropy Loss Pretraining AI Safety Standards Recursive Self-Improvement OpenAI
November 22, 2026
Source: OpenAI News

This summary and analysis were generated by AI from the original article at OpenAI News and may contain errors (how Viqus works). Read the source for full details.

Viqus Verdict Logo Viqus Verdict Logo 5
A Proposal, Not a Standard
Media Hype 4/10
Real Impact 5/10

Article Summary

OpenAI researchers argue that perplexity, or equivalently cross-entropy loss on a held-out dataset, should become a shared metric for comparing frontier model developers and informing common safety standards. The post reports OpenAI's internal log-perplexity values starting with GPT-3.5 (about 0.99), GPT-4 (about 0.91), and recent models below 0.8, which the authors link to emergent capabilities in cybersecurity and mathematics. The authors claim that AI contributions to pretraining research could soon produce a sudden acceleration, and they propose that frontier labs agree on shared evaluation norms, transparency standards, and perplexity-based limits on development until monitoring milestones are met. The post acknowledges limits of the approach: datasets differ across labs, perplexity ignores gains from chain-of-thought reasoning, and the entropy floor is unknown. The proposal is a policy position rather than a product release, and it is not yet backed by an industry agreement.

Key Points

  • OpenAI publishes internal pretraining perplexity numbers, from GPT-3.5 at about 0.99 log-perplexity to recent models below 0.8, and proposes this as a cross-lab metric.
  • The authors propose shared evaluation norms, transparency standards, and perplexity-based development limits that labs commit not to cross until monitoring milestones are met.
  • The post notes significant limits: labs use different datasets, perplexity does not capture reasoning gains, and no industry-wide agreement exists yet.

Why It Matters

This is a notable policy proposal from a frontier lab, since it suggests a concrete quantitative threshold for pacing development. Its impact depends on whether competitors adopt the metric, which is uncertain given the dataset and comparability problems the authors themselves list. The claim that AI could accelerate its own pretraining research as early as 2027 is speculative and self-reported, so the post should be read as a position paper rather than an established result.


You might also be interested in