Academia

Percy Liang

Stanford University and Together AI

Why we track: Evaluation and open models

Career moves

No published career moves.

Papers

Paper2026-10-22

Discovered the laws for improving performance using artificial intelligence-generated synthetic data

Rewriting Scaling Laws for Synthetic Data with Compute and Data Constraints

Percy Liang +6 more

What: When using synthetic rewrites where language models rewrite natural language documents into different styles for pre-training, we studied the laws for predicting the optimal ratio and performance under compute and data constraints.
Key result: As compute increases, the optimal proportion of synthetic data increases, and models trained on synthetic data generated by larger or equally sized models showed superior performance compared to models using only original data.
Why it matters: This suggests that AI training methods utilizing synthetic data are predictable, enabling the overcoming of data limitations and efficient performance improvement.AI summary

Training & scaling

Full summary Read paper ↗ PDF ↗

Paper2026-10-04 · arXiv (Cornell University)

Measuring how easily AI comes up with future research ideas

Priced Guidance: Can Language Models Generate Future Research Ideas?

Percy Liang +2 more

What: The study proposes a method to indirectly estimate the likelihood that a language model generates novel research ideas without hints, using the amount of information (compression cost) required to reveal the correct answer.
Key result: Five models were evaluated on the core ideas of 87 recent deep learning papers, and the median compression cost of the best single model was 69.9 bits. This is significantly lower than the median of 5,712 bits for the general compression tool gzip.
Why it matters: This research demonstrates a new evaluation framework that can calculate the lower bound of the probability that a model generates ideas. However, the abstract does not address research productivity or market effects.AI summary

AI for science

Full summary Read paper ↗ PDF ↗

Paper2026-09-10

Analysis of Overfitting in Mixture-of-Experts Models When Repeating AI Training Data

Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data

Percy Liang +4 more

What: It was revealed that when AI training data is repeated, the recently widely used Mixture-of-Experts architecture degrades in performance faster than conventional models.
Key result: When data is repeated, Mixture-of-Experts models easily overfit, and while existing regularization techniques can partially mitigate this, they fall short of the performance achieved by training on unique data.
Why it matters: This demonstrates the necessity of establishing training strategies that consider the characteristics of each AI model architecture in situations where iterative learning is required due to data scarcity.AI summary

Training & scaling

Full summary Read paper ↗ PDF ↗

Paper2026-05-06 · arXiv.org

Development of a Platform for Evaluating Security Risks of AI Agents

DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents

Percy Liang +16 more

What: Developed a platform and automated attack tools to evaluate the security risks of AI agents in various real-world environments.
Key result: This revealed systematic vulnerability patterns across various AI agents and provided insights into security.
Why it matters: It can be used as useful foundational data for developing next-generation safe AI agents.AI summary

Agents & coding Safety & policy

Full summary Read paper ↗ PDF ↗

This page compiles published papers, official announcements and press reports for information only. Paper summaries are written by AI from the abstract and reviewed by the editor; they may differ from the original, so check the paper for anything important. Nothing here is a recommendation to buy or sell any security or investment advice. Career moves cite official announcements or major press reports; some career records come from Wikidata (CC0). Summaries on this page were translated by AI from the editor-reviewed Korean version.