Stanford Law Finds Professors Favor AI Answers 75%

A new study led by Stanford Law School has found that law professors significantly prefer artificial intelligence-generated answers to student questions over responses written by their peers. In a blind evaluation, professors selected AI responses as the better answer approximately 75 percent of the time across nearly 3,000 comparisons.

The research, conducted by Professor Julian Nyarko and colleagues, involved 16 contract law professors from 14 U.S. law schools. Participants first created 40 questions representative of those students ask during office hours, covering legal doctrine, case law, hypotheticals, and policy issues. Each professor wrote answers to a subset of the questions. Researchers then used AI models, including Google’s Gemini 2.5 Pro and NotebookLM, to generate responses to the same questions.

In the evaluation phase, professors reviewed anonymized pairs of answers without knowing whether the author was a colleague or an AI system. They selected which response they would rather give a student. The AI models won roughly three out of every four head-to-head matchups. Notably, the performance of the AI was comparable to that of the single highest-rated human...