Can AI Evaluate Engineering Creativity as Well as Humans?
Large language models can now rate design ideas in seconds. Our research asks when those ratings can be trusted — and when they can't.
britelab · Sep 1, 2026 · 5 min read
Judging creativity has always been slow, expensive, and subjective. Engineering educators and design teams rely on trained human raters to score ideas for novelty, usefulness, and quality — a process that can take weeks for a single study.
What we are testing
britelab compares AI-generated creativity ratings against expert human judges across thousands of engineering design concepts. We look not only at average agreement, but at where AI and humans diverge: highly novel ideas, sketches versus text, and concepts outside the training data.
What we are learning
AI can be a fast, consistent first pass, but agreement drops for the most original ideas — exactly the ones designers care about. The most reliable results come from human-AI rating teams, where AI handles volume and people handle judgment.
Why it matters
If AI is going to help assess creativity in classrooms, hiring, or R&D pipelines, we need evidence about its blind spots. Our goal is practical guidance for using AI as a partner in evaluation, not a replacement for it.
Related project
Crowd-AI Teams
See related work in our publications or meet the research team.
