Funded Projects in This Area
Active sponsored research in this area.
Crowd-AI Teams
Leveraging Crowd-AI teams for scalable novelty ratings of heterogeneous design representations
Overview
This NSF-funded research project advances engineering design by developing a framework for rating the novelty and quality of design artifacts using a combination of human and machine expertise. The project addresses a significant challenge: how to accurately evaluate the vast amounts of complex, heterogeneous ideas generated in design studies — including CAD drawings, text, sketches, and prototypes — against established design metrics at scale. It investigates three distinct means of evaluation: (1) human expert designers and crowd-sourced raters, (2) pure ML-based methods, and (3) expert designer-assisted ML methods. The team also uses deep learning, eye-tracking, and physiological sensing to measure and understand how engineers think, create, and assess during design tasks — mapping cognitive patterns like attention, anticipation, and motion efficiency that were previously invisible.
Research Goals
- Understand the impact of design representation on the reliability of crowd-based design ratings
- Establish and validate multi-modal representation learning methods for design evaluation
- Create methods for deploying crowd-ML collaborations for scalable design evaluation
- Use eye-tracking to understand expert vs. novice cognitive strategies during design
- Create objective measures of design quality, creativity, and innovation
- Contribute to ML research on what, when, and how to learn from humans when evaluating heterogeneous datasets
Impact
Contributing to improved product design by effectively leveraging human and computational expertise in evaluating design artifacts. The team partners with the Women in Science and Engineering Research (WISER) program and the Multicultural Engineering Program (MEP) at Penn State to encourage participation of underrepresented groups.
Research Team
- Dr. Scarlett Miller (PI)
- Dr. Farnaz Tehranchi (Co-PI)
- Dr. Faez Ahmed (PI, MIT)
- Duk Hee Ka (PhD Student, MIT)