The Anatomy of a Generative Prototyping Session
In generative prototyping, the user formulates a request, the system realizes it, and the user inspects the prototype before the next request. This paradigm is rapidly evolving, yet we know little about how these steps relate within a session. In a preregistered study, 100 non-experts used an instrumented generative prototyping environment to complete three tasks in familiar and unfamiliar domains. We dissected 1,643 generations into prompt formulation, extent of change, verification, and next move. We measured the abstraction and specificity of prompts and found both associated with the structural extent of the generated change. In a follow-up study, twelve designers and eight non-experts were interviewed about selected events of their sessions. Based on the qualitative findings, we discuss how reflective practice extends to verification and how expertise informs formulation and evaluation.
Materials
Mark
Explore the tool used in both studies: write prompts, point, sketch, revisit earlier versions, and see the results in a live preview.
The session, as participants saw it
Step through the consent form, survey, tutorial, and three ten-minute prototyping tasks in a demo session.
Replay interviews and the shared vignette
Follow one replay interview: the moments picked from a designer's session, the questions asked about each, the shared vignette, and the debrief.
Annotating generation events
Try the researchers’ annotation workspace for prompt abstraction, specificity, and the next move, with the definitions and examples used in the study.
Hypotheses, materials, data, and code
Read the Study 1 preregistration and explore the task briefs, prompts, data, and analysis code.