Sycophancy in synthetic research
Also called AI sycophancy, LLM sycophancy
Sycophancy is the tendency of language models to tell the person asking what they seem to want to hear. In synthetic research it shows up as simulated users who like every concept they are shown.
Ask a model to play a customer and react to your idea, and it will usually find something to like. The answer is shaped by the question, and nothing in the exchange can contradict it.
The fix is not a better prompt. It is to stop asking for opinions and ask for behaviour: give the persona a task in the real product and record whether it completed it. A flow either got finished or it did not, and a screenshot either shows the error or it does not.
Where Stunt Double fits
This is why actors report what they did rather than what they think. Every finding carries the step and the screenshot, so you can check it yourself instead of trusting the summary.
Related terms
- Synthetic researchSynthetic research is user research run with AI participants instead of recruited people: interviews, usability tasks and studies where the participants are personas played by AI agents.
- Synthetic usersSynthetic users are AI agents given a persona (a background, goals and constraints) that stand in for real users in research or testing.
- User personaA user persona is a profile of a type of person a product serves: who they are, what they are trying to do, what they know already and what gets in their way.