A preference test shows people two or more designs side by side and asks which they prefer. A short follow-up question, "What made you choose this design?", tells you why. It's one of the quickest ways to settle a debate between design directions, and to give a client evidence rather than opinion.
When to use one
- Choosing a direction: two or three homepage concepts, layouts or visual styles.
- Details that matter: logos, hero images, headlines, pricing tables, call-to-action wording.
- Checking a redesign: the current page against the new one.
A preference test tells you what people like and what they say. It doesn't tell you whether a design works. If you need to know whether people can find things or complete a task, pair it with a first-click test or a tree test.
Setting up a fair comparison
- Change one thing, or be clear that you changed many. If designs differ in layout, colour and copy at once, you'll know which people preferred but not why. The reasons help, but a focused comparison is easier to act on.
- Match the level of finish. A polished design will usually beat a rough one, whatever its merits. Show designs at the same fidelity and size.
- Shuffle the order. People tend to favour whichever option they see first. ClickTree shows designs in a random order for each person by default, and checks your results for this kind of position bias.
- Hide your names. Participants see "Option 1, 2…", not "New brand direction" or "Current site", so labels don't sway them.
- Ask a specific question. "Which do you prefer?" works, but "Which homepage would make you more likely to shop here?" ties the choice to what you care about.
- Always ask why. The reasons are often more useful than the vote. ClickTree adds "What made you choose this design?" to every question, and groups the answers by design.
How many people?
It depends on how big a difference you want to detect. For two designs, roughly:
| If the real split is… | People needed to show it reliably |
|---|---|
| 75 / 25 | about 30 |
| 70 / 30 | about 50 |
| 65 / 35 | about 85 |
| 60 / 40 | about 200 |
These figures give an 80% chance of a clear result at the usual 5% significance level. A strong preference shows up with small numbers; a slight one needs many people, and may not matter much in practice anyway. Around 50 people is a sensible default for two designs. With three or more, plan for a few more.
Reading the results
ClickTree shows each design's share of choices, with a likely range for people like your participants. Then it gives a verdict:
- Clear favourite: the split is unlikely to be chance, and the leading design is ahead of the next one by more than chance would explain.
- Leaning: choices aren't spread evenly, but the top two are close. More responses, or a head-to-head test of just those two, would settle it.
- Too close to call: the split could easily be chance. Either people genuinely like the designs equally, or you need more responses.
Then read the reasons. Common words for each design show what people noticed. For example, "calm", "clear" and "premium" for one, "bold" and "modern" for another. Those words are often what you take into the next iteration, even when the vote is close.
Common pitfalls
- Treating a preference as proof of performance. People sometimes prefer the design that's harder to use. Test the task too.
- Testing with the wrong people. Colleagues and stakeholders aren't your customers. Recruit people like your real users.
- Stopping too early. With 10 people, a 7–3 split could easily be chance. Wait for enough responses before declaring a winner.
Ready to try it? Create a preference test in ClickTree, add your designs as images or screenshots of live pages, and share one link.