Repeat after me: Preference testing is not A/B testing — David Travis — Userfocus ↗

Reasons why you shouldn’t simply ask users to choose which design they prefer.

It turns out people aren’t good at answering this kind of question. People don’t know why, or they don’t care enough to answer, or they may not want to tell you. When asked for an opinion, most people will form one on the spot. Such opinions aren’t carefully considered or deeply held. It’s not that UX researchers don’t care what people like: it’s just risky making important design decisions based on fickle opinions.

User experience isn’t about discovering what people think they want. It’s about finding out what they need.