The many flaws with design polls

An example of a social design poll post recreated for this debate.

Take a moment to observe the above image: a typical social media design poll, recreated to save the designer from embarrassment, but the structure, styles, and content remain the same.

Now consider the question: which [design] is better? Before you decide on your answer, read through the following problems:

Problem 1: Design is subjective

If you ask someone to choose what they regard as the best design, they will use limited available information, internally or externally, to create a judgement. However, this fast, intuitive type of thinking can sometimes lead to incorrect and irrational assumptions.

For example, you might choose one design over another based on attractiveness rather than ease of use for the target user.

With design being subjective, better means different things for different people.

In reality, when you ask someone to compare two designs by framing the question: which one is better, you are merely asking them to form a personal judgement with little context. Design is subjective, and better means different things for different people.

Problem 2: Little context on the user

Design polls often fail to provide audiences with context on the target user and behaviours. Without any direction on business requirements and user needs, it leaves the entire challenge incredibly open to pure subjectivity.

Which chicken is better? Without any context on the target user and behaviours, design polls are meaningless.

In addition to who the user is and what their behaviours might be, context can also include environmental, accessibility, social, political, and time-based factors, all of which can shape your judgment. For example, a person with no physical or cognitive issues might value a design more than someone with impairments.

Problem 3: Attitudes and behaviours are entirely two separate things

Asking for a response to a collection of flat designs is only going to yield attitudinal insight. This type of feedback has very little to do with user behaviours — what they do.

To design the best UX, pay attention to what users do, not what they say — Jacob Nielsen.

When you ask someone their opinion on which design they think is better, they will likely base their views on attractiveness and surface features. However, what people say does not always correlate to their actions.

In contrast, observing people using a design can provide insights that can vastly improve solutions. Paying attention to how users behave will reveal their preferences and help establish patterns in their actions.

Read: First Rule of Usability? Don’t Listen to Users for more information on this subject.

Problem 4: One size does not fit all

In a world where people have different needs, you cannot create a homogeneous design or a one-size-fits-all approach. When asking an audience to choose between design A or B, you have to be specific about the target user needs you are trying to support.

To achieve an experience tailored to support multiple user needs, you need to test more than just two variants. So, instead of A vs B, it should be something like ABCDEFG…

Problem 5: There is no explanation for the choices made

A design poll can tell you the most popular choice, but it cannot provide you with an explanation for the reasons why. Although members can add comments on LinkedIn and other social media platforms, they do not always contain enough depth on what factors influenced their decision-making process.

Good research requires qualitative user insights to provide further clues and to add meaning to the quantitative methods. However, when the designer has little insight into their users, they have little learning opportunities to make vital improvements.

Problem 6: Most people will not pay attention to the task

Regardless of how much context you provide, it is highly likely that people will glance at the designs and then quickly cast their vote. This type of behaviour resonates with most social media sites, whereby users have the habit of scrolling and scanning for content and then making knee-jerk responses without sufficient thought.

Problem 7: Design mistakes affect the outcome

Design inconsistencies and errors can influence the outcome of a design poll. For example, people might deviate from variant A for it having a missing email field — a simple design mistake.

Variant A (left) has a missing input field label — a sloppy mistake that might affect the outcome

Problem 8: You are asking for feedback too late in the process

If the designer has created high-fidelity mock-ups without conducting or using any research on their target user, they have jumped some critical steps.

When they have arrived this far into a design process, too much effort has gone into perfecting a solution without truly understanding the problem.