By Dan Busso and Laura Brown

Many of the attitudes and behaviors we seek to understand as UX researchers are sensitive. The views and experiences we ask participants to share often touch upon weighty issues like race, class, sexuality, and health. In other instances, people may be invited to describe behaviors that are stigmatized, such as those that violate a platform’s policies. And some conversations may involve a power dynamic — discussions between a child and their parent, for example, or between an employee and their boss. In potentially charged situations like these, participants may respond in ways they think will be viewed favorably by others, either by tailoring their answers to social norms, or by avoiding answering a question entirely.

Remote research can amplify this effect, since it’s harder for interviewers to build rapport with study participants or to control who has access to a conversation. We felt the impact of these constraints during a recent study using remote video interviews. A newly married woman in Mumbai, India, spoke at length about how she loved sharing everything with her mother-in-law and was glad to seek her advice on every topic. When we finally asked, “Can she hear you now?” our participant responded, “Oh yes. She’s sitting on the other side of the room.” When participants are observed by others whom they fear will judge them or take offense at their responses, it can be difficult to accurately and completely describe their own behavior.

Addressing social desirability bias

Many UX researchers are aware of strategies to make socially desirable reporting less likely. Diary studies, for example, can uncover what participants are experiencing in the moment, rather than what they’re comfortable revealing directly to an interviewer in a one-on-one conversation. Minimizing the presence of the researcher and others during data collection may also help participants feel more comfortable addressing intimate topics.

Another strategy is the use of third-party and hypothetical dilemmas, which can enable participants to share feelings, beliefs, and knowledge they might not feel comfortable sharing directly. This strategy disrupts the power dynamic that pointed one-on-one questions can create, making the sharing of knowledge feel more natural and acceptable.

Exploring parent/teen dynamics through ‘digital dilemmas’

A recent study at Meta was designed to help us understand young people’s online privacy needs so we could build tools that might allow parents and other guardians to successfully support them. We knew that social desirability bias might be a problem; past research had shown us that parents often say they use parental controls and monitoring more than they actually do. Similarly, we suspected that teens might be reluctant to share rich discussions of their digital lives in the presence of parents. In prior research, teens had answered very differently when we talked to them with siblings in their bedrooms than they did in focus groups with parents and guardians present.

To tackle this issue in joint interviews with parents and teens, we engaged them in what other researchers have called ‘digital dilemmas.’ After asking parents (and teens, with their parents’ consent) directly about their feelings, attitudes, and behaviors, we gave them imaginary dilemmas and invited them to collaborate in coming up with responses.

In one scenario, we asked a teen to “imagine you had a 13-year-old cousin who had asked for some help with their messaging app, and you found that they were chatting with strangers, some of whom were older than they are.” We followed up with questions such as, “What might you do in this situation? Is there anyone you should talk to about it?” We then invited the parent to respond to the teen’s response, and to explain how they might approach the same situation.

This form of questioning was effective in uncovering attitudes and behaviors that teens and parents hadn’t described in earlier research. In response to the above scenario, for example, teens were particularly likely to seek additional details, stressing that it was important to know what kind of strangers the imaginary cousin had chatted with, as well as what they’d chatted about. They saw a significant difference between speaking to older teens and adults who were part of a shared online community versus those who lacked a common interest. Parents, in contrast, tended to emphasize the difference between friends and strangers, but were sometimes intrigued or persuaded by their teen’s discussion of the situation.

By depersonalizing the discussion, hypothetical dilemmas allowed teens and their parents to report on behaviors or attitudes they might not have been comfortable sharing if they’d been asked directly about themselves. On several occasions, participants described experiences that they’d left out of earlier interviews and diaries, and that they hadn’t previously shared with one another. The digital dilemmas also helped us confirm that even parents who thought that they were carefully supervising their teen’s digital activities often knew far less than they thought they did.

More tips for asking about sensitive topics

Another simple technique is to avoid using “you” pronouns in questions, as these tend to center participants’ identity and experiences within the discussion in a way that might lead to biased responses. In practice, this might look like asking “Why might someone feel that way?” as opposed to “Why might you feel that way?”

Inviting participants to engage in forms of interaction other than interviewer-directed question-and-answer can also be effective. You might ask them, for example, to show you how to do something, to reenact a behavior, to draw a picture, to make a mixtape, or to write a break-up letter to a product or institution. While these engagements can be time-consuming to analyze, and may require follow-up interviews for clarification, they provide insight into participants’ reasoning (with the added benefit of producing artifacts and images that can help illustrate research findings for audiences). When the tone and topic of the research allow, using activities that are fun, empowering, or collaborative to solicit responses can also help to build rapport with study participants — who may then feel more comfortable sharing their perspectives in direct interviews.

Further reading

Here are some great examples of work that uses hypothetical dilemmas, indirect questions, or activities to enrich qualitative research on sensitive topics:

• Academic work on digital dilemmas
• Using vignettes in qualitative research
• Briggs, C. L. (1986). Learning how to ask: A sociolinguistic appraisal of the role of the interview in social science research. Cambridge University Press.

We’re always learning new ways to address bias in our research. What methods have you found to be effective in overcoming social desirability bias in your own work?

Authors:

Contributor:

Illustrator: Drew Bardana