The clock on user trust & safety is ticking

The well-being of internet users has been facing a growing increase of threats with the misuse of digital services over the past few years. This led to an increasing urgency in understanding, supporting, and evaluating effective ways to reduce harms associated with online content and behavior.

As a major portion of these users involves the most vulnerable of our generation today: our children, the call for trust & safety management to rise up has never come at a more important time.

Agenda

This article aims to reveal the latest dangers & harms terrorizing our favorite digital services and to shine a light on the present issues at hand. It has no intention to throw blame except raise awareness and inspire action to convert such risks into opportunities.

Disclaimer!

Warning — the following content and images maybe disturbing and can cause unease and discomfort in some viewers.

Viewer Discretion is advised.

Hot Off The Press

Here are the top 3 highlights of the Content & Conduct-Related Risks that has happened over the past few years on TikTok:

1. Dangerous Trendy Challenges

Coronavirus Challenge

Videos of individuals licking objects in public, especially toilet seats, started to circulate during the world pandemic, 2020. The users did this as a public demonstration to prove that they had immunity to the COVID-19 virus. This misinformation went against health code violations and spooked most of the public.

DIY Dentist Challenge

Users started filming videos of themselves filing their teeth using nail files, making false claims that this made their teeth look nicer. This movement immediately sparked outcry amongst dentists, who criticized the ‘trend’ with great concern, explaining the danger in the process: filing down the teeth would remove the tooth enamel, risking tooth sensitivity, nerve inflammation, irritation and pain.

Skull Breaker Challenge

The name in the title should have conveyed how dangerous this challenge was already. The prank would involve two individuals tricking a third person to jump up while the pair issues a kick on the legs. This prank led to serious injuries and ‘landed’ the victims in the hospitals.

Autism Challenge

What started out as an act to raise awareness on autism snowballed into mockery when users made videos of themselves faking seizures and impersonating people with disabilities while music played in the background. This disgusting challenge faced severe backlash due to its insensitivity, causing a reasonable amount of hurt to the innocent individuals who truly suffered from such conditions.

Lost Trust

These videos were a clear display of the rotten user behaviour plaguing TikTok, which eventually led to the loss of trust from the community. It was every parent’s worst nightmare.

2. Minor Sexualization

WAP Challenge

WAP stands for a “wet a__ p_s_y” (Did you know that? I did not either until I googled it). This challenge involved the recreation of sexy dance moves from Cardi B and Megan’s music video.

This dance challenge was so popular that it inspired users, even the youngest & most innocent members of the community: the minors, to give it a go. Unlike the other challenges, the common impression was that the dance was provocative, nevertheless harmless.

A video by Project Nightfalltold a different story…

Pedophilia

These videos were subsequently found on several adult websites, revealing that it had become a source of pleasure to individuals who had a particular sexual interest in prepubescent children.

3. Misinformation

Anti-Vaccination Covid-19 Conspiracies

Many people went on social media to share about their beliefs on COVID-19 and advocated against vaccination. These non-medically trained users resorted to distorting facts, contorting elaborate lies as a means of spreading their pseudoscientific propaganda.

UX Psychology

Take a guess. What content do the majority of users enjoy the most?

Bad news/drama/disaster/misery

The fact is bad news travels fast. According to the College of Marin, bad news may be shared with nearly twice as many people as good news.

Why Do People Like This?

Schadenfreude. It means finding joy in someone else’s misfortune.

TikTok’s intelligent technology is both a blessing and a curse for all users. TikTokstarts by showing every video posted to a small group of users on their For You Page (FYP). If the video performs well with this small group, TikTokwill continue to push it out to more people that they think will like it.

Unfortunately, with aspirations of going viral, many users usually try to take advantage and ride on TikTok’s algorithm, creating videos with little regard for the safety of others and their own.

Slapstick Humor

The Jackass franchise is one of the kings of slapstick humour, being the most popular American reality comedy television & movies series of all time. It featured a team of knuckleheads performing stunts and pranks; smashing the box offices, with a gross of $487 million USD worldwide then. Apparently people do have an inert appetite for violence and stupidity. Also, memorable movies of this genre included classics like Dumb & Dumber and Home Alone.

Who could forget our beloved Tom & Jerry shows? Do these childhood shows mirror the harmful behaviour we observe on our social networking platforms today?

User Cognitive Biases

Watching people perform dangerous stunts isn’t necessarily harmful in the short term. However when videos go viral for the wrong reasons, it indirectly promotes it, as seen from the emergence of countless copycat videos. This phenomenon happens because of the cognitive bias that distorts our critical thinking.

The Bandwagon Effect

The bandwagon effect happens when the majority of the group is doing a certain thing, not doing that thing becomes increasingly difficult. Users believe and do things, solely for the reason that many others also do. They end up ‘hopping onto the bandwagon’ without proper decision making, only to suffer much later.

Groupthink

Similar to the bandwagon effect, group-thinking is a psychological phenomenon that occurs to individuals who put aside their personal beliefs and conform to a group’s mindset, resulting in dysfunctional decision-making.

Monkey See, Monkey Do

Have you heard of mirror neurons? In a study, these special brain cells appeared to be activated both when a monkey did something itself and when it simply watched another monkey do the same thing.

When exposed to videos with misconduct, users tend to feel a strong urge to imitate.

Fear of missing out: psychologically, users experience a feeling of anxiety or fear arising from the realization that they might be missing out on a rewarding experience. Watching short-clip videos that only capture the highlights of an act, gives viewers an impression that they might be missing out on the fun and thrill. Little do they know of the dangerous and risks they are exposed to.

Breaking Free from the Herd

There are ways to avoid this bias, for example, one can slow down their decision making process before adopting a trendy idea.

“Whenever you find yourself on the side of the majority, it is time to pause and reflect.” — Mark Twain

However this is indeed very difficult for teenagers and kids, who feel the need to fit in and dream of always being on the winning side. They are young and naive and lack the maturity to make sound judgement and smart choices. This makes them the most vulnerable group of us all.

TikTok’s Current Safety Features

With an overwhelming pressure to keep danger at bay, the first line of defense against harmful behaviour and content is to remove it. 89,132,938 videos were removed globally in the second half of 2020 for violating TikTok’s Community Guidelines.

Furthermore, a TikTok safety center has been set up in dedication to employ safe measures, for example features, to promote positivity and safety for their community.

In this case study, I will be evaluating TikTok’s current safety features with results from a simple user research and usability test I conducted. I recruited 5 participants between the age of 13–20, to learn from their behaviours and user experience.

1) Disclaimers

As an attempt to prevent copycat behaviour of dangerous stunts and combat misinformation, TikTokhas implemented a safety disclaimer as a banner attached to the bottom of videos to inform users to make responsible decisions.

Disclaimer 1 reads: “The actions in this video are performed by professionals or supervised by professionals. Do not attempt.”

Disclaimer 2 reads: “ The actions in this video could result in serious injury or adverse health effects.”

Evaluation

Technically, a disclaimer is important because it helps protect the business against legal claims. Disclaimers tend to notify users that they will not be held responsible for the damages that arise from the use of the products, or services.

So, was this disclaimer designed to keep TikTokor it’s users safe?

User Issues 1

• All of the research participants hardly noticed the banner, with 3 of them missing it all together
• Users understood the message on the disclaimer perfectly when questioned after but expressed that it exhibited no physical or emotional influence over them

“Was there a disclaimer? I didn't even notice. My attention was solely focused on the center of the video where the action was. I personally think that this method is ineffective because even after seeing the message, I did not feel anything at all.”

Problematic Mindset

• 1 user actually shared his honest perspective and experience on deriving enjoyment and entertainment from the “exhilarating” challenges on TikTok

“I enjoy watching these videos. It’s a great way for me to pass the time. Sometimes, I try and do them myself. Oh, I do not believe I will get injured. Those who do get hurt are losers who are too stupid and not careful. But I am very careful.”

UX Insights 1

• The disclaimer is buried under 4 different texts, making it very difficult for users to internalize the safety information present

By placing the disclaimer at the bottom, where it is the coldest, important messages like these would go unnoticed. My guess is that TikTokwent with this design with intentions of ‘protecting’ the aesthetics of the interface.

• Still, the safety warning may work for mature and older audiences
• For the younger and less mature viewers, this measure may be limited in its effectiveness, maybe even becoming counterproductive, tempting users to react with reverse psychology

The disclaimer feature needs to go beyond being just a vessel for information; it must disrupt the existing cognitive biases to empower users to make smart decisions by themselves.

2) Report

There is a feature on TikTokfor users who wish report harmful conduct and content which violates the community guidelines. It can be used to report 14 different scenarios which include:

• Misleading information
• Dangerous organization and individuals

Reports raise the organization’s awareness about existing issues so that corrective and preventative actions can be taken promptly.

Evaluation

However, does reporting actually give users a sense of trust and safety?

User Issues 2

• 4 out of 5 users shared that they have never reported a case before
• 3 out 5 users did not even know of the whereabouts of the feature
• 1 user actually conveyed strong distrust of it

“What is the point of reporting when I know that nothing will be done at the end of the day? How would I even know that my reports are taken seriously when I do not know how they work? Reports are useless and a waste of my time. Even if an account/video gets removed, the user can just recreate another account/make another video.”

UX Insights 2

• Users generally do not have the natural behaviour to report due to a lack of both motivation and ability
• The lack of motivation stems from the low perceived value of their actions. This is due to a failure in the unempathetic approach, leading users to believe that TikTokdoes not have the users interests at heart
• The lack of ability also stems from the mental effort and physical effort required to perform the report. It can get very tedious over time to repeat the process over and over

Despite how useful the reporting feature is, it lacks an empathetic approach, creating a barrier of doubt instead of trust. Users either end up ignoring it or develop a reluctance to use it.

Using UX Design to Battle the Dangers of Social Media

“Safety and trust go hand in hand. One cannot exist without the other when it comes to user experience design”

-Billy Aw

Approach

The Trust Triangle was created by Frances X. Frei and Anne Morris. They have made a case to argue that the three core drivers of trust are: authenticity, logic, and empathy. Only by fulfilling all corners of the triangle, trust can be achieved.

Based on their theory, I have specially made design changes to the existing safety features, striking a balance between user trust & safety.

1) Disclaimers

Design 1

Similar to the concept of the existing disclaimer, modifications have been made to its position and color.

Eye tracking research shows that users generally have their eye glaze point at the center of the screen. Hence,

• A strategic placement of a disclaimer in the center of screen capitalizes on the users’ main point of attention, communicating any message in the most effective manner
• Rather than plain black, a dark red colored banner sets a serious tone, also paving the way for users to associate any content they view with danger or warning sign (like a red light at a traffic stop); to be cautious and careful

Design 2

• Similarly, a more conservative design of using TikTok’s brand color would still capture the users’ attention in a warm manner
• Deployment of intelligent copywriting will send a logical message with empathy

Through bold changes to its user interface design, a clear commitment to its users is exhibited by TikTok. Establishing a safe experience is more important than preserving a brand’s aesthetics. This grand gesture is just what Tiktok needs to do to show their users and the community that they care.

2) Kids Safe

Stealing a page out of Youtube safety features, before a video is posted, users should be given the option to rate their content according to a safety rating system

• This measure will serve as a mental check for all users before posting any content, disrupting any form of inappropriate behavior fueled by irrationality
• This also places gives an opportunity for content-creators to lay the foundation for trust, setting up community full of socially responsible individuals

For trust to be earned, it first must be given.

3) Fact Check

With a new call-to-action button placed onto the home screen, users are able to have direct access to fact check the content from videos they watch and conduct an evaluation of the content presented, without even leaving the application.

• Blind imitation is probably the most dangerous user behavior a user can adopt
• This convenient method of validation empowers users to absorb information actively rather than passively. This expands ability to question, encouraging critical thinking and aiding in sound decision-making

Users must learn how to protect themselves from misinformation and trust themselves. When self-trust is established, confidence in the platform will also be restored.

4) Reflective Sharing

People love sharing videos. The issue doesn’t lie in acts of sharing but the breakdown in communication when videos are shared without much context. This means a video can go viral, based on the number of views/times it is being shared, regardless of the quality of content present. False information can be spread; harmful behavior can be promoted.

To combat this problem, whenever a user wants to share a video, a question “How does this make you feel?” will first appear.

• The additional step will add meaning to the sharing of the video, attaching the emotional response of the sender to the receiver
• Using emotional design, this feature grants users reflection time on their user experience that they will be sharing with others, leaving little room for misinterpretation and misinformation to be spread

Engaging users to practice empathy influences emotions, in which rationality and trust will follow.

Conclusion

Can the internet be a safer place? Yes it can. From my standpoint, TikTokneeds to start employing empathetic approaches to address the emotional deficits in their safety features and designs. Only by achieving the tri-factor of trust will only then trust be earned. Without it, users will never truly feel safe, even if they are protected from the external risks because the danger lies within.

So, let’s start simple. Let’s begin with trust.