Designing for your worst user

Design can't save the world, but design_ does _have a huge impact on how people behave.

In my experience working in research and design, I’m accustomed to doing user interviews, analysis, personas, and experience mapping. These are all tools that we use to understand who the person is at the other end of the computer, and how we can help them.

But what if we instead thought about how we could deter people? What if we started designing not only for the personas we want to help, but also the bad actors? The racists and conspiracy theorists who are intentionally acting in bad faith to hurt others and promote violence.

We know there are bad actors on the internet

The world can be a harsh and cruel place. And much like the world, the internet can quickly turn dark, violent, and harmful. Over the past few years, we’ve seen people on social media platforms like Twitter and Facebook say and do terrible things to others. Sometimes they get reported, sometimes their account gets taken down, but a lot of times it doesn’t.

Instead, we leave it up to social media moderators to view and pull the worst content from these platforms. They spend each of their days combing through hateful and horrific images and videos. Hours spent reading horrible messages aimed from one human to another.

This is wrong. It’s wrong to make people sit and watch disgusting and painful content just so we can watch someone post their puppy videos. It's wrong for us to expect so little and assume that this will be taken care of for us.

It's wrong to allow the people doing harm to continue hurting others.

It's a hard problem to solve, but throwing more people power at the problem is only a fleeting solution. In fact, it's a lot like a garden. Right now, we have a garden bed full of flowers, with weeds coming in and choking out the blossoms. We are hand-picking every weed, hoping that it doesn't come back later.

Bad actors will use your design to do bad things

In every case, users who intend to do harm will exploit your design for bad purposes. They will push boundaries. Cause scenes. Distract and gaslight until they are either banned or move on to the next attack.

And now, in a space where the angriest and most hateful people have a voice, others don't feel safe. In some cases, people are actually in physical danger.

From a fundamental human perspective, this is wrong.

From a business perspective this is fiscally irresponsible. If people don’t feel safe enough to engage with your product, they will eventually leave and use something else.

We can stop bad actors

There's a better, albeit harder, way.

**I think that we should start designing against the asshole. **

We should design for that person who trolls. The person posting violent content. That person who throws out conspiracy after conspiracy in an attempt to block real conversation.

What if we designed to deter that asshole?

Design against assholes

We should already be studying how people use our platforms through research and data metrics.

And we should also start talking about how bad actors are using the site, their common behaviors, and their goals. We should design for the user who has malicious intent by thinking about how someone could use the product to hurt or take advantage of other people.

We should make it as difficult as possible for them to feel empowered to act in this way. And if they do make it through the design, we must be committed as a society to making sure their harmful rhetoric doesn't get promoted.

And for those of you saying, "Yeah, but what about freedom of speech," I hear you. Freedom of speech means that you will not face legal punishment or imprisonment for discussing your ideas. It does not mean that we as a community can't decide what we will and won't allow in society.

We can think about how to deter these people through personas. When you’re designing for a developer going through your workflow, you create a customer journey and persona that maps to them.

You’re logging what the person is doing, thinking, and feeling, and then making design choices based on that. Here’s an example:

A developer wants to share this information with their colleague through an email. They will go to this link, select share and input their coworkers email. Because of this behavior, we need to present these controls/screens.

You can go through the same thought exercise, but for someone who has bad intentions:

An abusive ex will want to send abusive, anonymous hate to their former partner. They will do this by posting anonymously, or creating new accounts in rapid succession. Because of this behavior, we need to remove their ability to hide behind anonymity or multiple accounts.

In both cases, we are taking into consideration three things:

The persona's ultimate goal
How the person currently achieves that goal
How we can adjust design to impact that goal

We have to create the spaces we want

This system will never be perfect, or work every time. It will have flaws, but we are already working in a horribly flawed system.

Again, it’s a lot like a garden. When you have weeds, it takes a lot of time and effort to individually pull each weed, and often they still come back. But if you pull the weeds and then start planting ground-cover, you’re creating a strong root system that deters weeds from ever gaining another foothold.