May 5, 2021 9:00 AM

Social Apps That Go Suddenly Viral Put Kids at Risk

While parents focus on Facebook and Twitter, children often turn to apps that allow users to remain anonymous, aren't moderated, and that adults have never heard of.

Anonymous apps like YOLO, Whisper, and the now-defunct Ask.fm have been linked to cyberbullying, pedophilia, unsolicited sexual image sharing, and even child suicides. Concerns over these apps largely focus on their anonymity, because they let people talk with limited accountability. But they’re also risky because they usually become popular by surprise. Such anonymous apps often gain popularity beyond their founders’ wildest dreams, leaving them unprepared to scale their content moderation enough to protect their largely young userbase.

Kids have the power to make an app go from zero to unmanageable in a matter of days, but anonymous apps tend to enjoy only fleeting success because they become too dangerous, and app stores remove them or their founders shut them down. For all the recent debate about Instagram Kids, anonymous apps pose one of the biggest current threats to children’s safety.

Take Sarahah as an example. Founded in 2016, the app was designed as a way to give anonymous feedback to your coworkers. It invited anyone with a link to answer a user’s question anonymously. Much to its founder’s surprise, Sarahah was quickly hijacked by teens and at one point attracted a staggering 300 million users. Researchers don’t know much about the kinds of questions teens asked, and overwhelmingly negative press coverage might not reflect the realities of the app. But we do know that users weren’t always on their best behavior: Sarahah was plagued with more complaints of cyberbullying than it could safely handle and was subsequently removed from app stores in 2018.

Sarahah is a perfect example of popularity by surprise: The app didn’t collapse because it was unpopular but because it became too popular too quickly. Its founders couldn’t scale its content moderation in time to protect its unexpected user base of children. Not all social media startups assume they will make money early on, which means the moderation expertise and staffing levels that come with popularity by surprise are often woefully inadequate.

Secret, an anonymous app founded in 2014, suffered a similar fate. Allowing users to share a “secret” with friends, the app was extremely popular with kids, earning the top spot in app stores in eight countries. But former CEO David Byttow said his team could not “control” the extent of users’ cyberbullying and other harassment, leading him to shut the app down in 2015, less than a year after it launched.

Anonymous apps that become popular by surprise pose huge risks to children’s safety, and yet they don’t seem to get the same volume of attention as the big players. To my knowledge, no countries today have laws requiring social media startups to have content moderation workforces at all, or for them to take a particular shape. This means kids can use anonymous apps mostly unsupervised, not just by their parents but also by app workers.

There is some growing recognition that smaller businesses might warrant different obligations than the more established players, but whether those obligations are looser or tighter is still up for debate. For example, the UK’s new Online Harms Bill proposes a “tiered approach” to its regulatory framework, dividing companies into two categories depending on the size of their user base and their functionalities, including the ability to communicate anonymously. But as the UK’s 5Rights Foundation notes, the tier system proposal fails to account for popular by surprise services that start out with a very small audience but quickly grow. To protect young users, the organization argues that Ofcom, the UK’s communication regulator and competition authority, “will have to ensure that new services which present a high level of risk are subject to the requisite regulatory requirements before reaching the [larger tier] threshold.”

Regulating new anonymous apps is a tricky balancing act: Do they need looser regulations so they can grow? Or do they need stricter rules, because a lack of regulation might make their young users more vulnerable to harm? While kids use globally popular apps like Instagram, TikTok, and Snapchat, they’re also drawn to apps no one has ever heard of, and one-size-fits-all policies that only imagine established platforms are never going to accommodate the unique challenges popular by surprise apps present.

Anonymous apps rise and fall in popularity with immense speed, and this creates another huge problem for children’s safety: Adults often know nothing about them. For my current research project, I gave a talk to around 200 people involved in children’s protection. While the talk focused on harmful online content and used anonymous apps as a case study, I foolishly assumed the attendees would know what anonymous apps were. They didn’t, and so I abandoned the talk and gave them an introduction to anonymous apps instead.

This anecdote highlights a need to keep tabs on what kids are actually doing on social media. While I’m not absolving tech giants of responsibility, we also need to talk about kids’ preference to fly under the radar when they use social media, reappropriating platforms like Sarahah for entirely new purposes and using apps adults have never even heard of. Our singleminded focus on the platform giants is fundamentally affecting children’s safety.

Whether their parents like it or not, kids want to spend time on anonymous apps. It’s not our job to judge their pleasures, but it is our job to make sure they’re safe.

WIRED Opinion publishes articles by outside contributors representing a wide range of viewpoints. Read more opinions here, and see our submission guidelines here. Submit an op-ed at opinion@wired.com.

nproxy.org