Social Apps That Go Suddenly Viral Put Kids at Risk

Anonymous apps like YOLO, Whisper, and the now-defunct Ask.fm have been linked to cyberbullying, pedophilia, unsolicited sexual image sharing, and even child suicides. Concerns over these apps largely focus on their anonymity, because they let people talk with limited accountability. But they’re also risky because they usually become popular by surprise. Such anonymous apps often gain popularity beyond their founders’ wildest dreams, leaving them unprepared to scale their content moderation enough to protect their largely young userbase.

Kids have the power to make an app go from zero to unmanageable in a matter of days, but anonymous apps tend to enjoy only fleeting success because they become too dangerous, and app stores remove them or their founders shut them down. For all the recent debate about Instagram Kids, anonymous apps pose one of the biggest current threats to children’s safety.

Take Sarahah as an example. Founded in 2016, the app was designed as a way to give anonymous feedback to your coworkers. It invited anyone with a link to answer a user’s question anonymously. Much to its founder’s surprise, Sarahah was quickly hijacked by teens and at one point attracted a staggering 300 million users. Researchers don’t know much about the kinds of questions teens asked, and overwhelmingly negative press coverage might not reflect the realities of the app. But we do know that users weren’t always on their best behavior: Sarahah was plagued with more complaints of cyberbullying than it could safely handle and was subsequently removed from app stores in 2018.

Sarahah is a perfect example of popularity by surprise: The app didn’t collapse because it was unpopular but because it became too popular too quickly. Its founders couldn’t scale its content moderation in time to protect its unexpected user base of children. Not all social media startups assume they will make money early on, which means the moderation expertise and staffing levels that come with popularity by surprise are often woefully inadequate.

Secret, an anonymous app founded in 2014, suffered a similar fate. Allowing users to share a “secret” with friends, the app was extremely popular with kids, earning the top spot in app stores in eight countries. But former CEO David Byttow said his team could not “control” the extent of users’ cyberbullying and other harassment, leading him to shut the app down in 2015, less than a year after it launched.

Anonymous apps that become popular by surprise pose huge risks to children’s safety, and yet they don’t seem to get the same volume of attention as the big players. To my knowledge, no countries today have laws requiring social media startups to have content moderation workforces at all, or for them to take a particular shape. This means kids can use anonymous apps mostly unsupervised, not just by their parents but also by app workers.

There is some growing recognition that smaller businesses might warrant different obligations than the more established players, but whether those obligations are looser or tighter is still up for debate. For example, the UK’s new Online Harms Bill proposes a “tiered approach” to its regulatory framework, dividing companies into two categories depending on the size of their user base and their functionalities, including the ability to communicate anonymously. But as the UK’s 5Rights Foundation notes, the tier system proposal fails to account for popular by surprise services that start out with a very small audience but quickly grow. To protect young users, the organization argues that Ofcom, the UK’s communication regulator and competition authority, “will have to ensure that new services which present a high level of risk are subject to the requisite regulatory requirements before reaching the [larger tier] threshold.”

Regulating new anonymous apps is a tricky balancing act: Do they need looser regulations so they can grow? Or do they need stricter rules, because a lack of regulation might make their young users more vulnerable to harm? While kids use globally popular apps like Instagram, TikTok, and Snapchat, they’re also drawn to apps no one has ever heard of, and one-size-fits-all policies that only imagine established platforms are never going to accommodate the unique challenges popular by surprise apps present.