For years, Instagram has been on a mission to make itself the nicest place online. It’s a quixotic mission for a social media company, especially one whose core users are teenagers—an age group that has proven particularly adept at making each other miserable. Cyberbullying is hard to define and even harder to measure; even Facebook, Instagram’s parent company, can’t estimate how prevalent the behavior is on its platform, or whether it’s worsened as Instagram replaces school cafeterias and shopping malls as the main place where teenagers interact. Still, that hasn’t stopped Instagram from rolling out feature after feature to mitigate bad behavior, in its effort to clean up cyberbullying for good.
On Tuesday, Instagram is adding two new tools to its repertoire. First, the platform will automatically hide comments that look like they might constitute bullying even if they aren’t obviously breaking the rules. Second, it will send a new warning message for users whose comments are repeatedly flagged as toxic, in the hope of changing behavior at the onset. Both tools will roll out to Instagram users globally, starting with those who speak English, Portuguese, Spanish, French, Russian, Chinese, and Arabic.
Instagram already proactively hides and removes the nastiest comments. Now it’s trying to make more borderline cases less visible. The new feature uses machine learning to find comments that look like ones reported to Instagram for bullying in the past. (The tool catches emoji too—as in, “You look like ?.”) Of course, context matters. The same words might be playful ribbing in one conversation and straight-up meanness in another. Even humans can’t always agree on whether or not something crosses the line: A 2017 study from the Pew Research Center found Americans split on whether behaviors like name calling or purposefully embarrassing someone counted as “harassment” at all.
Instagram, in this case, has chosen to err on the side of caution. “We’re trying to catch as much bullying and harassment as we can,” says Carolyn Merrell, Instagram’s global head of policy programs, who acknowledged a trade-off between protecting its users and stifling their free speech. While Facebook’s rules against bullying and harassment contain a dizzying array of prohibited language, Merrell says that “some comments may not violate our community guidelines but may still be seen as harmful, harassing, or bullying.”
If flagged by the AI, Instagram will now place those comments behind a box of text that says “View hidden comments.” Anyone can tap on that box to reveal the offending content, and people can still report those comments to Instagram if they violate the community guidelines. Instagram will also give people the option to remove the content cover from comments that they receive on their page. So if the crude joke your friend makes on your photo gets buried by the comment cover, you can choose to move it back out into the open.
