a scientific ranking of subreddits liable to trigger intersectional allies and others...,

   
   

idibon |  In any community there’s bound to be friction, but some… take it further than others. Reddit is a platform for thousands of online communities (known as “subreddits”), where community members can submit content, and upvote, downvote, or comment on content that others have submitted. Topics of discussion on Reddit run the gamut of human interest, but one of Reddit’s favorite topics to talk about is, unsurprisingly, Reddit itself.

A recent post on AskReddit posing the question – “What popular subreddit has a really toxic community?” – surged to the top of the front page with 4,000 upvotes and over 10,000 comments as Redditors voiced their opinions on which Reddit communities they found to be the most abhorrent (the “/r/” prefix denotes a subreddit):

• /r/parenting: “there are a TON of sanctimommies in there”
• /r/MakeUpAddiction: “[has] cult-like tendencies” encouraging a “race to be the palest and most translucent snowflake”
• /r/leagueoflegends: “[we] made professional players quit the game, that’s how lovely we are”

As I sifted through the thread, my data geek sensibilities tingled as I wondered “Why must we rely upon opinion for such a question? Shouldn’t there be an objective way to measure toxicity?”

With this in mind, I set out to scientifically measure toxicity and supportiveness in Reddit comments and communities. I then compared Reddit’s own evaluation of its subreddits to see where they were right, where they were wrong, and what they may have missed. While this post is specific to Reddit, our methodology here could be applied to offer an objective score of community health for any data set featuring user comments.

Defining Toxicity and Supportiveness

So what is Toxicity? Before we could do any analysis around which subreddits were the most Toxic, we needed to define what we would be measuring. At a high level, Toxic comments are ones that would make someone who disagrees with the viewpoint of the commenter feel uncomfortable and less likely to want to participate in that Reddit community. To be more specific, we defined a comment as Toxic if it met either of the following criteria:

1. Ad hominem attack: a comment that directly attacks another Redditor (e.g. “your mother was a hamster and your father smelt of elderberries”) or otherwise shows contempt/disagrees in a completely non-constructive manner (e.g. “GASP are they trying CENSOR your FREE SPEECH??? I weep for you /s”)
2. Overt bigotry:  the use of bigoted (racist/sexist/homophobic etc.) language, whether targeting any particular individual or more generally, which would make members of the referenced group feel highly uncomfortable

However, the problem with only measuring Toxic comments is it biases against subreddits that simply tend to be more polarizing and evoke more emotional responses generally. In order to account for this, we also measured Supportiveness in comments – defined as language that is directly addressing another Redditor in a supportive (e.g. “We’re rooting for you!”) or appreciative (e.g. “Thanks for the awesome post!”) manner.

By measuring both Toxicity and Supportiveness we are able to get a holistic view of community health that can be used to more fairly compare and contrast subreddit communities.