The CHOsen KHHV
- Jul 11, 2020
- 34d 5h 27m
would read with the paragraphs
Online Hatred of Women in the Incels.me Forum: Linguistic Analysis and Automatic Detection Sylvia Jaki,1 Tom De Smedt,2 Maja Gwóźdź, 3 Rudresh Panchal,4 Alexander Rossa,5 Guy De Pauw6 1 University of Hildesheim, 2 University of Antwerp, 3 University of Munich, 4 Columbia University, 5 University of Hull, 6 Textgain Abstract This paper presents a study of the (now suspended) online discussion forum Incels.me and its users, involuntary celibates or incels, a virtual community of isolated men without a sexual life, who see women as the cause of their problems and often use the forum for misogynistic hate speech and other forms of incitement. Involuntary celibates have attracted media attention and concern, after a killing spree in April 2018 in Toronto, Canada. The aim of this study is to shed light on the group dynamics of the incel community, by applying mixed-methods quantitative and qualitative approaches to analyze how the users of the forum create in-group identity and how they construct major out-groups, particularly women. We investigate the vernacular used by incels, apply automatic profiling techniques to determine who they are, discuss the hate speech posted in the forum, and propose a Deep Learning system that is able to detect instances of misogyny, homophobia, and racism, with approximately 95% accuracy. Keywords: misogyny, hate speech, social media, forensic linguistics, text analytics, text profiling 1 Introduction On April 23, 2018, 25-year old Alek Minassian killed 10 and injured 16 in Toronto, Canada, by driving a rented van into pedestrians “like he was playing a video game” (The Telegraph, April 23, 2018). This incident has become known as the Toronto van attack. Shortly before the attack, he posted a cryptic message on Facebook stating: “The Incel Rebellion has begun!” (Reuters, April 24, 2018). The term is shorthand for involuntary celibates and refers to an online community of men that blame women for their celibacy (Ging 2017). Evidence shows that Minassian was inspired by the 2014 Isla Vista killings, where 22-year old Elliot Rodger stabbed and shot 6 people to death near the UCLA campus, after posting a video on YouTube in which he complained about being rejected by women, while envying sexually active men (Blommaert 2017; Larkin 2018). While misogyny is not inherently linked to radicalization, these examples show that it can play a role in radicalization processes. Several studies suggest that “echo chambers”, online forums where like-minded people share disparaging views, can be a catalyst for radicalization (e.g., Accepted Manuscript for Journal of Language Aggression and Conflict, July 8, 2019, https://doi.org/10.1075/jlac.00026.jak © John Benjamins. To re-use, contact the publisher. Colleoni, Rozza, and Arvidsson 2014). To illustrate this, on November 7, 2017, the /r/incels community was banned from Reddit for spreading misogyny.1 However, a new incel forum, Incels.me, appeared online shortly after, only to be suspended for spreading violence and hate speech in October 2018.2 In this paper, we analyze the discourse in the Incels.me forum to shed more light on the nature of the incel movement and its affinity to violent extremism. We have combined quantitative techniques from Natural Language Processing (NLP) with an in-depth qualitative analysis, which allows for a systematic description of the data and the development of an automatic detection system using Machine Learning (ML) techniques. In section 2, we provide a brief overview of research in the field of online misogyny. In section 3, we outline the research questions motivating the study, the dataset compiled for the quantitative and qualitative analysis, and an overview of the methods employed. Section 4 examines the language variation used in the forum’s discussion threads, and section 5 discusses the forum’s users. Section 6 contains a description of the group dynamics in the forum and an assessment of the community in terms of violent extremism, as well as a case study with automatic hate speech detection. In section 7, we formulate our conclusions. 2 State of the Art Misogyny has reached a new peak in the last years and this is inherently linked to its expression on the World Wide Web, particularly social media, as these are found to be ecosystems that encourage trolling, name-calling, and profanity (Rego 2018, 472; see also Haines et al. 2014). This phenomenon is caused by what Suler (2004) calls the online disinhibition effect: aspects such as anonymity of the producer and invisibility of the recipient increase the likelihood of people acting out. Misogyny online is often expressed via flaming, “messages showing attributes such as hostility, aggression, intimidation, insults, offensiveness, unfriendly tone, uninhibited language, and sarcasm” (Turnage 2007, 44). Related terms are e-bile, which is any utterance communicated with the help of technology and perceived as hostile by a sender, receiver, or outsider (Jane 2014a, 533), or hate speech (used in the analysis at hand), which refers to “any communication that disparages a person or a group on the basis of some characteristic such as race, color, ethnicity, gender, sexual orientation, nationality, religion, or other 1 As of July 2018, r/braincels was still active on Reddit, featuring a mix of less inflammatory incel rhetoric and role-playing. 2 “The suspension of incels.me”, https://domain.me/the-suspension-of-incels-me characteristic” (Nockleby 2000).3 Mantilla (2013) describes another specific case of misogyny online, gendertrolling, which is used to silence women online by using gender-based insults, other vicious language, and threats (see also Mantilla 2015). This phenomenon often concerns journalists or activists (e.g., Rego 2018) and outspoken women in the hypermasculine culture of gaming communities (e.g., Salter and Blodgett 2012). Such attacks against women online include portraying women as unintelligent, hysterical, or ugly, and they often express rape threats or fantasies (Jane 2014a, 533), which can be called Rapeglish (Jane 2017). As Kleinke and Bös (2015, 50) show, online discussion groups display intergroup rudeness such as the construction of a majority out-group outside the discussion forum and persecuted out-groups within the in-group. Usually, hate groups create in-group identity through selfportrayal in positive terms, as any group is bound to look for ways to see the in-group as positively distinctive from other groups (cf. Brown and Zagefka 2005, 57). This does not apply to involuntary celibates, a group belonging to the so-called Manosphere, “hard-line men’s rights and interest communities online” (Jane 2017, 662), because they typically have a very negative self-image (section 5). Studies that analyze misogyny online, like Bou-Franch and Garcés-Conejos Blitvich (2016), reveal that violence against women is often justified by negative out-group depiction, i.e., blaming women. In the case of incels, out-group identity is created by depicting women and normies (attractive men) as flawed and deplorable. In their work on involuntary celibacy, Donnelly et al. (2001, 167) showed that test persons “appeared to be using the Internet more to find moral support than for sexual stimulation. For most, the Internet was used to create a sense of community and to fill emotional needs”. This is what Dholakia and Bagozzi (2004, 258) also refer to as the companionship motive for participating in virtual communities. This motive is true for Incels.me, where users turned to because they felt isolated. The users felt part of an in-group, creating a sense of community between like-minded people.4 The downside of this sense of community is that it also created a toxic platform for anti-feminist radicalization (Ging 2017). For example, Blommaert (2017, 20) highlights the importance of online communities in the radicalization of Elliot Rodger, which resulted in the incel expression to go ER (to commit a mass killing spree): Rodger derived from his engagement in those communities an absolute certainty about his identity as a victim of a world that conspired to steal away his (sexually 3 There are different concepts on an international level, e.g., the EU’s Code of Conduct on countering illegal hate speech online defines hate speech as “the public incitement to violence or hatred directed to groups or individuals on the basis of certain characteristics, including race, colour, religion, descent and national or ethnic origin”, http://europa.eu/rapid/press-release_MEMO-18-262_en.htm. 4 There is even merchandise such as T-shirts that could be purchased on the website: https://teespring.com/stores/fast-banana. focused) happiness, and enough of a commitment to take this logic of action to its very end, where the victim becomes the perpetrator. Cases like Elliot Rodger and Alek Minassian could be labelled as lone-wolf terrorism at first sight, but, as Hamm and Spaij (2017, 59) argue, “[l]one wolves do not operate in isolation, and their radicalization can be traced to various social networks”, among them online networks such as Incels.me. The internet enables small hate groups to reach a large audience, whereas in pre-internet times extremists had to work in more isolation (Douglas 2009, 155). Due to cases like Rodger and Minassian, the incel ideology is often considered a form of violent extremism, which is “[t]he belief that an in-group’s success is inseparable from violence against an outgroup” (Berger 2017, 6). Accordingly, Zimmerman, Ryan, and Duriesmith (2018, 3) demand it to be treated “with the same seriousness as other forms of violent extremism”. Given that incels suffer from being celibate and blame women for their situation, one aspect of radicalization, termed the threat and vulnerability gap by Berger (2017, 4), is particularly relevant in the light of involuntary celibates, namely, that the in-group is seen as increasingly vulnerable, and the out-group(s) as increasingly threatening. In this context, we will also examine to what extent incel vernacular constitutes dangerous speech, as subcategory of hate speech or “speech that increases the risk for violence targeting certain people because of their membership in a group, such as an ethnic, religious, or racial” (Brown 2016, 7). 3 Methods & Materials Aim of the Study & Research Questions Broadly, there are two variables in sociolinguistics studies: a social variable that determines language variation (e.g., age, gender) and a linguistic variable (e.g., vocabulary, style). To study the language use on Incels.me, we assumed that it is produced mainly by men, of different age, education and ethnicity, and compared it to a reference corpus of Wikipedia articles, which have a neutral style but are biased by being mainly written by men, of different age, education and ethnicity.5 This controls the social variable, so that linguistic variation is more likely to be determined by the medium (i.e., Incels.me). We employed mixed methods with quantitative tests (in Python code using toolkits Pattern, LIWC, Keras), along with qualitative study to back up the quantitative results and provide a more complete view. The aim of ourstudy is to provide insight into the discourse of hatred against women that is fostered in unmoderated online forums like Incels.me. We will analyze the misogynistic rhetoric in the forum (Q1. How do 5 https://en.wikipedia.org/wiki/Gender_bias_on_Wikipedia users speak about women and other out-groups?), who the users are, as well as the reasons for their behavior (Q2. What do users say about themselves and other members of the in-group?). We believe that this is an important step in understanding this online community. There has been increased media coverage on involuntary celibates, but, in fact, Incels.me discussion threads related to the Toronto attack show that incels often feel misrepresented in the media, being framed as white conservatives: “I’m sick of these Ameritards trying to label us all as white and right wingers. They don’t want to learn about inceldom all they want to do is push their narrative and political agenda”. According to another user, the forum is a “community made up of disparate groups ranging from edgy shitposting teens to Salafi Jihadist apologists and Christian Identity white supremacists”. We also attempt to shed light on the potential danger of forums like Incels.me, since involuntary celibates have often been portrayed as violent extremists after Alek Minassian’s attack (Q3. How likely is it that the discourse in unmoderated forums for involuntary celibates leads to violent extremism?). Materials We used the Pattern toolkit for the Python programming language (De Smedt and Daelemans 2012) to crawl and collect approximately 65,000 messages (about 1.5M words) from Incels.me. The resulting dataset covers 3,500 threads (i.e., discussion topics) with messages by 1,250 users, posted in the 6-month period between November 2017 and May 2018. Approximately, each thread contains about 20 messages, by 10 users. Each user posted about 50 messages on average. About 350 new messages were posted each day. The number of posts increased significantly in April, after the news of the Toronto van attack, as shown in Figure 1 below. Figure 1. Timeline of messages posted on Incels.me (November 2017 – April 2018). 0 1000 2000 3000 4000 Dec 2017 Jan 2018 Feb 2018 Mar 2018 Apr 2018 TIMELINE OF POSTED MESSAGES Methods This study combines a range of state-of-the-art quantitative approaches from Natural Language Processing (NLP) and Machine Learning (ML). To analyze the rhetoric present in the discourse, we perform word frequency tests by looking at keywords and word combinations. We selected a set of common offensive words to assess how much hate speech exists in the forum, heuristically chosen from movie culture and lists of prevalent swear words around the internet (e.g., bitch, faggot, nigger). For user profiling, we employed sentiment analysis as well stylometric techniques for demographic and psychological profiling. Recently, in the field of ML, a growing number of attempts have been made to automatically detect hate speech (for an overview, see Schmidt and Wiegand 2017). Employing deep neural networks, we attempt to detect misogynistic vernacular online, evaluating predictive accuracy using customary metrics in computational linguistics (F1-score). In addition, we manually reviewed a subset of a 100 discussion threads (~3%) for a qualitative analysis that is based on self-provided categories and essentially follows a content-analytical approach. The threads were selected to reflect the variety of topics discussed in the forum, including definitions of inceldom, the prospect of being an incel, feminism, female promiscuity, male attractiveness, ethnicity, suicide and violence as coping mechanisms, and so on. Providing insight into the debate in the forum does not go without showing concrete examples, hence the reproduction of discriminatory or offensive language is unavoidable (see Marx 2018, 1 for a similar observation). All of the examples are indicated as published, i.e., without changing spelling or grammar mistakes. 4 Text Analysis This section provides an analysis of incel rhetoric, and its relation to hate speech. We identify three broad trends in the messages that users posted and the language that they used: 1) negative discourse that pertains to being incel, 2) offensive discourse that targets (attractive) women (and men), and 3) infighting and bullying while competing for in-group status. 4.1 Quantitative Analysis Keyword Analysis We compared 50,000 Incels.me messages to 50,000 more neutral texts, composed of 40,000 paragraphs from random English Wikipedia articles (which are moderated for neutrality) and 10,000 random English tweets (to account for internet slang). In theory, all words should be evenly distributed between both sets. For example, most sentences require function words such as a and the to be comprehensible. However, there are also content words that occur more frequently in a specific context, e.g., sports tweets will often mention winner while messages on Incels.me may often mention loser. 6 Below is an overview of words that 9 out of 10 times occur in incel messages, and which are statistically significantly biased (for a chi-square test of word counts in incel messages vs. neutral texts, p < 0.05 for these words). Keywords Top 10 The top 10 of the most frequent significantly biased keywords consists of I (30,000x), you (20,000x), n’t (10,000x), do (7,500x), if (6,500x), just (6,000x), like (6,000x), me (6,000x), get (4,500x), even (4,000x). The occurrence of personal pronouns (I, me, you) is explained by the difference in genre: Wikipedia articles tend to represent facts while Incels.me posts reflect personal opinions. Wikipedia also avoids informal contractions, using do not instead of don’t. Keywords Top 100 The top 100 of biased keywords has references to gender (female/male, girl/guy, woman/man), physical traits (attractive, fat, pretty, ugly, white), and sex. Swear words (fuck, fucking, shit) and internet slang (kek, lmao, lol) occur frequently. Also common are more negations (nothing, never), negative adjectives (bad, hard), modal adverbs of uncertainty (maybe, probably, why), and emotion verbs (hate, hope, want, wish). Some words constitute coded language. For example, Chad or Tyrone are derogatory denominations for an attractive and successful young man (cf., alpha male, bad boy, bro, jock) while Stacy denotes an attractive roastie, a promiscuous young woman. A central metaphor in the incel jargon refers to The Matrix, a popular science fiction film in which the protagonist is offered either a red pill (knowledge and misery) or a blue pill (ignorance and bliss). In the Manosphere,7 the red pill refers to the belief that men are oppressed (cf. Van Valkenburgh 2018 for a detailed description). Particularly in incel subculture, this means that a minority of attractive men have access to the majority of all women, while the other men are left to compete over a minority of women. The black pill (Figure 2, rank 89) is the nihilistic belief that unattractive men will never “score”. 6 “Incels.me top 2,000 keywords”, View: https://docs.google.com/spreadsheets/d/1j6GNs075HQjF-D3GWbSBk3QEAgi1ms-AfYOO8kGPvxQ
7 “Manosphere glossary”, https://rationalwiki.org/wiki/Manosphere_glossary Figure 2. Top 100 keywords. WORD COUNT RANK women 3473 11 fuck 2547 15 fucking 2322 17 want 2266 19 shit 2258 20 why 2165 22 Chad 2098 24 ugly 1539 36 sex 1480 37 girl 1195 45 bad 905 62 pretty 774 75 maybe 703 80 hope 693 83 blackpill 630 89 Figure 3. Top 1,000 keywords. WORD COUNT RANK myself 556 101 dick 542 105 virgin 510 114 rape 276 185 dumb 230 215 pathetic 214 226 faggot 214 227 girlfriend 210 234 slut 169 267 cum 120 350 scum 81 499 nigger 76 527 landwhale 57 617 horny 48 690 pillow 38 799 Keywords Top 1,000 The top 1,000 of biased keywords (Figure 3) has references to romance (girlfriend/boyfriend, date), sexuality (horny, masturbating, virgin), and pornography (ass, cum, cunt, dick). Misogyny (bitch, slut, whore) and other hate speech (faggot, nigger, retard) start to surface. There are many more negative adjectives (e.g., brutal, disgusting, dumb, lazy, insane, lonely, pathetic, useless) and verbs expressing aggression (crush, die, hate, kill, rape, shoot, slay). Keywords Top 2,000 Beyond the top 1,000 biased keywords, we find more (and more specific) references to male sexuality (boner, micropenis, urges), more negative adjectives (infuriating, submissive), and more aggressive verbs (beat, stalk, torture). There are noticeably many more compounds of a disparaging nature (e.g., noodlewhore, soyboy, wageslave). Word Combinations Commonly co-occurring adjective-noun pairs are high school (~200x), white man (200x), black woman (80x), good looking (125x), ugly man (125x), ugly woman (30x), social anxiety (30x), social skills (65x), low IQ (50x), big dick (40x), big deal (30x), little bitch (15x), and little girl (15x). There are many more, usually referring to physical attributes and the size of body parts. As an example, Figure 4 shows the context of the word women in the dataset, with size representing frequency. Figure 4. Words preceding or succeeding the word women. Hate Speech We examine to what extent incel vernacular constitutes hate speech, to understand how disparaging the incel discourse is, in particular towards women. To get a first impression, we looked for the occurrence of 10 offensive words in each discussion thread, in particular words that constitute misogyny (bitch, landwhale, roastie, slut, whore), homophobia (fag, faggot, soyboy), and racism (coon, nigger). Using this approach, about 30% of the threads are misogynistic. About 15% are homophobic, and 3% are racist. For example, three of the most hateful discussion threads are titled “Caught this fat bitch swiping left on Tinder”, “Faggots are just as bad as women!”, and “Black man chases femoid through streets”. About 5% of all messages in our dataset contain one or more of these 10 offensive words. By comparison, the likelihood that a random Twitter message contains one of these words is 0.4% (about 40 out of 10,000 tweets), or over 10x less than incel messages. About half of the users in our dataset posted hateful messages at one time or another. Nearly 500 use misogynistic slurs, 250 (also) use homophobic slurs, and 75 (also) use racial slurs. But most users posted no more than two or three hateful messages. About 10% of the users are responsible for the majority of the hate speech. The most aggressive user posted nearly 500 hateful messages in as many threads in a 6-month period. The reported numbers are an approximation based on a small selection of 10 offensive words, while hate speech is a heterogeneous phenomenon that does not necessarily always involve attitudes of hatred (Brown 2017, 432). Most likely, our estimates err on the conservative side. If we do a search for 50 offensive words, word combinations (dumb girl, fat chick), and verb constructions (beat / teach her), the percentage of misogynistic discussion threads steadily rises over 50%, most of it again posted by 10% of the users. Figure 5 presents an overview. Figure 5. Threads with hate speech. MISOGYNY HOMOPHOBIA RACISM 0% 25% 50% 75% 100% TOP 10 SLURS TOP 50 SLURS ... THREADS WITH HATE SPEECH 4.2 Qualitative Analysis Homophobia Wikipedia states that incels are mostly heterosexual and white.8 The former is true for the users of the Incels.me forum (the latter is not).9 For example, one user states: “gays cant be incel theyre too busy having sex and shopping at bed bath and beyond”. Homosexuals are often despised and called faggots (“being a faggot is a mental disorder same with being trans”), and only once in the subsample of 100 threads does a user refer to himself as homosexual. The pervasiveness of heterosexuality can be attributed to 1) the fact that homosexuals are not wanted in the forum, and 2) that the world views (particularly misogyny) of heterosexual incels may not be something that homosexuals can relate to. Racism It has been shown that the anti-feminist movement is connected to white supremacism and to anti-semitism (Kämper 2011), and the same has been suggested for the incel community in the literature (e.g., Zimmermann, Ryan, and Duriesmith 2018) and various news media10. It is impossible to say whether the majority of Incels.me users are white men, but our data implies that this may be less true than expected. There are some mentions of Hitler or gas chambers and some anti-semitic remarks (“The damage that Jews have done to our species is almost unfathomable. Fucking kikes”). There are some racist comments in general, targeted either at other users (“go back to freaking syria, find some camel or goat and fuck”) or at a group as a whole (“Muslims need to be exterminated”). But in the 100 threads we examined such cases were sporadic rather than systematic. However, race is an extensively treated topic in the forum, but primarily in relation to which race has more incels and which race has more (dis)loyal women. There is a general consent that unattractive non-white men have a harder time than unattractive white men (“Ethnic subhumans like me […] are either instantly rejected or friendzoned”), which is referred to as the “just be white theory”. There is also consent that “[r]ace is a big part of looks” and that Indian incels (currycels) are most badly off. Other ethnic groups mentioned in this context are Asians, and less often African Americans and Arabs. This may reflect, to some extent, the ethnic variety of the forum. One user believes that only half of the people on Incels.me are 8 “Incel”, https://en.wikipedia.org/wiki/Incel 9 We also found sporadic evidence of sexual deviation, including pedophilia (“A lot of pedos in this community actually”), necrophilia (“No pics of its dead body Disappointed tbh”), and zoophilia (“I have sex with willing mares in heat”). 10 “Growing Extremism Targets Women”, https://www.nytimes.com/2018/05/09/world/americas/incels-toronto-attack.html white. Other users claim that most incels are not white, and one user substantiates this by referring to a poll in the forum. Misogyny Misogyny can be defined as “an unreasonable fear or hatred of women that takes on some palpable form in any given society” and “sexual prejudice that is symbolically exchanged (shared) among men” (Gilmore 2001, 9). Misogyny is one of the most pervasive features of the Incels.me forum. One user says: “I think acceptance of ‘misogyny’ and disregard for outsiders is what separates us from other communities”. Hating women is seen as an inherent characteristic of incels (“hating women is a requirement”), who attribute their situation to undesirable female behavior. Indian and Asian women are hated the most (“Ethnik Indian and noodlewhores are the worse of the worst”) because they are said to prefer only white men, while at the same time Asian women are recommended as prostitutes. Unsurprisingly, the forum is thus replete with derogatory designations for women (cum dumpster, cum rag, it, roastie, slut, whore, etc.). The reason why incels have such a negative attitude towards women is the perceived female “degeneracy”, i.e., an exclusive interest in (sex with) attractive men, who are also a target of incel hatred. Women are portrayed as being shallow, immoral, promiscuous, and responsible for the incels’ isolation. Users post pictures of women taken from the news and social media, upon which the women’s physical traits are derided, especially obesity (“I loathe fat women. Bunch of useless fucking hogs”). The users believe that men generally “date down”, while women are not willing to do so. However, it is important to emphasize that a lot of features of incel misogyny extends to online misogyny in general, such as detailed depictions of sexual violence, extreme insults, or the shaming of women for bodily flaws (Jane 2014b). Nevertheless, not all users hate all women. A discussion thread designed as a poll asks whether the forum users hate all women and shows that opinions are mixed: about 55% (33x) answer positively, while about 45% (28x) answer negatively (“No, I like my mom and grandma”).11 One user states the following: “I don’t hate all woman. And I believe that most brothers here in the Forum also don’t hate them. We hate the situation we are in”. Anti-feminism Many users criticize feminism, which is seen as a threat to masculinity, or as brainwashing (Ging 2017, 3), or the reason for “loneliness and suicides”, “the decay of society”, “terror”, and so on. Multiple messages call for the abolishment of women’s rights, to the extent that women 11 As self-reported comments in the poll suggest, 45% hate only some women, or no women at all. should not be allowed to vote and should be regarded as the property of their husbands: “Female HAVE TO become property again. They should not have the right to even SPEAK without male permission”. Concrete advice includes treating women like wild animals (“You have to get used to seeing them as animals”) and keeping them on a leash and caging them when their owner leaves the house. In turn, men should have a legal claim on virgin brides, and the government should mediate their desire for sexual contact by “redistribut[ing] pussy”, i.e., legalizing and subsidizing prostitution. This extreme dehumanization of women12 entails several other more or less systematic opinions about women’s rights, such as capital punishment (e.g., stoning, acid throwing) of adultery or wearing “revealing clothing”. 5 Text Profiling Text Profiling pertains to gaining insight into the demographic and psychosocial aspects of the (anonymous) author of a text, by the words that they use, the topics that they discuss, their personal writing style, and metadata such as what usernames they choose. Relevant techniques from NLP include sentiment analysis, and age, gender and personality prediction (Argamon et al. 2009), which have for example been used to identify school shooters (Neuman et al. 2015) or jihadists on social media (De Smedt, De Pauw, and Van Ostaeyen 2018). We will attempt to shed further light on who the incels are, in terms of age, gender, ethnicity, and state-of-mind, using a combination of quantitative and qualitative analysis to reveal a negative mindset of mostly male adolescents who suffer from anger issues, uncertainty, and social inhibition. The main risk of using automated techniques to make inferences about the author of a text is that we cannot be 100% sure what we detect: the author’s characteristics, or the picture that they want to convey of themselves. This problem is also acknowledged in forensic linguistics, where stylometric methods may be tricked by authors who do not use their own writing style (Brennan, Afroz, and Greenstadt 2012). In the context of Incels.me, this means that we need to draw conclusions carefully. For example, some members of the forum may have adjusted their writing to the style commonly used in the forum to become part of the in-group. They may have staged themselves as young and socially inhibited. But it is likely that this is only true for a fraction of the users. 12 Dehumanization in discriminatory talk has often been subject to Critical Discourse Analysis (Musolff 2012). 5.1 Quantitative Analysis Sentiment Analysis Sentiment Analysis is the NLP task of automatically predicting whether a text is positive, negative or neutral (Liu 2012). Using the Pattern toolkit (75% accuracy), we find that over 60% of Incel messages are negative. By comparison, less than 5% of articles on Wikipedia are negative. Demographic Profiling We used the Textgain API to detect age, gender, education and personality of the forum’s users (~80% accuracy).13 Most of them are flagged as adolescent, male, and slightly less educated (Figure 6). Figure 6. Sentiment analysis & profiling. In prior work, Pennebaker (2011) demonstrated how personal writing style contains subtle cues for age, gender, state-of-mind, and personality. For example, statistically, women tend to use more pronouns (I, my, we) to talk about relationships, while men tend to use more determiners and quantifiers to talk about objects and concepts (the, one, more, …), and adolescents use more informal language and profanity, while introverted persons use more negative adjectives. The results indicate that 35% of the users are women, likely because most of the discourse is about interpersonal relationships and physical functions, topics that are statistically more prevalent with women (Newman et al. 2008, 219). It could be that the API is correct, and that there really are 35% women participating on Incels.me (e.g., fake profiles,) but given the level of aggression we consider this chance slight. Younger age correlates with lower education, suggesting that the users are not necessarily less educated than other people, although there are multiple references to low IQ (~50x), which may indicate low self-esteem (or contempt). 13 “Textgain Real-time Text Analytics”, https://www.textgain.com incels.me Wikipedia 0%% 2500%% 5000%% 7500%% 10000%% NEGATIVE NEUTRAL POSITIVE SENTIMENT 25+ 40.0% 25− 60.0% AGE M 50.0% F 35.0% N 15.0% GENDER HIGH 45.0% LOW 55.0% EDUCATION E 55.0% I 45.0% PERSONALITY Psychological Profiling The Linguistic Inquiry and Word Count analysis (LIWC; Pennebaker, Francis, and Booth 2001) automatically identifies psychological categories for common words, e.g., cried = Affective, Negative, Past, Sadness, Verb. In general, the results support our earlier observations that the forum users tend to use more swear words, more personal pronouns, more modal adverbs, more negative adjectives, and fewer positive adjectives. They tend to express more negative emotions like anger and uncertainty, and display social inhibition (i.e., avoidance, anxiety). They talk less about their family, work, hobbies, goals, beliefs, and more about relationships and sexuality. 5.2 Qualitative Analysis Due to the heterogeneity of the community, as well as the subjective nature of attractiveness, there is no standard in-group definition of inceldom. The question of who can be considered to be an incel repeatedly came up in the forum, mainly because not all users of the forum were welcome, especially normies or volcels (voluntary celibates). All sorts of threads contained discussions about who is a real incel (truecel), i.e., a legitimate and particularly a badly-off user, which is important since it often entails receiving emotional support. Demographic Profiling Gender. The forum contains a section with terminology, rules, and FAQs. It defines an incel as a “[p]erson who is not in a relationship nor has had sex in a significant amount of time, despite numerous attempts”. A person here has to be understood as a man, since women are considered not to have problems finding partners, which is noted both in the rules section and in the threads. The minimal criteria of inceldom are: being male, and not having had a sexual partner for a long time. The majority of users understand that this entails being a virgin, without hope of changing status (“Im going to be a kissless virgin for life. its truly over”). Age. The age of the average user is harder to determine, since most users did not make explicit reference to their age, nor their place of residence. However, many of the users appear to be adolescent men. This is implied by references to school (my school, my college), parents (my mom, my dad), and video games that are popular with teens (League of Legends, World of Warcraft). But other users seem to be considerably older. The age that they self-report in our 100 sample threads varies between 21 and 33. That a non-negligible part of the forum consists of older members can also be inferred from indirect references to age, from references to a former university, and by talking about school in the past tense. Race. There is no definite evidence that Incels.me users are predominantly white, contrary to what is often reported about incels (see also section 4.2). Physical Profiling The users generally attribute their lack of sexual experience and social contacts to ugliness. 14 Many refer to the absence of physical attraction by their username,15 such as Hunchback, MicroDong, YunGandUgly, Uglyinkel, blackletcel, Asianmanletcel, patheticmanletcel (where -let refers to a shorter height). Some users describe their physical shortcomings openly (“manlet stature with abnormally long arms and a huge skull”) because this attracts attention, pity, and confirmation of their perceived hopeless situation. The incels’ views on beauty are elaborate. They discuss it using coded designations such as lookism or LMS, which involves a categorization system based on specific facial features such as a broad chin or a normal nose, build, height, personality, and/or normally-sized genitals. Psychological Profiling Isolation. There is another defining criterion with general consent: being alone. Some incels have never had a date (see also Donnelly et al. 2001, 163) or any contact with women (“I’ve never touched a non-related female even for a handshake or something”). Even establishing any social contact whatsoever seems to be difficult for some incels (“no one ask me to hang out in my entire 22 years of existence. Not even men or my neighbors”). Inhibition. Some users argue that mental disorders like autism can also render a person an incel. This opinion is in line with the forum’s rules section, where mentalcels are implicitly included as a “[t]ype of incel whose reason for failure in relationships/sex is related to mental illness or major insecurities”. Some users report taking psychotropic drugs and having been diagnosed with schizophrenia, autism, and/or personality disorders, as in “I’m diagnosed autistic aspergers, social anxiety and anti social personality disorder aka sociopath”. These members have usernames such as Schizoidcel, Psychocel, or HopelessMentalcel. Anxiety and depression are common as well, but these afflictions are believed to be a consequence of the lack of physical attractiveness, since “those are just usual symptoms of being a subhuman outcast”. Some users feel left out because of their low IQ, which is discussed as an inherent feature of inceldom in one thread: “We are not only uglier than beautiful people but also dumber on average”. Usernames that hint at a low IQ or depression include retarded_dumbshit, lowIQretard, Sadness, Eternaldarkness and Melancholy_Worm. 14 See DiMauro (2008) and Donnelly et al. (2001: 165) on the connection between self-image and an involuntary lack of sexual experience. 15 Usernames can be divided into three groups: 1) names based on incel or -cel (e.g., theonlytruecel, Poverty Cel, Weirdcel, Suicel), or 2) names referring to the mental disposition of the user (fukmylyf, lonelyistheworld, Subhuman Trash, itsOVER), or 3) names based on incel terminology (Blackpill101, ropenotcope, Heightframeface, Cuck). These categories are not mutually exclusive. Negativity. The comments by the users show that incels suffer because of their condition (e.g., feeling lonely, deprived of sexual contacts, stared at in public, lacking a career).16 There is a large number of discussion threads (~25%) containing “it’s over”, which stands for the lack of hope to escape their situation. There are a few positive users in the forum who address their situation, for example by going to the gym (gymcel), but this is often experienced as a strain because of the high number of Chads in the gym (“gymcelling is literally torture”). When discussing what to do on a free night, self-reported activities point to an isolated life, with solitary exercising, drinking, playing video games, cleaning, masturbating, or doing nothing at all as the most common answers. Cautiously positive messages like “I’m probably coping but maybe there is a glimmer of hope” are rare. This prevalence of negativity is substantiated by our sentiment analysis. Neglect. Self-pity and hopelessness drive part of the users to neglect their daily grooming, as in: “I haven’t brushed my teeth in over six months. I did shower yesterday, but it was for the first time in like 2 to 3 weeks”. This contributes to a vicious circle of depression. Once their impression of hopelessness is reinforced by other users, they call it suicidefuel (in contrast to lifefuel). Direct messages expressing the desire for suicide are quite common: “This is so fucking brutal and I also want to end it as soon as possible. I don’t even get emotional when thinking about suicide anymore. I’m just waiting for the right time”. Several users claim a right to euthanasia (“Painless suicide pills should be freely distributed to people like us”). Finally, the most prominent topics in all discussions are sexuality (from physical, psychological as well as sociocultural perspectives), sexual selection, and sex (from a pornographic perspective). Incels are concerned with what constitutes attractiveness, how to approach women, and how and why they are unsuccessful in this regard. The users reflect on the physical aspects of sexuality and invent ranking systems (e.g., by chin or penis size), discuss detrimental psychological aspects (e.g., anxiety, anger, loneliness), and construct theories about the sociocultural aspects of sexual relationships: “Throughout history women frequently killed their own offspring to sleep with the conquerors”. The users’ references to pornography are ambivalent: they either blame the pornography industry for their low self-esteem (most notably, insecurity about penis size), or praise mainstream heterosexual pornography. Some messages include hyperlinks to freely available pornographic content such as pornhub.com, advertised in the forum as “convincing rape porn”, “humiliation”, etc. 16 “How Men Get To The Point Where They Identify As Incel”, https://www.huffingtonpost.ca/2018/06/05/incel_a_23451320 Figure 7. Summary of core and peripheral aspects of inceldom. Profile Summary Within the community, there is considerable disagreement on which defining factors are the most prevalent, and also on the relation between ethnicity and attractiveness. Other potential criteria are a lack of stable personality (as a long-term result of physical and mental flaws) and success in one’s professional life (multiple incels seem to be unemployed). Figure 7 sums up factors often discussed in the forum as defining inceldom. While some are apparently more defining than others, a prototypical incel would be somebody like: “22 yr old kissless, dateless, virgin. Chubby cheeks, crooked nose, 5"10,17 balding at age 22, weak frame, and awkward voice. Shitty career prospects. Mild autism and social anxiety. My life is fucking fucked […]”. 6 Relation to Violent Extremism In general, the earliest reference to an Incel Rebellion is only made on April 24, the day after the Toronto van attack, in a thread titled “Well I’ll be damned”. However, references to a Beta Uprising, which is the term that the forum’s users seem to prefer, occur as early as February, in a number of threads where some users attempt to incite others to commit acts of violence (“Even if it’s shooting up a school, don’t sit back and do nothing”). This is illegal according to the International Covenant on Civil and Political Rights,18 but most likely protected by the First Amendment of the United States Constitution and Brandenburg v. Ohio, 395 U.S. 444 (1969). 17 Incels partly assess their attractivity by penis size, which is a common focus in a penis-centered view of sex (Plummer 2005, 179). 18 “International Covenant on Civil and Political Rights”, https://www.ohchr.org/EN/ProfessionalInterest/Pages/CCPR.aspx NO SEX MALE VIRGIN LONELY UGLY HATE WOMEN MENTAL DISORDER NO SELF ESTEEM NO SOCIAL LIFE NO CAREER ETHNIC MINORITY FORUM’S RULES MENTIONED CONSTANTLY MENTIONED LESS OFTEN MENTIONED VERY OFTEN That the problem persists is shown by the case of Christopher Clearly, a 27-year-old virgin from Denver accused of threatening to kill women in a mass shooting for having been turned down repeatedly (January 2019).19 As a first cue, Figure 8 shows the relative frequency of messages that contain common references to violence (~2%), i.e., the words kill, rape and/or shoot. Notably, these references toned down around Christmas, but the frequency of messages that contain kill then increased in the weeks before the attack. In general, violence against women is often judged as acceptable or even desirable. It is important to discuss how likely it is that users of Incels.me (or replacement forums after its suspension) would resort to actual violence, for example under peer pressure. In this section, we examine whether and in what ways the forum is compatible with the definition of violent extremism, and if our data indicates an increased risk of terrorist attacks by involuntary celibates. Figure 8. References to violence on Incels.me (November 2017 – April 2018). Group Dynamics The reward that users derive from a forum like Incels.me is approval, as they spread ideas or initiate threads that are considered as “high IQ”. Such ideas often involve complex (but also absurd) reasoning, leading to “the emerging of individual and collective identity categories (‘victims’, ‘beta males’) and commonly ratified (‘normal’) lines of action” according to Blommaert (2017, 16). Messages that proliferate misogyny or incite crime help a user to cement his reputation as an alpha user. As as result, the Incels.me forum brims with subcultural language, partly composed of in-group terminology for much-discussed agents, and partly composed of youth language in general. The terms most often recurred to can broadly be grouped in 1) designations for themselves (see above for different word formations including the creatively 19 “Christopher Cleary: 5 Fast Facts You Need to Know”, https://heavy.com/news/2019/01/christopher-cleary 0.0% 0.5% 1.0% 1.5% 2.0% Nov 2017 Dec 2017 Jan 2018 Feb 2018 Mar 2018 Apr 2018 KILL RAPE SHOOT REFERENCES TO VIOLENCE coined morpheme -cel), 2) their targets, i.e., women (Staceys, roasties, femoids) and attractive men (Chads, Tyrones), and 3) the incel ideology (e.g., blue pill, red pill, blackpill, lookism, go ER). This aspect is particularly important, since online communities emerge through language, and the use of a group-specific language determines the boundaries of the community (cf. Haythornthwaite 2009, 123f). One reason why people were able to engage in the toxic communication on the forum is that there was little or no content moderation. There were some rules restricting what could be said and how, and infringement was penalized by users being banned. These included: 1) no persecution or bullying of other users, 2) no race-baiting, 3) no stories of sexual experiences, 4) no discussion of illegal activities, and 5) violent content and pornography need a spoiler tag. The rules also stated that positivity is appreciated “as long it is not bluepill ideology or whiteknighting”. These rules were enforced only half-heartedly, and banned users could ask the moderator to lift their ban again. Hence, Incels.me data is rife with hate speech, filled with discussions about hate crimes, and offensive infighting. It was not uncommon for users to comment with insults like “you little worthless sub-human scum”. Although positivity was officially endorsed, it was often those users who brought optimism into the forum that were viciously attacked by their peers. It is not surprising that some users recognized the dangers of the forum: “This site isn’t good for anyone’s mental health, let’s be real”. Incitement to Violence We find indicators in the data that identify the incel ideology as violent extremism, since the idea was widespread in the forum that the situation can only be improved by harming one of the out-groups (attractive men or women). This becomes evident in the frequent claims to abolish women’s rights (see section 4.2), but even more in utterances that can be interpreted as direct incitement to violence against women (“Disobedient wives should be beaten”). One particular topic that frequently comes up is rape, which is encouraged. Other cases even show incitement to kill women. Some users want to see all women dead (“I want them all to die”), writing minutely detailed instructions of how they should be raped or killed. Incitement also includes appeals to kill people in the course of the Beta Uprising using lengthy descriptions, for example titled “How a crazy school shooter is made, and how women play a part in this whole”, that explain how the perceived discrimination that incels experience leads to becoming a mass shooter. Some users want to exact revenge on society (“I’m driven by hate, any action that can lead to the slightest bit of revenge upon society is worth the effort”). The users do not necessarily agree on what revenge should be like. While some would derive satisfaction from systematically blackpilling the world, i.e., spreading their negative world view, others actively incite violence towards women, or even call for human extinction in general. One user states: “They don’t give a shit about us. Nobody does. That’s why I have no problem if any of us starts killing as many people as possible. The more young women, Chads, Chadlites, normies, cucks, and Boomers who get slaughtered, the better”. We find very concrete fantasies about how people could be injured or killed, for example by beheading, poisoning, stabbing, “spray[ing] that class with bullets”, renting a van to run over people, or driving a fire truck into the Coachella music and arts festival. Likewise, discussion threads show considerable glorification of incel-related crime: “Alek Minassian. Spread that name, speak of his sacrifice for our cause, worship him for he gave his life for our future”. Several users have Rodger’s or Minassian’s face as their profile picture and choose usernames that refer to hate, rage, rape, murder, and so on. Some user profiles contain explicit depictions of violence, for example GIFs (animated images) that show women being hit in the face, shot in the head, or stabbed with a knife. Threatening The dataset also contains a number of threats, in varying stages of concreteness, with users considering to follow in the footsteps of previous attackers such as Rodger and Minassian (e.g. “I’m not even exaggerating, I’m probably gonna snap one day” or “Can’t wait until the next physical blackpill movement. I will join in”). We found five such messages (~0.25%) in the subsample of a 100 discussion threads. By extrapolation, this means that there might be a 150+ more in our entire dataset that should be examined. Of course, this is a small proportion in comparison to the size of the data, and it is doubtful whether these threats will be realized. On the other hand, it is also dangerous to trivialize utterances containing threats, which is one reason why the /r/incels community was banned from Reddit in 2017. The forum’s users themselves agree that incel attacks will become more common in the future. We must note that there is a visible faction among the users that objects to violence, in particular to mass killings (“I don’t condone violence. Especially mindless violence against innocent people”). They see a dangerous trend in the incel movement: “The spreading of the blackpill would no doubt increase shooting (and suicides) because some young men lose control”. According to these users, part of the blame for future attacks lies with the media: how the “media makes these killers into overnight celebrities” could increase the motivation for such killings. However, group dynamics in the forum lead to toxic discourse that cannot always be taken at face value because of echo chamber effects (Colleoni, Rozza, and Arvidsson 2014). These identity-forming mechanisms, described earlier in this section, have the downside that discussions motivate users to write content that attracts attention, i.e., producing comments that are more extreme than previous comments in a thread. For example, one thread on how to rape a woman shows that users try to come up with more “creative” methods of accomplishing this. It is unclear whether such extreme views about killing women are fantasies or warning signs. What is clear is that the existence of these forums contributes to cementing a negative worldview. As Waltman and Haas (2011, 63) state, “[t]he Internet has become the lifeblood of the organized hate movement”. One user answers the question if he hates all women with: “efore incels.me: no. After incels.me: YES YES YES”. As Langman’s study on school shooters (including Eliot Rodger) shows, “[r]ampage attacks are not caused simply by being short, psychopathic, psychotic, bicultural, failing in school, or being a ‘kissless virgin’” (Langman 2016, 7), but are committed by individuals who fail in all major domains of adulthood. We can conclude that, on the one hand, the image of the incel community as a homogeneous, highly aggressive group does not correspond to reality, since, unlike other hate communities, incels are not united by a common goal, but by a feature that its individuals share, i.e., not being able to engage in social and/or sexual contacts. On the other hand, it has also become obvious that there is a number of forum members (albeit small) that display an increased risk of committing attacks in the future. This motivates a case study to attempt to automatically detect early warning signs of misogynistic hate speech. Automatic Detection One way to mitigate the surge of online hate speech is to deploy systems that assist with the identification of objectionable content. Machine Learning (ML) is a field of AI that uses statistical techniques to train systems that “learn by example” to make automatic predictions. For example, given a 1,000 junk emails and a 1,000 real emails, a ML algorithm will infer that words such as viagra and lottery occur more often in junk emails. Such cues can then be used to classify unknown emails as junk or real. We used state-of-the-art Deep Learning techniques to train a Multichannel Convolutional Neural Network (CNN; Kim 2014) in Keras (Chollet 2015) with 50,000 Incels.me messages and 50,000 neutral texts composed of 40,000 paragraphs from random English Wikipedia articles and 10,000 random English tweets. The statistical accuracy of the model is 95.1%, which makes applications for monitoring online hatred of women viable and useful. We also experimented with the Perceptron algorithm (see Collins 2002), which does not require GPU hardware during the training sequence. We used character unigrams (u, g, l, y), bigrams (ug, gl, ly), trigrams (ugl, gly), and word unigrams (ugly) and bigrams (ugly guy) to encode both lexical (e.g., word endings) and contextual features (e.g., word combinations). Using 10- fold cross-validation (i.e., 10 tests on 10 different samples), the statistical accuracy (F1-score) is 92.5%, which is a competitive tradeoff between cost and effectiveness. 7 Conclusion In this paper, we demonstrated how the key to identifying features of an online subculture is examining their language use, both quantitatively and qualitatively. We have shown that the forum is interspersed with subcultural language and adolescent slang (Research Question 1). Words used focus on physical appearance, sexuality, and gender. A considerable proportion of the discourse classifies as hate speech, with the forum brimming with misogynist, anti-feminist, and homophobic utterances. Using a combination of methods, we outlined an alliance of necessity of isolated young men with a highly negative mindset and a pronounced inclination towards misogynistic spite (Research Question 2). Our findings correspond in part to how incels have been depicted by the media, but the analysis also reveals that the group is more heterogeneous than assumed. Our dataset displays a mixed picture of how dangerous or violent the members of the community may be, which is important to assess in regard to potential future attacks (Research Question 3). On the one hand, the forum is replete with incitement and explicitly violent fantasies. On the other, part of these may be no more than verbal tactics of self-enhancement in an online echo chamber. How do we deal with involuntary celibates? The removal from Reddit demonstrated that a community does not simply disappear, but will find new ways to spread and share hatred, for example, on Incels.me. Likely, the community discussing on Incels.me will reappear elsewhere after its removal. Close monitoring of such platforms in terms of violent extremism should be considered, with the possibility to act upon explicit threats. Automatic techniques to detect hateful speech can facilitate this process, in our case with up to 95% accuracy. This paper has aimed at providing an overview of the community and its vernacular and introduce different tools suitable for the analysis. Each of the findings would lend itself to a more detailed analysis in future study – preferably with a larger corpus. In addition, it would be interesting to monitor how the incel vernacular changes over the next few years, and how stable the picture is that the community conveys both of the in-group and the out-group(s). Acknowledgments The research was conducted during Google Summer of Code 2018 (GSoC). The algorithms for linguistic analysis and the ML models were developed by GSoC students Maja Gwóźdź, Rudresh Panchal and Alexander Rossa, and are available on request. A Textgain API license key was donated by Textgain to conduct the automatic profiling experiments. Bibliography Argamon, Shlomo, Moshe Koppel, James W. Pennebaker, and Jonathan Schler. 2009. “Automatically profiling the author of an anonymous text.” Communications of the ACM 52(2):119-23. doi:10.1145/1461928.1461959. Berger, J. M. 2017: “Extremist Construction of Identity: How Escalating Demands for Legitimacy Shape and Define In-Group and Out-Group Dynamics.” ICCT Research Paper. https://icct.nl/wpcontent/uploads/2...ist-Construction-of-Identity-April-2017-2.pdf. Blommaert, Jan. 2017. “Online-offline modes of identity and community: Elliot Rodger’s twisted world of masculine victimhood”. Tilburg Papers in Culture Studies 200. https://biblio.ugent.be/publication/8551305/file/8551306.pdf. Bou-Franch, Patricia, and Pilar Garcés-Conejos Blitvich. 2016. “Gender ideology and social identity processes in online language aggression against women”. In Exploring Language Aggression against Women, edited by Patricia Bou-Franch, 59-81. Amsterdam/Philadelphia: John Benjamins. Brennan, Michael, Zadia Afroz, and Rachel Greenstadt. 2012. “Adversarial Stylometry: Circumventing Authorship Recognition to Preserve Privacy and Anonymity.” ACM Transactions on Information and System Security 15(3): Art. 12. doi:10.1145/2382448.2382450. Brown, Alexander. 2017. “What is hate speech? Part 1: The myth of hate.” Law and Philosophy 36:419-468. doi:10.1007/s10982-017-9297-1. Brown, Rachel. 2016. Defusing Hate: A Strategic Communication Guide to Counteract Dangerous Speech. https://www.ushmm.org/m/pdfs/20160229-Defusing-Hate-Guide.pdf. Brown, Rupert, and Hanna Zagefka. 2005. “Ingroup Affiliations and Prejudice.” In On the Nature of Prejudice, edited by John F. Dovidio, Peter Glick, and Laurie A. Rudman, 54-70. Maldon, MA, etc.: Blackwell. Chollet, François. 2015. Keras: Deep learning library for Theano and Tensorflow. https://keras.io. Colleoni, Elanor, Alessandro Rozza, and Adam Arvidsson. 2014. “Echo chamber or public sphere? Predicting political orientation and measuring political homophily in Twitter using big data.” Journal of Communication 64(2):317-32. doi:https://doi.org/10.1111/jcom.12084. Collins, Michael. 2002. “Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms.” Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing 10:1-8. doi:10.3115/1118693.1118694. De Smedt, Tom, and Walter Daelemans. 2012. “Pattern for Python.” Journal of Machine Learning Research 13:2063-2067. http://www.jmlr.org/papers/volume13/desmedt12a/desmedt12a.pdf. De Smedt, Tom, Guy De Pauw, and Pieter Van Ostaeyen. 2018. “Automatic detection of online Jihadist hate speech.” CLiPS Technical Report Series 7: 1-30. https://www.uantwerpen.be/images/uantwerpen/container2712/files/hate-speech-detection.pdf. Dholakia, Utpal M., and Richard P. Bagozzi. 2004. “Motivational antecedents, constituents and consequents of virtual community identity.” In Virtual and Collaborative Teams: Process, Technologies, and Practice, edited by Susan H. Godar and Sharmila P. Ferris, 252-67. Hershey/London: Idea Group Publishing. doi:10.4018/978-1-59140-204-6.ch014. DiMauro, Dina. 2008. “Reluctant virginity: the relationship between sexual status and self-esteem.” Theses and Dissertations 717. http://rdw.rowan.edu/etd/717. Donnelly, Denise, Elisabeth Burgess, Sally Anderson, Regina Davis, and Joy Dillard. 2001. “Involuntary Celibacy: A life course analysis.” The Journal of Sex Research 38(2):159-69. doi:10.1080/00224490109552083. Douglas, Karen M. 2009. “Psychology, discrimination and hate groups online.” In The Oxford Handbook of Internet Psychology, edited by Adam N. Joinson, Katelyn Y. A. McKenna, Tom Postmes, and Ulf-Dietrich Reips, 155-63. Oxford/New York: Oxford University Press. Gilmore, David D. 2001. Misogyny: The Male Malady. Philadelphia: University of Pennsylvania Press. Ging, Debbie. 2017. “Alphas, Betas, and Incels: theorizing the masculinities of the Manosphere.” Men and Masculinities. doi:10.1177/1097184X17706401. Haines, Russel, Jill Hough, Lan Cao, and Douglas Haines. 2014. “Anonymity in computer-mediated communication: More contrarian ideas with less influence.” Group Decision and Negotiation 23(4):765-86. doi:10.1007/s10726-012-9318-2. Hamm, Mark S., and Ramón F. J. Spaij. 2017. The Age of Lone Wolf Terrorism. New York/Chichester: Columbia University Press. Haythornthwaite, Caroline. 2009. “Social networks and online community.” In The Oxford Handbook of Internet Psychology, edited by Adam N. Joinson, Katelyn Y. A. McKenna, Tom Postmes, and Ulf-Dietrich Reips, 121-37. Oxford/New York: Oxford University Press. Jane, Emma A. 2017. “Systemic misogyny exposed: Translating Rapeglish from the Manosphere with a Random Rape Threat Generator.” International Journal of Cultural Studies 21(6):661-80. doi:10.1177/1367877917734042. Jane, Emma A. 2014a: “Back to the kitchen, cunt”: speaking the unspeakable about online misogyny. Continuum: Journal of Media and Cultural Studies 28(4):558-70. doi:10.1080/10304312.2014.924479. Jane, Emma A. 2014b. “‘You’re a Ugly, Whorish, Slut’. Understanding E-bile.” Feminist Media Studies 14(4):431-46. doi:10.1080/14680777.2012.741073. Kämper, Andreas. 2011. [r]echte Kerle. Zur Kumpanei der MännerRECHTSbewegung. Münster: UNRAST. Kim, Yoon. 2014. “Convolutional neural networks for sentence classification.” arXiv:1408.5882, https://arxiv.org/pdf/1408.5882.pdf. Kleinke, Sonja, and Birte Bös. 2015. “Intergroup rudeness and the metapragmatics of its negotiation in online discussion fora.” Pragmatics 25(1):47-71. doi:https://doi.org/10.1075/prag.25.1.03kle. Langman, Peter. 2016. “Elliot Rodger: An Analysis.” https://schoolshooters.info/sites/default/files/rodger_analysis_2.0.pdf. Larkin, Ralph W. 2018. “Learning to be a rampage shooter. The case of Elliot Rodger.” In The Wiley Handbook on Violence in Education: Forms, Factors, and Preventions, edited by Harvey Shapiro, 69-84. Hoboken, NJ: Wiley & Sons. doi:https://doi.org/10.1002/9781118966709.ch4. Liu, Bing. 2012. Sentiment Analysis and Opinion Mining. San Rafael, CA: Morgan & Claypool. https://doi.org/10.2200/S00416ED1V01Y201204HLT016. Mantilla, Karla, ed. 2015. Gendertrolling: How Misogyny Went Viral. Santa Barbara, CA: Praeger Press. Mantilla, Karla. 2013. “Gendertrolling: Misogyny Adapts to New Media.” Feminist Studies 39(2):563-70. https://www.jstor.org/stable/23719068. Marx, Konstanze. 2018. “Cybermobbing aus sprachwissenschaftlicher Perspektive.” Sprachreport 1/18:1-9. https://ids-pub.bsz-bw.de/frontdoor/index/index/docId/7209. Musolff, Andreas. 2012. “The study of metaphor as part of critical discourse analysis.” Critical Discourse Studies 9(3):301-10. doi:http://dx.doi.org/10.1080/17405904.2012.688300. Neuman, Yair, Dan Assaf, Yochai Cohen, and James L. Knoll. 2015. “Profiling school shooters: Automatic textbased analysis.” Frontiers in Psychiatry 6: Art. 86. doi:10.3389/fpsyt.2015.00086. Newman, Matthew L., Carla J. Groom, Lori D. Handelman, and James W. Pennebaker. 2008. “Gender differences in language use: An analysis of 14,000 text samples.” Discourse Processes 45:211-36. doi:10.1080/01638530802073712. Nockleby, John T. 2000. “Hate speech.” In Encyclopedia of the American Constitution. Vol. 3, 2nd ed., edited by Leonard W. Levy & Kenneth L. Karst, 1277-79. Detroit: Macmillan Reference US. Pennebaker, James W. 2011. The Secret Life of Pronouns. What Our Words Say About Us. New York: Bloomsbury. Pennebaker, James W., Martha E. Francis, and Roger J. Booth. 2001. Linguistic Inquiry and Word Count: LIWC 2001. Mahwah, NJ: Lawrence Erlbaum Associates. Plummer, Ken. 2005. “Male sexualities.” In Handbook of Studies on Men & Masculinities, edited by Michael S. Kimmel, Jeff Hearn, and R. W. Connell, 178-95. Thousand Oaks/London/New Delhi: Sage. Rego, Richard. 2018. “Changing Forms and Platforms of Misogyny. Sexual Harassment of Women Journalists on Twitter.” Media Watch 9(3):472-85. doi:10.15655/mw/2018/v9i3/49480. Salter, Anastasia, and Bridget Blodgett. 2012. “Hypermasculinity & Dickwolves: The Contentious Role of Women in the New Gaming Public.” Journal of Broadcasting & Electronic Media 56(3):401-16. doi:10.1080/08838151.2012.705199. Schmidt, Anna, and Michael Wiegand. 2017. “A survey on hate speech detection using natural language processing.” Proceedings of the Fifth Workshop on Natural Language Processing for Social Media:1-10. doi:10.18653/v1/W17-1101. Suler, John. 2004. “The Online Disinhibition Effect.” CyberPsychology & Behavior 7(3):321-6. doi:10.1089/1094931041291295. Turnage, Anna K. 2007. “Email Flaming Behaviors and Organizational Conflict.” Computer-Mediated Communication 13(1):43-59. doi:https://doi.org/10.1111/j.1083-6101.2007.00385.x. Van Valkenburgh, Shawn P. 2018. “Digesting the Red Pill: Masculinity and Neoliberalism in the Manosphere.” Men and Masculinities. doi:10.1177/1097184X18816118. Waltman, Michael, and John Haas. 2011. The Communication of Hate. New York: Peter Lang. Xu, Wei, Xin Liu, and Yihong Gong. 2003. “Document clustering based on non-negative matrix factorization.” Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval:267-73. doi:10.1145/860435.860485. Zimmerman, Shannon, Luisa Ryan, and David Duriesmith. 2018. “Recognizing the Violent Extremist Ideology of ‘Incels’.” Women In International Security POLICYbrief 09/2018:1-5. http://www.academia.edu/37532571/Recognizing_the_Violent_Extremist_Ideology_of_Incels.
This. I've never seen anything more useless than this academic gibberishjfl at studies like this even existing. No one bothers to study as much how autistic ugly males are bullied online, because nobody cares about that and research grants in sociological departments are probably tailored to studies examining any possible 'threat' to women.