I spent the past three years embedded with internet trolls and propagandists in order to write a new nonfiction book, ANTISOCIAL, about how the internet is breaking our society. I also spent a lot of time reporting from Reddit's HQ in San Francisc...

What made you decide that you wanted to write a book about online trolls?

Well it certainly wasn't my deep and abiding desire to spend three years hanging out in the living rooms and Airbnbs and hotel rooms of misogynists and Islamophobes and propagandists :) I wanted to find out what the internet was doing to our brains, to our informational ecosystem, and to our society. I didn't want to just think about it abstractly and have an opinion or an argument about it -- I wanted to live and breathe it, to see in vivid and immersive detail how it actually worked. I think it's hard to gain real understanding about a phenomenon without first truly seeing and understanding what it looks like up close.

How did you prepare? And what if anything shocked you the most ?

What shocked me the most was how easy it is to be a highly successful propagandist. I spent a lot of time with some people whose names you may know -- Mike Cernovich, Lucian Wintrich, Milo Yiannopoulos, Richard Spencer -- and some people whose names you surely don't know. In every case, I was shocked by how with just a bit of skill, some practice, and essentially no investment of resources, they could take whatever fringe meme or talking point they wanted and propel it into the middle of the national discourse (get it trending on Twitter, on the front page of Drudge, on Fox News, even on CNN). I watched this happen again and again, in front of my eyes, in a matter of minutes.

Have you taken any of their tactics on board or did involve a lot of shady stuff ?

Yeah I do not think it would be wise for me to copy their tactics. I do think however that the tactics are worth learning so that they can be understood, and in some cases countered

In what primary way is the internet breaking our society?

This stuff is complicated, for sure, so I don't want to be too reductive. I could point to some very tangible outward manifestations—Trump, Brexit, Duterte, the Rohingya massacres, and on and on—and argue that all of those were spurred, either partially or directly, by the internet. But I actually think that the primary problem is deeper than that. In the book, I talk a lot about what the pragmatist philosopher Richard Rorty called "vocabularies"—the deep moral and political assumptions in which a society is embedded. A functioning society should have a functioning vocabulary, one in which people can discern the truth and be mutually intelligible to one another. Our vocabulary is deeply broken, and I think the internet, particularly the social internet, is one of a few culprits.

Why do you feel our society is any more broken than it used to be?

Good question -- I definitely don't want to imply that our society has ever been perfect. I am not advocating a return to some golden age. I'm referring to new and specific kinds of brokenness, not so much a quantitative comparison between past and future.

Have you ever come across anything resembling 'guilt' by any of the social-media entrepreneaurs for things like lack of foresight, or the unforeseen consequences of what they've created?

Definitely. I think they're all feeling some form of guilt, and I think they're all processing it in their own ways. It's recently become fashionable to think of social media executives as mere robber barons, making decisions only out of greed and the profit motive. There's some truth to that -- they are businesspeople, after all -- but I also think they're former idealists who have, in a sense, been betrayed by the naivete of their own ideals. In a way, this makes the problem harder to solve, because idealists are in many ways not purely rational actors.

I'm very interested in the idea of techno-utopianism and the conceptual hold it has on a large segment of the most influential people in society. Is it possible (or desirable) to convince tech leaders that the solution to technology driven problems isn't alway technological solutions or get them to consider that society needs time to evaluate and integrate the radical disruptions caused by technological advance?

I definitely think it's desirable! The tech industry still suffers from the "when you have a hammer everything looks like a nail" problem. And, as I outline in some detail in my book, tech has come to dominate everything -- media, transportation, politics, you name it. So it's a big problem. I do think, though, that tech leaders are conscientious enough (or, some would say, vain enough) to be coaxed into changing their ideology. The key, or one of the keys, is to convince them that their legacy, their standing in society, depends on it. Above all -- even above money -- many tech leaders want to feel smart and important and universally revered. The phrase I use for this in the book: they want to feel like Big Swinging Brains.

I don't think that we should be relying on millionaire and billionaire tech leaders to change their ideology enough that they won't be hostile to necessary societal change that may harm their fortunes. That's some pretty wishful thinking.

I definitely don't feel confident that it will happen. just saying it's possible! There is definitely a rule for other carrots and sticks (government regulation, etc). I don't think any single solution will work on its own

It seems like things go viral naturally--cute cat meme rises to the top of the twitter feed. Huzzah! But how much manipulation is happening behind the scenes? Wondering how the trolls you met with became particularly good at drawing people in, and what that says about the social media platforms they're using to make it happen?

Yes, it all seems accidental when you look at it from the outside -- like the popular memes that spring to the top of your feed are just the cream naturally rising to the top. But there's always a person behind the curtain, whether it's a human actively manipulating a feed or whether it's simply an algorithm doing it (that algorithm, after all, was designed by humans). This isn't necessarily bad news -- it just means that we need to be honest about the fact that there are humans in charge of this stuff, and we need to reckon with what that means. Humans are fallible creatures. We can't expect the massive experiment of social media to be perfect either. We just have to stop wishing and waiting for it to perfect itself and actually demand that the people in charge of the algorithms make them better -- not just better from a profit-making perspective, but better from a civic and prosocial and moral perspective.

What would you say is the best way to handle internet trolls?

I don't think there's a one-size-fits-all solution. In the book I talk about how trolls set an ingenious trap: to react to them in any way is to give them attention, which is what they want, but to ignore them is to risk seeming complicit, as if you have no problem with what they're saying or doing. I guess one good rule of thumb is to try to use what Daniel Kahneman would call the slow brain (as opposed to the fast brain) when reacting to a meme, a talking point, an internet Nazi, etc. It's sometimes a good idea to react, but if you're reacting out of pure shock, or pure anger, then you might want to take a breath and see if your frontal cortex agrees with your lizard brain that the meme you're about to respond to is actually worth responding to...

What will social media look like in 20 years? Who will win? Will society ever be able to recover?

I don't want to spoil the last scene in the book, but you can look to the great social experiment of r/Place, in 2017, to suggest one possible answer to that ;)

Is there hope?

There is hope! I think there's a huge and important difference between hope and faith, or between blind optimism and what my friend Bina Venkataraman calls "engaged optimism." I don't believe that we can just sit back and wait for the arc of history to bend itself toward justice. I think the arc of history bends whichever way people bend it. And people = all of us. Which is bad news but also good news!

Are the people who are trolling/misogynist/etc. actually complete assholes? Like, does their IRL persona reflect their online beliefs at all? I look at something like 4Chan where everyone is constantly making comments about Jews, Hitler being right, etc. and find it hard to believe these people are all like that.

It seems more plausible that they are all sharing some big stupid joke under the cover of fascism/sexism/etc and just want to see how far it can go regardless of consequences.

There's a huge range, from people who are purely nihilistic/opportunistic to people who are true believers. I spent time with both. Some people seem to have no internal life at all, no true beliefs, just a bottomless need for attention. Other people are really truly red-pilled into believing, e.g., that Jews are trying to destroy the white race. You can't really know which kind you're dealing with unless you put in the time, ideally in person...luckily you can skip that step and just read the book to find out :)

How do you define a threat?

There's a legal definition of true threats (https://www.mtsu.edu/first-amendment/article/1025/true-threats). That's a pretty narrow definition, as I think it should be -- when thinking about when the government should be allowed to restrict political speech, I think the bar should be very high. The bar is (and, imho, should be) considerably lower when it comes to how much private platforms can or should restrict speech.

Also, for anyone interested in the questions this raises about free speech, the history of the legality/illegality of hate speech, etc., listen to On the Media this Friday for an episode co-hosted by me about these very topics! https://www.wnycstudios.org/podcasts/otm

Just how long were you on /pol/, /r9k/, and their cripplechan variants?

I visited all the terrible places online, but I was mostly interested in meeting people in real life. The channers are what they are, and they're important to understand, but they're mostly anonymous. I gave preference to people who were courageous enough, or foolish enough, to let me meet them IRL and observe as they lived their daily lives.

Can you just make a meme version of the book for all of us with no attention span?

https://media.giphy.com/media/h5NXof7XfEYHm/giphy.gif

[deleted]

Thanks! I definitely understand the brain fog feeling -- I think we've all experienced that. Actually a lot of the most interesting stuff happened while I was embedded at the headquarters of Reddit itself! A lot of the biggest social media companies, including Reddit, were founded by people who were very idealistic (even utopian, to quote the subtitle) about what would happen if they redistributed the power of information as broadly and freely as possible. Those idealistic expectations didn't really come to pass, to say the least. So seeing them cope with that in real time was fascinating. After Charlottesville, I was in the room while Reddit admins banned over 100 subreddits -- Nazi subs, bestiality subs -- to try to make the atmosphere on Reddit less toxic. It wasn't perfect, it was messy and subjective, but I think you could pretty easily argue that it was better than nothing...and yet I had a lot of conflicting reactions while sitting in that room.

Not to be weird, but what does bestiality subreddits have to do with neo-nazis? It seems weird that those are the examples, lol.

I'm guessing that when the admins were seeing a concrete example unfolding in front of them of how online forums could cause substantial damage, they were moved to crack down on other subreddits that were likely harmful.

Fair enough, that makes sense.

They were changing their policy against violence -- expanding it to include more speech that could be considered harmful -- and in doing so they added a clause about subs that encouraged harm against animals. The fact that they never had a rule about that before didn't mean that Reddit was pro-bestiality prior to 2017. Just that there are a lot of oversights when you're invented speech codes from scratch!

Should we just ban social media? How do we prevent it from being a propoganda machine?

No we can't ban social media any more than we can ban cars or cities or political parties. We just have to work to make them better, which is a more difficult task.

What are some proven effective policy/structural changes that could be passed by governments to minimize the political impact of these antisocial hate groups? What about policies by social media companies themselves?

I am very wary of using government to restrict speech, including hate speech. But there's a huge amount that the companies could do. However, it's not as easy as deciding whether to kick off this or that troll, or to censure this or that group. The platforms would have to be redesigned in a much more fundamental way.

How do you keep your sanity?

good family, good music, good bourbon, 8 hours of sleep a night :)

This is a great IAMA subject and kudos for you to doing this research for the last 3 years. I dip my toe into elements of it and its mind breaking and scary. Quick Question: Is it a case that a lot of the individual "hate" ideas were just silenced because there was no way to connect individuals, because those ideas (like killing all drug dealers, banning a whole religion, grooming) weren't possible with the internet and the internet has just given these people the network to support and then validate their ideas?

Yes, a lot of it has to do with the speed and scope of the internet, the ability for large groups to form quickly with relatively little friction. But it also has to do with the mechanics of the social internet -- specifically, the way that engagement is a product of what scientists call "activating emotions." Thus, social feeds are not flat reflections of reality but are warped in specific and predictable ways.

Hi Andrew. First of all, i admire your perseverance into the unbearable world of non-sense. I used to argue with these individuals until i realize i was wasting my time. There are so many reasons to not waste our time with these guys.

My question is, now with the problem of troll farms, how do you differentiate a simple troll from a real fanatic ?

You have to meet them in person and spend a lot of time with them. That doesn't get you all the way there, but it's a start