We're the journalists behind the "Reddit for Sale Story". We managed to get a story to the #1 spot on /r/videos for a very small amount of money. AMA

Have you been able to replicate it consistently? I can probably get a post to the top of /r/videos if it's interesting content that reddit would enjoy.

It's been know for a long time that you can give a post/comment a significant boost with only a few early fake upvotes. So long as the content is interesting to reddit as a whole, the early upvotes can start a chain reaction as they place it higher in users' feeds.

It's the same principle that let the_donald moderators send half their subreddit's content to the front page by stickying new posts - reddit's algorithm sees lots of early votes on something and assumes it's amazing. So you can definitely game that.

That said, the account you did this stunt on was banned, and your video was probably removed fairly quickly. So I'd argue that's an example of the system working, and this isn't some major accomplishment.

The video we did this on is here. It was using an account we purchased from a contact we found in our investigation.

We then used a vote system to provide 1000 upvotes to the link, and that was enough to start a kind of chain reaction / feedback loop. People started to upvote the link massively because (we assume) the link was already on the front page.

The perceived popularity of the link made it super popular. For the record I think we got it pretty close to the #1 spot just with the paid for votes. It wasn't 'one or two' it was a thousand.

/u/tweninger could give you great information on redditor behaviour, what's influential and what's not.

UPDATE:

We deliberated for quite a few weeks about what kind of video to use fake votes with. We wanted something that wasn't going to be obviously terrible and arouse suspicion, but something that wouldn't have gone viral on its own. In the end we chose Narcos. The trailer had already been out for a month or so, and not done particularly well. We gave it a headline to appear like an Ad, and then gave it the up vote boost. It worked way more effectively than we could ever have believed.

As for getting some content that would NEVER get upvoted, our pro-Brexit story on /r/UnitedKingdom proves that quite well. It was removed by the mods in the end, but that's because we inadvertently tipped off the commenters that the character was Norwegian. This sent a few curious people on a quick google search to find the professor didn't exist at all, neither did the University, so we shot ourselves in the foot and the post was removed. Hats off to the /r/unitedkingdom mods, but had we not tipped them off, the story would likely have remained.

That said, the account you did this stunt on was banned, and your video was probably removed fairly quickly. So I'd argue that's an example of the system working, and this isn't some major accomplishment.

ummmmm no, not before it reached number one and none of the accounts used to perpetrate the up/down voting were banned. So anyone inclined to do this sort of thing can just make ONE new account to continue. so i would argue that you are talking out of your ass.

I expect that the accounts associated with our investigation are now banned. Reddit definitely banned the account we used to submit to /r/unitedkingdom because it was an account of someone we actually know. So the account is gone now.

They only did that when we contacted them about the story on Monday and gave them a heads up that we were publishing. So it's entirely plausible that you could buy multiple accounts and commit multiple vote manipulations and not get caught. And if you do get caught? Buy a new account, it's about $20 for an account to old enough and with enough karma to post in most of the subreddits.

Was this AMA bought and paid for?

If it were bought and paid for, it wouldn't have four points.

True. Follow up question, is there a way reddit can stop paid for posts?

I'm not sure how it can be done. To be fair to reddit, I think they've made it very difficult to spam the website. Our investigation showed that hackers/spammers really struggled to get it to work. However, because it's so difficult to hijack, the trust the users have in the website is very high. That trust is the honey in the jar to special interests.

Ultimately, with enough money and planning it seems it's possible to get passed the spam filters and hijack yours and my trust in the links.

This is a big issue.

I think more data provided to redditors might help. So we could see in real time where and when votes are coming in. It's very important to get this right, so we all need to figure out a way to fix it.

What are the limits to something like this? Could you get a pro-Trump headline to the top of /r/politics? Could you get an anti-OSS headline to the top of /r/linux?

I'm well aware that it's not very hard to get content that Reddit likes on the front page of Reddit using fake votes to put your well liked content ahead of other people's well liked content. I'm far less convinced that you can put pretty much anything you want on the top of Reddit for a few bucks. I'm not convinced by the Brexit story because /r/unitedkingdom has a sizable pro-Brexit population, and the headline could easily be read as anti-Brexit sarcasm.

The system was incredible effective. Could you get to #1 everywhere? Probably not, but you could have a very sizeable impact for not much money at all.

Do you have a solution where this can be prevented in the future?

Stick to small subs, downvote content that feels spammy or like an advertisement.

Do you have 500 down votes ready? Because the vote trolls have 500 upvotes.

Downvoting spam is certainly helpful, but the idea that we should stick to the smaller subs (I think) sells the website short.

I hear that a lot when I talk with people about it 'The defaults aren't where it's at any more.' By conceding the default subs we are basically giving up on the website, and is that what we want? Do we want the reddit outsiders see to be dysfunctional? Spreading half truths etc? Are we happy to just retreat and make the reddit experience a very personal one in which small niche communities work?

Reddit used to have a very vocal 'central' sub called reddit.com which unified all the other defaults together, but it's gone now. It would be great to bring something like /r/reddit.com back specifically to talk about issues like this. There's /r/theoryofreddit but it's not a default, and subs like that always end up being fragmented and small.

Not saying it's a good solution, it's the only one I know.

Fair. Its a conversation we all need to have. Thanks for the comment. :)

We did this because we love reddit, and we think it's way more important than people realise. We think it's way more influential than people realise. The danger with a website like reddit is that we all really believe the links are prominently placed because the community likes the links.

If a company can sidestep that authentication process and cheat their way to the top - commercially or politically - that's a very serious issue that needs solving.

How we do that is to be seen. Perhaps there should be much more transparency with data. Perhaps we need paper trails to see who voted on what, and from where those votes came. Some way for the community to keep a better watchful eye over this.

At the moment we're totally blind to how much this is going on, and that should be worrying to anyone who knows and loves reddit.com

Seems like this relies on the upvote/downvote, so what if it was removed?

Yeah that might work. I like your thinking. :)

I feel like you're focusing explicitly on corporations/marketing firms - but completely ignoring the role of obsessive ideologues. It's fairly easy to create multiple accounts, you can use proxies/vpn etc.. if necessary - you don't NEED to spend hundreds of dollars to do this. I'd be more worried about ideologues obsessed with pushing an agenda on reddit, rather than marketing firms. Why did you choose to limit the potential for manipulation only from paid agencies? This will completely skew the discussion on reddit so that anything remotely pro 'establishment' (such as something that might be complementary to anything connected to government or a business) will be likely met with massive amounts of unwarranted suspicion and accusations of shilling, while sensationalist 'anti-establishment' posts won't meet any suspicion, when they could just have easily been manipulated from places like 4chan?

I take your point, but this has started a conversation and hopefully the conversations will continue.

I actually believe the ideologue problem to be the most concerning, and it's something I'd like to explore more.

There was no agenda in focusing in a particular area, we just wanted to show that this is possible, and therefore likely to be going on.

I take your point

Indeed you do.

Dumb comment aside, the most impressive feat was buying your way to the top of r/videos imo. Do you think that it'll be harder to gain the system now that we are seeing bigger vote counts hitting the front page? Your recent video got 20k organically, so I'm guessing it would be more expensive and difficult to keep fake large numbers away from reddit's spam catching mechanisms.

The hope is that this problem can be tackled yes.

We haven't used the system on this video at all. Those votes (unless someone else has hijacked it) are all organic.

Did you spot the China Brexit article getting any attention outside of Reddit, in the mainstream news or any political blogs for example?

It didn't stay there long enough, and nor would it have made an impact. It was one story after all. The point was that we managed to get it on the front page entirely from bought upvotes.

Firstly, I would like to thank you guys for bringing this to attention again, Reddit has vastly improved its spam recognition filters and the issue of fake upvotes or stories wasn't talked about much till recently, the fact is this is still a major issue and new users should be aware of this.

I would like to ask what is your opinion about abandoned accounts that are hacked then sold, should Reddit have a system to automatically disable inactive accounts?

There are people who accept money to promote certain brands/companies (such as the Burger King post in your article), what can we do about that?

Also, bands brands seem to be gaming the Reddit and Instagram space right now, by paying users to promote them, how do we discourage this?

Reddit's system is good but it needs improvement and further discussion.

Thanks for the question.

The accounts we bought often seemed to be abandoned. They'd have some Karma, some history, then they'd end up in the hands of a middleman who'll sell them for not much money. It would be trivial to control 10 or 20 seasoned accounts.

I don't know what the solution to that problem is. Perhaps better verification? Whatever the solution, I think its important that it's solved.

As for people accepting money to promote things, reddit SHOULD protect us from that with the vote system. But we've shown that the vote system can also be bought and paid for. Perhaps users need to be more aware that this is possible, and that it's almost definitely going on daily. We wanted to prove that in our investigation, but to do so would have meant 'catching' someone red handed and that's very difficult.

As for bands? I am really not aware of this issue so I couldn't comment.

/r/videos mod here.

Posting a Narcos trailer and getting front page isn't really surprising though. I don't think you are accounting for how many users will upvote a post without reading the comments or looking into it in any way. I mean look at the comment section, you got caught, but the proof showing it has way fewer upvotes than the score of the post even subtracting the votes you purchased.

That's why clickbait works. I'd be more curious about this using a relatively unknown or flat out unpopular video.

The things that do show weaknesses though are that you did it by getting through our spam filtering. But, it's not overly surprising, because again, it was a legitimate video. Vote rigging isn't something we can detect as mods.

I agree - the problem isn't something the mods can solve. It has to be some more information/data being provided to the users to see who/where votes are coming from.

In a previous comment we explained why we chose Narcos. We would be happy to try and demonstrate the system working (if it still does...) on a boring link. Not sure if would even be possible now.

Not sure if would even be possible now.

Soo... uhhh... Why should we care about this then if it's no longer possible to do?

Not sure != no longer possible.

Is the video on the front page now upvoted by your fake accounts?

No. But the fact that it's impossible for me to really prove that kind of proves the point right? I think the video has more than 20,000 points by now.

Do you feel paying for news that appears on the front page of Reddit is similar to organisations paying off smaller, supposedly "independent" news companies (i.e. Truthloader), and if so how do you think people can be made more aware of this and react?

Good barbed question.

The Truthloader project that you mention, is this a rebuke perhaps? We were always upfront about being a part of ITN, we told our audience. And we got complete editorial freedom, and it was a great project and I believe we did some great work. I wish it could have continued in many ways.

I think you're talking about transparency (along with a backhanded comment) and I'll say this: it's important.

How come this AMA is on a new throwaway account?

My account was shadowbanned when we told reddit about the story and asked for comment before publishing. I expect this account will be banned soon too? Or perhaps not? I don't know.

I did break the rules, but...kinda sad. My account is 8 years old!

Dang... sorry mate. What rules did you break and what do you think of the concept that all good ideas are exploited until they are no longer good ideas?

haha Yeah that's a good saying. Hopefully we can make it no longer a good idea to exploit reddit, so we can get our website back! Oh damn, I sound like Trump.

Make Reddit Great Again!

Thanks for your insightful documentary.

Is there anything you can recommend that we as the users ought to do, besides appealing to the admins?

And given that Reddit is a private company, does it really have a duty to ensure all its posts are legitimate or is just all about ensuring its continued future by remaining attractive to its userbase?

I'm sure that reddit want to ensure their users are happy. It's our job as users to ensure we know what's going on with the website we check in on every single day. We just need to be aware of it.

Perhaps there needs to be a conversation about the severity of this problem, and how much it impacts on things.

Would you say r/politics and news is being manipulated. R/the_donald is a clearly a fandom subbreddit and so is R/Hillary Clinton but should not r/politics and a r/news be non partisan?

I stopped reading /r/politics a long time ago unfortunately.

After reading your article it looks like you took enormous effort to achieve limited success.

In one case, you have a fake story being detected by the community and banned - in the other, you are using a piece of content that already has some relevancy to User interests.

It seems to me that this headline is a case of forcing a pre-desired conclusion despite the story the evidence shows.

Was there ever a consideration of publishing a story around not succeeding in this endeavor?

If not, what do you hope to accomplish by pushing this narrative?

Well we thought the investigation was a massive success. We could have gone further but we don't have limitless resources.

Would we have published had it not been a success? Probably not, because it's not a story really is it?

To us, it is surprising just how easy it is to sidestep the voting system.

We want more transparency about how votes are cast. We want people to be aware that this is entirely possible.

[deleted]

Heads up /u/MerryNexus - delete this

Cheers! :)

Were you guys basically just trying to prove what CTR/The_Donald have been doing here for the past year or more, aka fake upvoting to start a hive-mind up vote?

That was as very large part of the inspiration to do the investigation. I've been a redditor for 8 years, and I've noticed that the tone and voting behaviour has changed over my time here.

It's not just Correct The Record and The_Donald, though I wouldn't be surprised if it they were actively gaming reddit and other platforms - the subreddit /r/HailCorporate is pretty good at calling our corporate involvement too.

I think its really important that redditors are aware of, vigilant of, and working towards fixing this problem. It would be great to work with the admins on helping to address this issue also.

[deleted]

We didn't do this to 'promote' our content. We did it to highlight the very important point that hijacking your way to the top of a thread is relatively straight forward, AND its very impactful.

We need to be able to trust that the content on reddit is there for the right reasons, and perhaps that trust is a little misplaced at the moment.

What is your end goal for this project?

Point will produce more videos like this - we have one coming out next week about Instagram.

[deleted]

Point have been working on stories like this for more than a year. This is the first one they've released. There are more coming, and more planned.

Do you think /r/politics was manipulated or was it just an echo chamber changing opinion? Like going from Pro Bernie Anti Hillary --> Pro Hillary Anti Trump --> Anti Trump ?

Our investigation wasn't funded well enough to find out. Investigative journalism is a dying art, we're absolutely trying our best with the resources we have. We'd have loved to have done more.

Who exactly funded your investigation? It seemed like it was just you guys.

It was self funded yes.

Investigative journalism is a dying art

If you call posting two posts and paying for upvotes "investigative journalism" it sure is. Investigative journalism would be more in the line of actually finding out how this payment of accounts worked, and who was making money out of this.

We know how the payments of accounts work - people sell their accounts to middlemen for a couple of dollars. The middelmen then sell them.

As for 'who is making money' - we wanted to know too. We believe large companies are doing this, but catching them red handed is expensive. Instead we found a well functioning system being sold on a pay as you go model. It's not in the interest of the story to expose the tool. Anyone with a couple of hours to spare could figure it out.

How did you know about vote manipulation in r/Bitcoin?

Are you a regular to that sub?

I used to go in there quite a bit. I remember seeing something about vote manipulation in bitcoin and I followed up on the story.

Are there any good examples of people calling out your accounts getting shouted down by others who defend it?

/u/ShellOilNigeria

Have you guys ever thought of running a bot to data mine Reddit? I would thing is would be an easy monumental task to try and figure out how much is bought and sold on Reddit by understanding what users comment on what. Looks and smells like you guys are getting bashed by troll farms now...Somewhat negative thread and responses.

Its a great idea. I have no idea how to do that but if someone wanted to get that done I'd support it.

Nice video and I appreciate you guys raising awareness about the manipulation of Reddit, hopefully people can realize just how easy it is, especially with government agencies like GCHQ using programs like JTRG that you mentioned in the video.

My question is, with this "proof" what do you expect/want to happen in regards to online communities like Reddit?

Also, have you researched any other programs like JTRG? If not, I have a list that I comment about periodically when the topic arises that I would be happy to share.

P.S. - I made a guest appearance for calling you guys out in the video ;)

We researched JTRIG extensively yes. It's crazy. I think it's such a bizarre story that it didn't really find the audience it deserved. People assume that those tools must only be used overseas, but I'm not so sure.

When all you have is a hammer, every problem starts to look like a nail.

You win 1 internet point for calling us out in the thread.

1. Do you think this problem of easy vote manipulation can be solved?

2. Did you plan this whole thing just to get your video on the top of /r/all and rake in those juicy views?

3. Did you use reddit before you started this project?

I've answered a few times on the first question. It's a long process, and I can only offer an opinion. I believe there should be more data shared with the users so they can keep a better eye on what is going on.

Did we plan this whole thing? We just planned to make a video showing that this was possible. We did figure that redditors would be interested to know that the website was vulnerable to this kind of an attack.

And yes, I've been a redditor for 8 years. My account is now banned for an infraction of the site rules.

I'm a /r/Videos mod, you reached out to /r/bitcoin but didn't reach out to us, why?

Edit, additionally, why have you not told the admins either of this, either before or after so they can at least stop this kind of thing too.

I think I was banned from /r/videos actually so we stopped talking. I had a few debates with you guys about the "no politics" rule, and then you banned me.

UPDATE

Also, we contacted the press team at reddit on Monday before we published, but we didn't receive a comment.

Doesn't look like you're banned, sure it was us?

This is a throwaway. My main account is now site wide suspended. /u/phillyharper

Are you manipulating the votes on this submission?

No.

How did you get the idea? How long did the investigation take?

We worked on the story for months. Not because it was super difficult, just because we have many other things competing for our focus!

The idea I've had for years. I've been a redditor for years, and I've always had a niggling feeling that this was going on. I saw /r/hailcorporate and read about Correct The Record and decided it was time to investigate it properly.

Do you think we will ever see a social media platform that sustains itself without ads?

MySpace, Facebook, Twitter, and Reddit all became popular for one reason: it was all about the user. An ad blocker used to do wonders on those sites 7+ years ago but now with integrated ads, idk if it will ever be possible to have a true, user-centric social media platform.

Every single social media site goes to shit as soon as a corporation with an agenda gets its hands on it. It's really frustrating when you consider how easy it used to be to create a fun, carefree internet experience even less than a decade ago

We need to understand that if you don't pay for the product, you ARE the product.

We've got to start paying actual cash for our digital services, including journalism and social networking.

Hi and thank you for the video really opened up a new can off worms for me as surprising as it sounds being a lurker for many years. I was just wondering why? Is it because people just in it for them selfs as in for the money the popularity what?

I think it's likely to be a whole range of motivations. The important thing to know is that it's possible.