I work at reddit, Ask Me Anything!

Is it true that you guys can just give gold to anyone without having to pay for it?

Yes. But asking for it or hinting about it definitely doesn't help your odds.

Since your job is to be on reddit all day, do your bosses ever catch you filling out spread sheets or filing earnings reports?

I'm sure /u/yishan would fire me immediately if he saw me toiling over spreadsheets.

Which sub gives you the most day to day headaches?

I'd rather not name names. It is probably ones that you don't expect.

There are folks on the site which take joy in spending as much time and resources as possible to cause as much real-world damage as possible. It is an unending struggle to keep them at bay. I'm not going to name those folks, as doing so would only feed them.

should be under /r/InternetAMA as per karmanauticals rules

sorry boss man, i love the site :)

Also, is chooter replying for you?

:P

If /u/chooter was replying for me, my answers would likely be much less rambly and much more interesting.

Is there anything that counts as NSFW for you?

Nope.

In your view, how has the employee culture and attitude changed in the wake of the very high profile negative press Reddit has received via the /u/violentacrez affair, the /r/findbostonbombers debacle, and the recent seddit kickstarter controversies (and others)? Are there any clear, overt indications or movements or pressures by Reddit (big-R) to deal with these types of things before they arise?

How do you feel these types of events have changed the communities themselves as Reddit grows and changes (from the good ol days)?

Well, those incidents certainly have been learning experiences, as stressful as some of them were.

Obviously we would like to step in and address situations before they blow up. We're very passionate about the site and when bad shit happens, we feel horrible about it. Some stuff can't be anticipated, some stuff can't be prevented, and some stuff we purposefully stay out of to try to maintain as much neutrality as we can (despite that being very painful at times).

There have been no pressures from "on high" regarding dramatic situations. We the employees are in charge of our destiny, and we make the calls for what needs to be done to address problems. Since we are in charge of our destiny, that also means that this stuff falls directly on our shoulders.

I hope that the community has learned from some of these situations so that they can be more cognizant of some of the real-world consequences of actions on the site.

With the constant community turnover that comes with being one of the largest social media websites on the internet, the community can't really be relied on to know the history of our embarrassing moments. That leaves responsibility to keep things sane with subreddit mods and admins - when new subreddits pop up for something akin to the Boston bomber hunt, will the Admins be stepping in in the future? What did you learn from that incident and how has it effected your policy, written or unwritten?

With the constant community turnover that comes with being one of the largest social media websites on the internet, the community can't really be relied on to know the history of our embarrassing moments.

True enough. One point I'd like to bring up is that we won't grow forever. There will be scenarios where lessons are ingrained into the community mindset. Even if only 5% of users are able to recall, they can make a difference.

That leaves responsibility to keep things sane with subreddit mods and admins - when new subreddits pop up for something akin to the Boston bomber hunt, will the Admins be stepping in in the future? What did you learn from that incident and how has it effected your policy, written or unwritten?

God forbid another incident like this occurs, we will obviously use our experience from this last incident to help us make a decision. Every situation is different, so I can't begin to imagine how the lessons of the past might apply to the decisions of the future. All I can say is that we will use the knowledge this past experience combined with a heavy amount of judgement when deciding what is necessary. We obviously want to maintain our neutrality, but there is a threshold where we may be required to step in.

What's your favorite piece of Reddit history? ex. Today you tomorrow me, cumbox, or any significant thread

I'm having trouble finding the thread now, but there was a post after the earthquake in Japan where someone was worried about their grandmother (maybe mother?). Someone in the thread indicated he lived nearby, and he went to check on her and brought her some food.

Can't find that thread.. someone help me out here :P

Edit: /u/hoikarnage found it here. Followup thread is here. Thanks!

IMO, one of the major failings on Reddit is the moderator system. The top mod is, for all intents and purposes, king. He can close a subreddit visited by the President and Bill Gates on a whim, he can make the default news or political subreddits omit any news he doesn't deem worthy or relevant (or which goes against his political bias), he can do absolutely nothing while a gigantic subreddit withers away (so long as he merely logs in once every 2 months), and he can make a once-friendly subreddit a hive of hate and bigotry.

The various solutions I've seen proposed (users voting for moderators, all the moderators voting on policies, etc) have their own failings , but are there any discussions about revamping the mod hierachy/power system, particularly for default or large subreddits?

I think one of the primary things we need to do here is take some of focus off of the defaults. Right now the defaults define reddit for many people. There is a vast amount of non-default content and discussion out there, but so much focus goes into the defaults that they've become a defacto standard.

If we can make the front page a bit more dynamic and make it easier for people to quickly discover what other subreddits exist, I believe the site will be healthier overall.

Sorry for being a bit vague. We're aware that the current situation is less than ideal and we're working on improving it.

Theoretically, if the NSA started fucking Reddit, what safe-word would you use to let us know?

Foliage. (christ I hope i don't use that by mistake now)

When you say "at reddit" do you work from home or reddit HQ?

Both. I primarily work at the office in San Francisco, but I also work from home somewhat often. I have a considerable commute, and I like working in solitude at times :)

How big is the reddit team? Do you collaborate with different companies often?

I think we're in the mid-20s now. There are around 10 folks that focus on the tech side.

I don't do much collaboration with companies. If I'm speaking with another company, it is typically a vendor (Amazon, Akamai, etc).

Your alien looks like he was stepped on :(

What's the most nervous you've ever been during a time Reddit has gone down? What were the ccauses and circumstances, and what did you do to correct the issue?

Edit: I can't decide what to flair this post as, either science/tech, other, or nsfw just so you get a cutsie little purple alien winking.

The Obama IAmA. I actually developed a good eye twitch from that which lasted a few days.

When we were working on it, solving one bottleneck caused people to flood into a different bottleneck, tanking the site down once again. It was a marathon of wincing.

We learned a lot in the end, so I s'pose that is good.

Whats your favourite subreddit?

Lately I've really enjoyed keeping up with /r/redditdayof.

[deleted]

How did you gt your job at reddit.

I saw a job posting in the blog subreddit, and applied. I wasn't looking for a job, but I didn't to pass up the opportunity. I was living in TX at the time and the concept of moving to San Francisco seemed completely irrational at the time. Luckily it all worked out in the end :)

What do you mainly do?

My primary focus is on the infrastructure of reddit. I take care of the servers. Have a more detailed answer to this elsewhere in the thread.

How are you guys going to improve reddit in the future?

One thing I'm really looking forward to is multireddits (currently in beta for gold users). I had 500 subscriptions and a single front-page was less than ideal for that scenario. Using the multi feature I'm able to split subreddits up and categorize them. It's the biggest change we've done in a while, and I'm very excited to see what people end up using it for.

Don't you get tired of reddit and how much time do you spend on the website every day?

The time I spend on the site varies from week to week. I do occasionally get weary for different reasons, and just focus on the behind-the-curtain stuff for a while. Other times I'll spend a considerable amount of time on the site. All depends on my work load and mood for the week.

You work on infrastructure, servers. What's up with the servers anyway? Seems like they are overloaded regularly.

Kinda answered here.

How do you procrastinate?

I tend to procrastinate on things I need to do at home by working.

For example, I took a few days off recently to do some spring cleaning, and I conveniently ended up working all three days.

First off, ty to the admins on the domain level ban of Quickmeme. Manipulating our community ain't cool.

How much stock is really put into mod evidence gathered when tending to matters such as this fiasco?

Example

Whenever we get reports of something shady going on, we have to independently verify it before taking any action. There are obviously some sources which tend to be more reliable, but regardless we have to investigate it ourselves.

What is your favorite website that isn't Reddit?

Google Reader :( Oh, and TeamLiquid.

Is Reddit paid for some of the AMAs that it hosts?

We are not, nor do we ever want to. We do offer advice to anyone that wants it, but we do not accept money.

What kind of advice do you offer?

We try to give folks an idea of what they're getting into. Many of the notable people that do IAmAs are completely unfamiliar with reddit, so it is a very foreign environment.

We try to encourage people to engage in discussion instead of looking at it solely as another marketing loudspeaker. The people that engage in actual discussion with reddit users tend to have the best IAmAs.

Do you believe Reddit is currently at, or has past, its prime? If not, when do you see Reddit reaching its peak? What will cause its ultimate downfall?

I dunno, that's like asking if human society is past its prime or not, and what will cause its ultimate downfall. I really have no clue.

At the present, the site is still growing. That's all I can say for certain :)

Do you reddit at work when you're working for reddit?

Yes.

What would you say is your greatest contribution to Reddit?

Getting through the outage-filled months of 2011. Things were in a very bad way during those times. A tonne of technical debt was piled up, and working through it was very painful. At the end of 2011, I managed to get things in a somewhat stable state (with considerable help from the dev team). Stability is still pretty far from where I'd like it to be, but we've come very far, and we're continually working on improvements.

One of the things I'm really proud of is the SOPA/PIPA examination blog post. I was really disappointed in the hyperbole flying in both directions, so I wanted to take time to pull the text apart and explain what was being said. It was far outside my usual comfort zone, which I liked. It was also exhausting, since I'm by no means proficient in that area :)

What kind of infrastructure requirements does a site of this magnitude have? I can imagine the amount of rack space, servers, switches ETC are off the charts.

The site is entirely hosted on AWS. These days we're clocking in around 350-400 instances of varying sizes.

We use many different pieces of tech to keep running. To name a few:

• Postgres
• Cassandra
• memcached
• haproxy
• nginx
• rabbitmq
• zookeeper
• hadoop
• gunicorn

What do you guys use hadoop for?

Traffic stat processing, mostly.

[deleted]

I think the site has become fairly influential in certain crowds. When the President takes notice, it is obviously an indicator that we are nearing mainstream. We're still obviously orders of magnitude away from Facebook/Twitter scale.

All of our revenue comes from ads and gold.

What do you do, exactly? (no offense intended)

Day to day I manage the infrastructure of reddit. This entails keeping an eye on all of the servers, putting out any fires, building out new infrastructure, and planning for the future.

Have you ever had to literally put out a fire?

Only the ones started by my counterpart, /u/rram. My work desk is covered in a canvas, so I have to watch out when he's waving lighters around.

Neutrally speaking, what do you think about Reddit being primarily political supporters of liberal or progressive causes? What kind of culture on reddit do you think contributed to this overall political stance?

I think that a good chunk of this is a natural result of the demographic which visits the site. When you have a majority of young tech-inclined folks, it seems to me that the political average tends to swing to the left. Of course this then attracts other folks from the left, so it can be a bit self-feeding.

I imagine the majority of reddit will probably lean in that direction for some time, given current trends. Indeed the internet as a whole seems to lean in that direction. Time will tell if this shifts at all.

Is there a comprehensive list of subreddits somewhere?

Edit: I reddit using only my mobile....sorry if this is a dumb question.

reddit.com/reddits will give you a list, but you have to paginate through it so it isn't ideal.

An alternative for a full-ish list is http://stattit.com/subreddits/by_subscribers/. (/u/Deimorz is now going to murder me for linking to a page which is likely quite heavy)

Is systems administration fun?

I enjoy it quite a bit :) I like fixing things, and I like fixating on things. When an interesting problem pops up, it is quite interesting taking it apart and learning exactly what is going on or how to solve it.

Is there a reason for you guys not expanding faster? Several times a day the "servers are slow"-message on Reddit, and I mean, it's not like Reddit is some independent site worked on by college kids with their lunch money.

We can expand the servers as fast as we want. Unfortunately that doesn't really solve the problems. If simply expanding the servers would solve all of these issues, I would kick up everything this afternoon :)

Right now our biggest struggle is that some layers of the site are no longer holding up as they used to. The app has an extensive internal-caching system which has served us mostly well for years. However at our current scale it is beginning to cause a lot of problems. What sucks about this is that we have to completely re-develop those pieces of the app to solve the problems. Just developing those solutions takes a considerable amount of time. Additionally, trying to figure out a solution which will get us through the next set of years takes an incredible amount of thought.

Overall we've come a long ways from where we were in 2011, in terms of stability. Obviously we have a long ways to go, and the past month or two has definitely been challenging. All I can assure you of is that we're spending a huge amount of time on these problems.

Gotcha, thanks for the answer! Also, are there any plans on making an "official" iOS/Android-app for Reddit?

Our current strategy is to enable other app creators to flourish. We don't have plans on our own app at this time. Obviously that could change in the future.

IMO AlienBlue is by far the best iOS app.

What is your salary? (Are you having second thoughts about AMA instead of AMAA?)

You can ask me anything; doesn't mean I have to answer everything :)

Do you guys have any control over individual subs? Like can you monitor them and take them down if necessary? What's your least favorite sub?

We have the power to take down anything on the site. However we leave most of that decision making up to the moderators. The mods call the shots on what is acceptable in the subreddits which they manage. Obviously there are some cases where we must step-in, but overall it is extremely uncommon.

What stuff do you guys do behind the scenes to make our Reddit experience better, that we don't know about?

To quote a coworker: "Like the night janitor - you know that someone is emptying the garbage cans, but you don't really think about it".

We spend a huge amount of time going after spam, vote botting, and various other evildoers. Bad shit inevitably slips by, but the stuff that does is the tip of the iceberg.

Fellow sysadmin here. i have a couple of questions regarding your network.

-I read somewhere you use haproxy for load balancing is this still true?

-What software do you guys use to analyze your webserver logs? If you use logfiles from your LB's, how large are the logs from a 'normal' days usage?

Yep, haproxy all the way. It has been good to us.

We have around 200G of various logs a day. It is all shipped off to S3 and then deleted after 90 days.

How many cats do you own?

1.

Being a systems admin and having to put out fires I would assume that you are very intimate with the inner workings of EC2.

• Has using EC2 (or other mix of cloud architecture that I do not know about) made it possible for reddit to flourish?
  

reddit has been operating on a very lean staff since inception. Shortly after I joined the tech team consisted of 2 sysadmins and 1 dev. I don't think we could have survived if we also had to worry about managing the physical resources of our own infrastructure.

Obviously any platform-as-a-service product has its pros and cons. I think since we moved to it it has been positive overall, given our requirements (crazy fast growth with very small team).

[deleted]

I am degree-less.

Did NSA ask Reddit for information like they did to facebook and google?

We're not involved in that program. We can be compelled to turn over stuff with a subpoena, but we fight tooth and nail if a request is overly broad or bs.

Of course, this turns into a chicken-and-egg of whether you believe me or not. I've expounded on this heavily elsewhere, so you can dig through my comments if you'd like.