Had some people asking me to do an AMA after replying to questions in an ELI5 thread, so here trying this again. I did an AMA just after I left (about a year ago) on the same topic: http://redd.it/11pcvy

I did a lot of work on Flash performance there, so if you want to ask any questions in that direction, feel free.

edit proof: old business card

edit 2 it's 2 AM, so I'm gonna go to bed. I'll answer the questions already in my inbox now, and I'll get to anything else that gets asked tomorrow morning

edit 3 actually going to bed now.

edit 4 and I'm back, and this blew up while I was asleep. I suppose I should thank the guy who said this was front page material, since I didn't believe him. Will try to answer everyone, but there's a shit ton of you.

edit 5 and finally all direct questions answered and trolls ignored. Thanks to whoever game me reddit gold, I suppose I'll have to figure out what that does. I'll continue to answer anything posted that replies to me directly, there's too many comments for me to reply to every single one of them. might take me a while tho, cause I do actually have work I want to get down.

edit 6 since a few people have asked - I'm working on an indie horror game with an artist friend. we haven't shown much yet, but if you want to keep up to date you can follow us on FB: https://www.facebook.com/glowstickgames

Comments: 1002 • Responses: 61  • Date: 

ahmadalfy578 karma

Why does it feel like social games (games hosted on social networks like Facebook, Google+ ...) are very similar? The all share the following:

  • Usually there are 2 financial units in the game, one is hard to get and costs real money, one is easy to get and you can't really much with it.
  • Daily reward for playing each day.
  • Tasks/missions that require you have x-number of friends/neighbors who play the game.
  • Visit your friends/neighbors everyday for special reward.
  • Tasks that take time could be skipped using the first type of financial units I mentioned above.
  • Usually there is a limit to the time you can play. Your character have stamina or could do x-number of action and have to wait x-number of minutes before that stamina is renewed.

Who is responsible for the lack of creativity here? Who forced this model? Every game feels like it's exactly the same with the change of the graphics, dialogue, story line ...

Thanks for doing that AMA :-) I asked you mainly because I wanted to ask about Zynga ripping off other games but that was asked by some fellow redditors.

LagrangePt569 karma

Facebook games blew up at the same time as two other fundamental design concepts: Freemium and Games-As-A-Service.

A lot of innovation was done on how to make those models work, and everyone was copying mechanics from the games that were most successful. Some companies actually understood why those mechanics worked and used them well... some didn't.

2 currencies - 1 can be earned in game easily and can unlock most content. Easy reward to give players. The other could be earned in tiny amounts, and bought in large amounts. The most successful games were fun without the second currency, but made plaeyrs always want it.

various things over time - serve the game in limited bites, and make the player come back for more. this will create a lasting relationship with the customer, which is vital since players very rarely pay anything on the first day they play a freemium game.

require x friends / neighbors - creates artificial virality, where people would get other people to join the game. much cheaper then buying tons of ads.

skipping time for money - again, games do everything they can to make players want that second currency.

stamina - usually called energy mechanic. games that had an energy mechanic made a huge amount more than games that didn't.

Games that didn't have those mechanics didn't make enough money to keep updating, and if a game didn't continue to provide fresh content all the players would leave.

After 1 game is a huge success, everyone else makes a similar game and tries to steal a piece of that pie, and future games try to recreate the initial success.

There was a lot of innovation - early farmville didn't have a lot of those mechanics, they were added in later. We also tried a lot of mechanics that failed and were ignored. However, it was an environment where copying a feature took weeks instead of years, so all the games copied the most successful games.

tsujiku162 karma

Do you think there are any ethical considerations with some of these mechanics?

A lot of people oppose 'pay to win' mechanics, and I know I personally find the constant requests to join random games to be extremely annoying. Are those things that are considered at all when designing a game with these mechanics?

LagrangePt523 karma

ethical considerations? absolutely, it's why I quit. A few months ago someone tried to get me to add those mechanics to a real money gambling game, which I refused to do (and convinced the other guy not to pursue either).

re pay to win - the invest and express model is not a game that can be won, so their is no pay to win. Farmville and similar games were about growing and nurturing something, similar to sim city.

Being completely honest, while I didn't like spamming, if you weren't part of our target audience the business people didn't care, and if you didn't like the game invites then you weren't part of our target audience. Facebook eventually cracked down on the spam a lot, which caused Farmville to drop from 30 million users/day to about 23 million.

Design is done by a bunch of people, who then have to justify the designs to the business people at the top. so while there were always arguments for not spamming and not adding negative mechanics, at the end of the day Zynga was in the business of making money, so that's what got put into the games.

thekeanu446 karma

Was not expecting this level of thorough/quality answers.

Kudos, OP

LagrangePt573 karma

my fingers are tired, and I type for a living -.-

dangerdark304 karma

[deleted]

LagrangePt428 karma

I sorta answered this in my previous AMA, but I'll summarize a bit:

Zynga focuses on adding social mechanics to games, not innovating core game mechanics. You could take the original unreal tournament and Doom and say that one copied the other because both are about shooting stuff, while ignoring that UT was focused on shooting other players. Zynga had no shame about copying core game mechanics from other games, but the vast majority of our time was spent trying to come up with new features to add into games after release, and a lot of those were actually innovative.

TL;DR: Zygna ripped off core game mechanics and added tons of their own features and systems to them. As far as I can tell, this is standard practice in the games industry, though usually companies try to innovate the core gameplay as well. At no time did I witness Zynga trying to straight up clone another game, though I did hear rumors.

gnarfel190 karma

Where do you see HTML5 going, in regards to replacing the traditional plugin model?

Is HTML5 video a good idea?

How is writing for the Canvas element (and the associated optimization pitfalls) different from writing complex programs in Flash?

LagrangePt283 karma

I had to work with Cocos2dx (with JS bindings on mobile) and Cocos2dHTML5 for a while after I left zynga.

Overall, HTML5 replacing flash for basic stuff like video and interactive layout is (I feel) a good thing.

OTOH, Creating interesting interactive UI in flash is way easier than any other system I know of, which is a large part of why it's so popular. That hasn't been solved by HTLM5 yet, and it will take a lot of work on the tools before they match up to Flash creation tools in Adobe CS.

HTML5 also has a lot of really annoying parts when you have to accommodate older browsers, and sometimes it's actually faster to use heavily abused DOM rendering rather than using Canvas rendering (which makes for headache inducing sessions of trying to optimize your shared code for DOM on some browsers and Canvas on others). Having webGL isn't reliable yet, which means that a lot of things that are easy in flash are hard in html5.

HTML5 also suffers a lot of the problems that flash has, like really horrible perf gotchas that non-senior devs won't know to avoid. The browser makers are putting in a ton of effort to make HTML5 run fast, but even heavily optimized html5 code seems to have a lower overall performance than decently optimized flash code.

Any0neInCherno137 karma

I haven't got any questions, just wanted to say thanks for the detailed, thought out responses. Definitely one of the better AMAs I've read through.

LagrangePt102 karma

thanks!

synJstarcraft120 karma

Do you think Unity is worth the investment to learn? If not, where might one spend their time learning the necessary skills to compete?

LagrangePt212 karma

Absolutely. Unity is the easiest game engine I've ever worked with, though I haven't had time yet to dig into the really interesting stuff. It's a great way to get a basic idea of how 3D development happens and what important parts are, but it's really light on the coding side (a lot more development happens in the editor instead of the coding).

It's great for making a couple projects in. If you want to develop valuable skills for traditional industry, I'd point you to try out the UDK (unreal development kit), which is a lot more coding than unity, but still easier on you than I've heard other engines are. I had a lot of trouble with unrealScript, but I've been working in very c like languages for years.

Aside from that, I've heard that just picking your favorite PC game and getting involved in the modding scene is really good, but I've never tried it.

umar16790 karma

What has your career path been like so far?

LagrangePt230 karma

I started out as a research assistant to a professor I worked with in college. (summer job after graduating). I was mainly focused in #D graphics and physics simulations, with some work in AI and massively parallel computing. Joined zynga as a junior dev in 2009, starting on Farmville. I stayed on for about 18 months, which at the time left me as the programmer who had been on the project the longest. During that time I trained up as an Ops responder (servers crashing, go fix it!) and flash optimization ("We need to replace all XML accesses with Object accesses, we'll get another 2 FPS out of it"). I also made a few systems no one else had time / knowledge to, like arches you could walk through. I made a system for custom ground tiles that was really awesome, but the business people sat on it cause they could never budget the horribly complex UI they wanted to use to sell it to people (IT could have gone out with trivial UI work, but it wouldn't have made enough money that way). Near the end of that I started coordinating company wide performance tracking and arguing for improving it. I was also managing our engineering relationship with Adobe then, which really should have gone to a higher level employee.

After that I moved onto the central engine team, which meant optimization, generalization, integrating improvements from other teams, supporting other teams, and generally arguing for better engineering practices. Co-wrote a blitting rendering engine for flash that could be hugely faster than the default engine if used right, then spent ages helping teams replace it. Got flown out to remote studios to help them get launched and improve performance. Got to write most of a component system for our flash engine that would have made it a cross between linux library installation and Unity easy-to-add components, though I'm not sure if that every really got used before Zynga shifted everyone over to working on mobile.

After I left Zynga, I joined a small startup founded by the same engineers who had originally hired me. Worked on mobile games there for a while, but eventually they pivoted to focus on analytics software. Did a lot of work with Cocos2dx and Cocos2dHTML there, though I wish more had been open sourced. I think they're still considering selling that stuff as middleware, and if they don't I doubt they'll take the time to open source it.

After that, I joined up with an artist friend and we're working on making a horror game (with both of us focusing on making an awesome game first, tho we definitely want to make money off it still). Hopefully we'll have a kickstarter out for it relatively soon.

shiroboi49 karma

Any good tips or resources for resolving isometric z layer orders? I tried to make a tower defense game with terrain height and we've constantly had problems with objects on a higher layer moving behind objects on a lower layer. I'm an artist by the way but it gave my programmer fits.

LagrangePt119 karma

hahaha, whenever I ran out of questions to ask someone in a job interview, I'd ask them a depth sorting question. If they gave a good answer it'd be almost guaranteed that I'd recommend them for the job.

So I spent a lot of time working on a fast sorting algorithm that would work on rectangles on the ground. Farmville still has depth sorting bugs when you put a fence next to haybales.

In later games we went with the solution openGL uses, which is to cut up the image into multiple pieces and render them at different depths. this was when we were using a custom blitting engine - in normal flash rendering splitting up an object is expensive.

if everything is a cube, depth sorting is fairly easy. as soon as you have rectangles larger than 1x2, you break one of the fundamental assumptions of sorting algorithms (if A<B and B<C, then A<C. not always true in isometric depth sorting).

If you don't need to resort frequently, you can use graph search algorithms to construct a correct way to sort convex objects. If you are letting stuff move frequently, I'd recommend just accepting that it'll bug sometimes.

bananomgd-13 karma

Going by your username, you're from Portugal. What university/higher learning facility did you attend?

LagrangePt21 karma

um, what? Lagrange is a french name. LagrangePt = Lagrange Point http://en.wikipedia.org/wiki/Lagrangian_point

I went to University of Pennsylvania, and grew up in Dallas TX

Biblius5 karma

A for effort!

LagrangePt5 karma

thanks. I probably shouldn't have started this right before I was going ot go to bed.

dangerdark66 karma

[deleted]

LagrangePt115 karma

I'm 27 now, was 23 when I joined Zynga.

I originally started programming LogoWriter in 5th grade, but I dropped that and didn't pick it up again until Senior year of high school.

I graduated in 2009 when the US economy was in the shitter. I spent about 9 months looking for a job before I got referred to zynga (my mom met someone who knew someone who worked there). They were hiring like mad at that time, and I did well in the interviews they offered.

surealz27 karma

so you were self taught?

LagrangePt77 karma

not really, though I've taught myself a lot since I started working. In 5th grade my math teacher got some of us to volunteer to spend half of lunch period in the computer lab 'playing' with Logowriter. My high school offered a class on Maya and a class on some Lego robot software, both of which I took in senior year (we had an awesome retired engineer running our computer lab)

edit didn't realize this was a reply to a reply.

Angrysausagedog55 karma

What was it like working for Satan?

LagrangePt91 karma

At first, it was amazing how much fun people were having playing shitty flash games. Sure we encouraged people to spam some, but I also knew some of my college friends that sat on their FB feeds hunting posts to get rewards (and apparently quite enjoyed themselves). There was a scandal about the service we used to let people get farmcash without paying, but I thought it was handled pretty well (service was dumped, we signed on with another service where we had to approve every offer that was posted).

A few years later, I sent a (somewhat angry) email to our CEO arguing game design and how we had taken a series of small steps in a very bad direction. Then I quit a couple weeks later.

Angrysausagedog18 karma

where you guys encouraged to "borrow" content ideas from other companies?

I used to work administration for another web based flash game with a virtual world type gameplay that later went on to become a facebook integrated app as well, (much like YoVille but far more advanced) during that time I noticed a lot of the zynga apps starting to use content (specifically virtual items) that was remarkably similar to what we had recently released.. (the biggest offender being YoVille)

was this coincidence and I was just paranoid, or was it a deliberate attempt on Zynga's part to cash in what was new and popular in other games?

LagrangePt44 karma

So, keep in mind I was somewhat removed from this.

invest and express games like Yoville and farmville churned out shit tons of content, and the artists were always look everywhere for ideas. Farmville actually bussed the entire team out to farms in Napa valley to inspire new features and props (back when we were still trying to stay realistic).

That being said, we had business development and game designers who constantly played all the successful / similar games, looking to see what they did different, puzzle out why, maybe throw in an A/B test in the game to see if we should do the same, etc. I wouldn't be at all surprised to hear someone say "Game X just added a juke box. We should totally have one too". I would be very surprised to hear "go look at this game and see what items we should copy from it".

Angrysausagedog15 karma

I guess at the end of the day business is business.

The main victim here is originality I suppose.

LagrangePt37 karma

I've talked to and watched a lot of respected game designers. They all say they just take other people's components and mix them together in a new way (I think there's a good quote from Sid Meyer or something about that).

Even games like minecraft pull a lot of inspiration from others that came before

FattyMcPhatterson49 karma

Do you miss Zynga after a year away? How does your new job compare?

LagrangePt117 karma

I miss some of the perks, I miss a lot of the people, I miss the dogs, and I miss the paycheck.

Most of the perks aren't being offered anymore, a lot of the people (and their dogs) have left, but the paycheck still hurts.

twentythree-nineteen49 karma

I miss the dogs

???

LagrangePt12 karma

zynga allowed workers to bring their dogs into the office, which meant I could play with 5 to 10 dogs every day. I knew the name of several dogs without knowing their owner's names

poopie_pants41 karma

If you were made CEO of Zynga, how would you restructure the company?

LagrangePt67 karma

hmmm, at this point I'm not sure. I haven't paid much attention for a year.

If I had been made CEO a couple years ago, when I was really involved? I'll try to not use hindsight a lot, and I'm pretty sure some stuff is still covered by NDAs. I definitely would have approved some games that got canned, and canned some of the games that were just more of the same. Being able to deploy good experiences to mobile would have been a higher priority, and I may have mandated that more teams learn unity, cocos2dx, and other cross platform solutions (Zynga did have a big effort with tons of senior engineers to identify promising technologies for cross platform development... then completely ignored all the suggestions it made).

I would probably have done much worse a job than Pincus - the man has his faults, and I think Mattrick will do good things, but Pincus is a very good businessman.

EDIT two major things - merchandising! Zynga was horrible about ignoring it due to low margins, ignoring the brand an other benefits. Second, focus! We were scattered in all sorts of directions with all sorts of efforts. should have focused in on a few things and done them well.

dyl4001136 karma

Do you receive any pressure like deadlines etc. and how did you get hired and where did you start?

LagrangePt50 karma

deadline pressure was always there for feature devs, but I was a bit isolated from it since I was more Ops and systems development (fixing the game when someone broke it, improving performance / stability, etc)

I got hired 3 months after I graduated college (I had been working as a research assistant for one of the professors). I answered in another post more about how I got hired.

I started on Farmville only a month or two after it went live.

gnarfel28 karma

How did the requests from 'the business side' of the company affect your vision of the games you created?

LagrangePt69 karma

Well, I was there to make money - it was a job. At first I was trying to pay off student debt, and later I just wanted enough money to be able to 'retire' and spend a few years creating my own game.

All too often, it was a debate between idealism and realism - the game needs to make money in order to justify developing it. the business side is really good at uses stats to prove their side, which appealed to the higher ups. Unfortunately, business statistics tends to be very short term focused, which (I feel) cost a lot in the long run.

I was usually on the "but that isn't fun, which will ruin the game" side of the argument, but I often had to justify my ideas with "this will improve the game in this way, which these previous projects have shown will increase rev by this much"

Edit re read it and I feel I left out something important - I got into games to make people smile, not just to make money. I stayed at Zynga because I was reaching tens of millions of people every day and making at least most of them smile. I left zynga when I felt it was becomming far too negative on that metric.

nasty_eardrums20 karma

How do you measure how many people you made smile? (serious question)

LagrangePt121 karma

I had to spend a lot of time on the forums hunting for bugs (which is a soul destroying task). I would periodically dive into the more positive parts of the forum and remember that there were people who really loved the game for various reasons.

I also wore farmville shirts in public, and so I got to have random people coming up to me and talking about it sometimes.

My first day at zynga, I made it so that clicking on stuff was pixel perfect (instead of being driven by bounding boxes). 10 million people played in the next 24 hours, and a good number of them actually noticed the difference and posted happy feedback on the forums.

captsindbad24 karma

[deleted]

LagrangePt55 karma

Working on a horror game to come out on Steam and possibly other platforms. Will probably be launching a kickstarter for it soon, since I'm bootstrapping it with an artist.

Company name is Glowstick Games, tho we don't have a fancy website or anything yet.

freddymungo16 karma

Please let us know when that happens!

LagrangePt40 karma

We've tried to post a couple times. I'll post another AMA? not sure the best way to communicate it without being spammy / annoying.

poborskyyy13 karma

[deleted]

LagrangePt30 karma

I only did this cause people were asking me to in another thread. wasn't even going to use my personal account for promoting the game

Jefkezor21 karma

Just post a summary+screens/vids of your game on /r/gaming, /r/pcgaming and /r/pcmasterrace. There are probably a few more subreddits you could post in, but I'm not sure...

LagrangePt19 karma

thanks for the advice!

ahmadalfy10 karma

Now I feel bad for asking for that AMA :(

LagrangePt12 karma

no biggy. It might help with launching the game I'm workign on, so it's good you got me to do it. even if I'm now tired and my fingers hurt from typing too much

my-little-pwny3 karma

Maybe make a topic in /r/gaming about it as it releases? People could just ask you questions about it there.

LagrangePt6 karma

I'll almost certainly be doing that when we do the kickstarter.

Ryan_Bingham23 karma

Do you think Flash has reached it's final form? I stopped dev in Flash a bit after AS3 came out (yes, it's awesome i know), do you think Adobe will update AS3? id there is anywhere to update at all.

Also, do you use FlashDevelop/eclipse or the standred IDE?

LagrangePt43 karma

As far as I know, Flash remains the easiest way to develop awesome looking UI. Adobe might pull something awesome out of it's closet (they were working on some cool stuff when I left), but I haven't paid much attention for the last year.

That being said... I think they made a huge mistake with mobile, and ultimately that will be the downfall of flash as a wide spread web solution. HTML5 is not standardized well enough though, so people will still want a truly cross platform solution for ui... not sure what that will be tho.

edit I worked in flex builder for the most part, though I know a lot of other devs were starting to use another editor shortly before I left. one of the third party ones, I don't remember which.

krilnon9 karma

Did you go to the Flash gaming summit thing at Adobe SF the summer before you left? I was around for that and there were several Zynga people there. It would have been where we talked about some of the cool stuff you mentioned.

Edit: typo: were -> where.

LagrangePt14 karma

Yes, I was in fact at that summit (My office is across the street from there)

I also talked about a lot of stuff with adobe engineers in zynga-adobe meetings, since we were right next to eachother. We participated in a lot of early access tests since we were really needing the products and could give them face to face feedback really easily.

krilnon6 karma

Hmm, neat, then we've probably at least seen each other before. I don't remember who exactly I met there. I worked out of the San Jose office, but I walked past the giant Zynga dog a number of time when I was at the SF office temporarily.

LagrangePt9 karma

very possibly. If you were ever at a zynga/adobe engineer meeting, I was probably the youngest guy in the room.

krilnon5 karma

I wasn't ever at one of those, but I think you would have been the second youngest. I'm 23 now and was 22 when I was at Adobe.

LagrangePt3 karma

Last time I would have met with adobe would have been summer 2012, so aside from that big summit we were probably never in the same room.

edit my minds a bit fuzzy on the topic, but I think I may have presented at that summit. I may be mixing it up with something else tho.

Iwishiknewwhatiknew21 karma

Did you go to college? And if so, did it prepare you for game development?

Also, with all the game development software, which one would you say has been the best for you.

LagrangePt29 karma

I went to UPENN, in the Digital Media Design major 3/6 Comp Sci, 2/6 Art, 1/6 communications.

It prepared me really well for programming in general, though I wish I had signed up for more classes about internet development and architecture (running servers, making web services, etc). The most valuable part is the algorithms and problem solving approach, as all the tools and languages I used in college I haven't worked heavily in since. (aside from c++).

Hard to say with game dev software. I got really good at flash, I got pretty good at the Cocos2dx-JS / Cocos2dHTML5 stack (which is Javascript for scripting, with either a JS engine or a c++ engine, with the c++ engine binding down into either Java or objective C...) I think my current fav is Unity, but part of that is how easy it is and that it's one of 2 3d engines I've worked in since college. Unrealscript frustrated me in comparison (a task in unity took me 1 line of code... took me about 30 in unrealscript).

Richboy4557 karma

UPenn? My fiancee graduated from there this year and told me about someone she met that went to work for Zynga a few years back. Maybe it was you?

LagrangePt15 karma

After I got into Zynga, I referred a bunch of people that graduated with me (none of us had jobs... 2009 sucked for entertainment industry).

I know at least 3 other people (may have been 4 or 5) that graduated in my year and that went to Zynga shortly after I did.

dpwhitey21 karma

[deleted]

LagrangePt106 karma

umm, whatever I set it to? I remember that once I got pretty sick and couldn't think straight, so I stayed at home, gave myself a few million in farm cash, and proceeded to decorate the hell out of my rather boring farm.

I had direct access to the servers and to the admin tool. I could basically modify any blob I wanted in any way I wanted, though I generally only used that ability to fix player blobs and give the team artists tons of farm cash so they could display all their work on their own farms. And occasionally giving myself limited time items that I'd missed and decided I wanted >.>

nicknitros19 karma

Did they pay you well and/or did you feel that you were sufficiently compensated for your work?

LagrangePt32 karma

Pay was awesome, bonuses were frequent, and I was getting a lot of stock. Then the stock price dropped to $8, then $6, then $3... the pay was still good, but the perks and bonus were dramatically reduced, and the value of my stock was 1/5th what I had been hoping for.

ZigguratOfUr15 karma

Cool AMA. What do you think are some of the important engineering practices to successfully launch and maintain a social game? You mention that this was part of your job: Do you have a list of greatest hits in improving engineering practices?

LagrangePt27 karma

So Zynga actually did a lot of work and innovation in rapid iteration large team production. At The time I left it, Farmville had (I think) 4 different teams working on different features, each staggered so we were putting out one or two major features every week.

We also made a lot of mistakes. We frequently allowed 'tech debt' where we added a sloppy feature we would clean up later. These piled on themselves and slowed down production as time went on, to the point you needed a 4 man team to implement a relatively small feature.

Most important part - intelligent branching and merging system. we used SVN and it was a nightmare, we switched to GIT later but I was off the game teams by then.

Every dev had their own development app. work was merged into a pod app as they worked on their feature, which they could call a team of QA to test at any time. After one of those tests, shortly before their feature would release, they could merge their changes back to trunk, where it would undergo testing on the auto build app, then get pushed to 'staging' and undergo more testing (auto build was short build runnign on a single server, staging was a simulated production environment with databases and everything). The code would be deployed to all servers, verified, and only then would the game 'update' and switch to the never version.. After which we would test it again, watch the numbers and error logs, and eventually go to be.

We had a bunch of late night releases followed by early AM fires before we learned to just delay the release to the next mornign and fight the fire then.

We kept a details ops diary of EVERY incident that affected game availability. At one point, a senior ops guy joined the team and mentioned he would have paid millions for access to that diary at his previous job.

Whenever something breaks, Either fix it so it can never happen again, make a tool that can fix it in one click, or (best option) do both. Make a magic button that fixes every known save corruption / problem you've ever encountered, and let the CS guys have access to that switch for all players.

a typical feature had a dev cycle of 3 to 4 weeks.

Have dedicated engineers to work on server fires and engine performance.

We didn't do this much, but I'd stronlgy recommend it: Have at least one or two engineers working on completely new mechanics and systems at all times, and actually release the ones that wind up being fun.

ZigguratOfUr6 karma

A lot of this is very impressive, especially the attention to user experience metrics, and ESPECIALLY the ops diary/magic button (making a good magic button seems hard though; couldn't a fix eventually become a new way to corrupt a save?).

It also seems like bigger refactors or redesigns don't really fit into this cycle. Did Farmville get a substantial refactor to address its technical debt at any point, or was the plan basically for the initial architecture to survive the whole profitability cycle, then move to a new game?

LagrangePt17 karma

The initial architecture held up really well - it was the core engine from MyMiniLife, which was acquired by Zynga and whose team was the core of the original FV team. The main front end coders were an ex Epic developer and a guy who was awesome at enforcing coding standards and good architecture.

We did have some fairly substantial refactors, but in general every feature that was developed involved paying off some technical debt and adding new tech debt. The systems pod or a feature pod would spend a few weeks doing the refactor in a branch, periodically merging from trunk, and would eventually release the refactor as part of the normal schedule.

We never addressed tech debt in general, just specific bits of it when they couldn't be put off or when they actually caused problems. The ops and system team were the only ones who actually chased tech debt, and we usually had hirer priorities.

The magic button was tested every release, same as other features (the QA actually had to use it all the time to fix their own accounts).

In general, said button would hunt for specific problems and fix them. I.E. PHP was fine with a hash table that started with a 0 index, but AS3 would interpret that as an array and shit a brick at the next index not being 1. PHP was also fine with a null value for the index, but again AS3 would shit a brick.

We also would do things like checking that all the objects on the player's farm actually existing in our settings file.

Early on (shortly after I started working on ops) we were in a really bad state. A new manager came in with the philosphy "if it hurts, bandage the wound then spend the time to make sure it never happens again", which was immensely valuable. A database dieing went from "get a call and work for 1 to 2 hours to fix shit" to "read about it in the morning and make sure the backups were properly replaced"

migrafael12 karma

I just want to know one thing. Was it fun?

LagrangePt23 karma

At first, absolutely. the IPO changed the company a lot tho, and towards the end I wasn't having any fun at all.

I still have pictures of doing office chair jousting using nerf axes :)

giananimohit11 karma

Now I know this might be out of your purview, but I have to ask : is zynga poker rigged?

LagrangePt19 karma

As far as I've every heard, absolutely not. It also doesn't make any sense for Zynga to rig it, since they make money regardless of which player wins.

That being said, I have heard of people playing multiple accounts in the same room, so they have more knowledge then they should

Marfell9 karma

What is your opinion about bigger game developers? Does the stock ones tend to go more for porift and care about the shareholders profit rather than the product, or are they still willing to keep on producing good content at the risk of loseing some money?

LagrangePt32 karma

Being publicly owned ruins any company that is based on delivering hits (gaming is fundamentally a hits based business).

I can't really speak to the rest of the industry (though comparing public to private large game companies gives a very negative view). I do know that the culture of Zynga became a lot worse after the IPO.

Before IPO, it was all about long term health of the company, making games that would be fun and profitable, etc. After the IPO there was so much pressure to "just make the numbers for the quarter, don't let the stock drop any more"

blacklutefisk6 karma

Thanks for doing this.

I used to play Zynga Slots, and then one day, everyone with an iPhone received billions in coins (enough that one need never run out), enabling every machine to easily become unlocked. I don't know if you were involved in that game. If you were, do you know if that was a bug or by design? What was the response at Zynga when that happened? I recall it being around the time that the financial problems grew more visible.

LagrangePt7 karma

I didn't even know zynga had a mobile slots game >.>

I can make some guesses tho. It's possible that was a bug, but I can't see them not trying to reverse it if it was. It's also possible they were sunsetting the game and shutting it down, and did that for some odd reason.

Mobile at that time was coming out with a lot of games that didn't make much money relative to the big name games, so while I'm sure someone got chewed out over it, I doubt it made any huge difference. The big financial problems were because FV and CV were declining and nothing else was replacing them.

remyspam6 karma

Regarding performance. A few questions, although the last one is less important (and I'm not sure you can answer it anyway).

First: How did you do performance testing? Do you have certain scripts that populate games with hundreds of objects? Or do you let QA play long games?

Second: What is the main performance issue in Flash, is it the handling of sprites, background calculations, I/O, memory usage, etc.?

Third: My girlfriend is an avid Farmville player (level 190 or something), but Farmville is quite slow and bogs down, when playing. No doubt because of the gazillion animals, trees and whoknowswhat on that screen. Do you have any secret tips how to improve it? Eg. "play short sessions so the cache doesn't fill up", "use browser X or Y", "the games are heavily cpu/memory/fairydust dependent so improve that", etc?

LagrangePt13 karma

1) we had a number of ways. first thing we did was stick a framerate / memory tracker in the game client that reported to our stats systems. that lets us correlate framerate with items purchased, farm size, level, country, etc etc.

second we had an AIR app that would iterate over every swf in a given folder, would place a hundred or so on screen and test the framerate for a while. we ran that on a dedicated server, so we knew how expensive every asset we had was. not sure if that still happened after I left tho.

We were messing with more advanced stuff to give us full profiles of user's session (randomly sampled or selecting specific users) and a bunch of other stuff. I left before I could see data from that stuff tho.

2) rendering vector art is huge. parsing XML was big for us, but we had a lot of xml and used it stupidly. garbage collection can be a big problem, but we were pretty good about object pooling. Easy for GC to become big if devs are careless.

3) I don't know what it's like now - it's been 2.5 years since I worked on it. Most likely it's CPU bound. At least in the system as it was, having a lot of copies of the same animal in the same pose will make the game optimize that specific animal if it has memory space(faceing up is a different pose than faceing down, left/right normally didn't matter). If the devs haven't kept adjusting that memory limit, you're probably hitting it before it optimizes any of them. I do know I once had a farm with 2 or 3 thousand chickens running at 30 fps, but putting 1 each of 200 animals would slow it to a crawl. Animated objects (mostly animals) are tons more expensive than trees or crops.

I know at one point dogs that were trying to get to the player but were blocked off was hugely expensive, but I'm pretty sure that got fixed.

Yudhisthir6 karma

Were you self taught or go to a college? What was your learning "path", as in what technology/language you learned first, then next etc? Thanks for doing the AMA.

LagrangePt14 karma

answered most of this in other replys

I went to Upenn, majored in digital media design.

Started with Logowriter in 5th grade, dropped it after that summer.

In high school I got into Maya and a bit of lego robot programming. In college, we started with Java, then went to c++ (with MEL and Python thrown in later). Before senior year I interned at a defense contractor working on AR. In senior year I made a simple game in Gamebryo (horrible experience), worked on a pathfinding algorithm in CUDA (that eventually got published as a research paper, woot) and a bunch of other projects. Worked as a research assistant that summer to get the paper finished and help with some drone navigation and pathfinding, then got the job at zynga. See other posts for the rest

Yudhisthir5 karma

That is pretty awesome. I am reading other answers now. But, here's a potentially dumb question: what do you think a self taught programmer needs to know when entering the industry? Like, apart from the language's (Java/C++) syntax, does basic knowledge of Algorithms and Data structures suffices, or there's something more?

LagrangePt25 karma

aside from a general knowledge of c style syntax, knowing the detailed syntax of a language is almost worthless (unless you're selling yourself as a language expert, which takes a lot more than syntax knowledge).

Learning a new language when you start a new job is standard, and it's pretty easy if you've been exposed to a few different languages. I'd say you should do at least a couple projects in a low level language (c, or even an assembly simulator), a couple projects in java, a couple projects in c++, and a couple projects in high level languages like javascript, python, etc. It's more important to be able to rapidly become functional in a language and to write solid pseudocode solutions to problems.

I've seen self-taught programmers be functional and get paid without really knowing algorithms, data structures, and a lot else. I've spent a lot of time cleaning up their mistakes, but they're often a lot faster than me at just getting a working product. I've also seen that programmers who really understand the entire stack are pretty rare and hugely valuable, and often get promoted past their peers pretty quickly.

When I was doing job interviews, I would expect a definite must have pay him tons candidate to understand the entire stack from transistors up to pixels on the screen, including the different layers of code in between. They don't have to be able to do anything with every layer, just basically understand them. I would also expect people to really understand Big O complexity notation

knowing algorithms exist and being able know when to apply them, be able to look them up and implement them, as well as know how much they'll cost is important.

knowing how memory is managed underneath the language you use (garbage collector? manual? something else?) is also important.

etcpie6 karma

[deleted]

LagrangePt17 karma

I had never looked at AS3 when I started the job :)

It's a lot like c++ with all the hard parts removed, and as long as your algorithms, data structures, and low level understanding of CS is good, you can pick up AS3 and be awesome at it very quickly. As I posted above - in the morning I had never seen AS3, by that evening I had written a pixel perfect mouse click detection system and released it to 10 million people.

If you don't have a solid programming background, I'm probably not the best person to ask (since I went through basically a full CS degree to get that background). I'd say dive into something easy and write small programs that do something. You may need to follow a book course or something, but make sure you understand common algorithms, data structures, and other things that high level languages handle for you, as not understanding what that magic is doing is the most common source of stupid mistakes and frustrations I've seen

phearmymind5 karma

How dependent are companies like that on in game purchases? Are things programmed specifically so someone can't really advance without buying things?

LagrangePt17 karma

something like 80-90% of the revenue comes from in game purchases. (maybe lower as the advertising and partnership deals came into play).

The game designers typically argue for a fun game without spending a dime, while the business people argue for making the real money purchases as tempting as possible. Usually, they come to a balance where money buys you premium vanity items and lets you cheat a bit (skip past grinds, get that one last item you're having trouble collecting, skip past a quest, etc.)

Sometimes they reach a balance where the game is actually fun for a while, and sometimes they squeeze your balls.

MatureAgeStuden5 karma

I am a bit of a gamer (STALKER, FTL and Civ 5 recently) and also enjoy so called 'casual' games like Mr Crab and Triple Town. Yet I have never played a Zynga game or Facebook game. What am I missing?

LagrangePt11 karma

I haven't played any Zynga games since I left. I know that playing a lot of those games when they were first coming out was actually kinda fun, and they were very good and hooking you in and getting you to care about a stupid virtual farm / mafia / fish tank.

Some of their games were actually pretty fun, but expect to be squeezed for money and friend adds in any of the games that are still updating.

I'm a dedicated hard core gamer tho, so my perspective is a bit off.

ohmyshit5 karma

Wow, your previous AMA was fucking horrible. Top question: "Y U NO?". Jesus Christ.

Thanks for willingly attempting another AMA. Interesting stuff.

LagrangePt3 karma

yeah, there's a lot of blind zynga hate around here

Getdownlikesyndrome4 karma

Just wondering, what's Satan's place like inside? Nice condo?

LagrangePt8 karma

The original building was kinda sketchy had really bad acoustics (they had to buy the entire team headphones when we moved to a larger room cause it echoed so badly). On the other hand, lots of snacks, drinks, and good meals for free. Also free massages and haircuts if you scheduled far enough in advance.

There were dogs in every studio which is awesome for a dog lover like me.

The newer space was snazzier, had a bunch of arcade machines scattered around, and took longer to walk to any given studio. OTOH, I sat right near the cafeteria, we still had awesome dogs, and we had some themed snack kitchens that gave you the excuse to walk across the building as a break.

Also, monthly poker tournaments with awesome prizes (at least at first). At one point someone won a rental ferrari for a weekend (may have been a lambo, not sure).

MonsterTea8 karma

a bit off topic, did each worker take the dog out to do its business?

LagrangePt6 karma

yep, though there were occasional messes that the dog owners had to take care of.

krilnon3 karma

Did you ever notice that many of the AS3 vs. Haxe benchmarks that people posted on the web were really sloppy? (You mentioned that improving performance was part of your job, so I imagine you would have run across these things.)

LagrangePt3 karma

I didn't pay that much attention to Haxe - for various reasons I don't remember it wasn't practical for us to use it, even if several of our devs argued that we should. I do remember a great many sloppy benchmarks made by language fanbois of various types tho. I do remember jackson dunstan being really useful:http://jacksondunstan.com

Runeite513 karma

[deleted]

LagrangePt11 karma

um, after tax I think it came out to 2 to 3 thousand every 2 weeks, plus occasional bonuses.

I worked at a mobile gaming startup for a year after I left zynga, so yes. We would have been profitable had it been just the games, but we were also working on longer term projects with other engineers.

Unity is super easy to pick up, UDK can create better looking games but can't target as many platforms.

If you're looking at mobile games, you may want to look into cocos2dx with js bindings. if you know JS, should be easy enough for you to work in, and the JS scripts are mostly compatible with cocos2dHTML5.

herpderpherpderp3 karma

can you post some proof in your description please?

LagrangePt3 karma

added some

grant02 karma

Proof please? All AMAs require proof.

LagrangePt5 karma

added that in to OP

taramasu0 karma

help me! how can i better this? http://yaynoway.weebly.com/

LagrangePt3 karma

not at all sure what you mean? what are you trying to improve/ change?

Keep in mind I'm not at all a web designer

taramasu0 karma

what do you think i should add? is it stupid already?

LagrangePt2 karma

Well I have no idea what it's for or what you're trying to do with it. Seems like a blog where you're asking people to comment on stuff? Why would people bother posting there instead of just posting to reddit or creating their own blog?

taramasu0 karma

well im trying to make it to where people can create there own blogs. or post on blogs that are already made...

LagrangePt3 karma

Is it an original blog system? I thought I was just looking at a standard create-your-own blog site that you have made an account on.

If it's original, you need to ask yourself some fundamental questions:

1) why would someone use this over something that's proven / already out there

2) will existing sites just copy any new ideas you have

taramasu0 karma

yes. im trying to create my own blog-site type deal.. and, if i do think of really original things, is it a good thing if i think other sites will copy it?

LagrangePt2 karma

It's possible to displace industry leaders, but it takes a really solid idea, lots of hard work, and you still might get screwed.

You should read up on the stories / biographies of people that have been successful at displacing industry leaders.

Even if you don't displace the leaders, you can still grab a chunk of business if you offer a solid product and treat your customers well. I don't use many blog sites, so I can't really comment on standard features.

Chiner-3 karma

How do you sleep at night?

LagrangePt11 karma

why should I have trouble sleeping? for all that zynga games annoyed a lot of people, they were also enjoyed by a lot of people.

I was also primarily involved in fixing the game when it broke and in fixing bugs & performance problems when it wasn't broke, so I wasn't really involved with some of the stuff I now know about and find distasteful. I had a moment where I was testing out Farmville 2 and saw a certain feature, declared "that's bullshit", wrote an (sorta angry) email to the CEO, and turned in my 2 weeks notice a couple weeks later.