I am Joe Letteri, senior visual effects supervisor and Director of the Academy Award-winning Weta Digital studio. You might know my work from LOTR, Avatar, The Hobbit, King Kong or both Planet of the Apes - AMA!

Hi Mr. Letteri! Thanks for taking the time to do this AMA.

How would you describe the difference (in a graphic/visual effects artists mind) between how work was done in Jurassic Park and Dawn of the Planet of the Apes? Was it more challenging back then or is it more challenging now?

Also, did you get to keep anything cool from your work with the Jurassic Park crew?

Thanks again for doing this!

JURASSIC PARK was really the first time we created a character using digital techniques & computer graphics, so it was pretty new to us. Hopefully we got it right and it still holds up! Today, we do a lot more detailed character work, especially with dialogue & facial expressions & performance capture, and there is a lot more realism that we know how to bring to these characters. The computers back then had less power than your iPhone does, and the computers now have a lot more but the images are so complicated that it takes weeks to render each frame of film you see rendered onscreen.

In JURASSIC PARK, we used keyframe animation, there was no motion capture. And we really didn't have that many shots of digital dinosaurs in the film, there were only about 65. On DAWN OF THE PLANET OF THE APES, we had hundreds of apes in many of the shots, and 1200 shots in the film.

I got to keep a little plastic T-rex? And some t-shirts. That's about it.

Oh yeah and is there any chance I could get a hold of one of those t-shirts?? Haha

Hehehehe! I'd have to find them, they're in a closet somewhere. But no. I'm hanging onto them.

given that the effects house that did life of pi went out of business, how do you see the industry being sustainable as a whole? Do you think effects work may be pushed to the more hobbyist realms as computers become more affordable/powerful and the money dries up?

There's still a need to understand how to use FX to tell stories, and there's always work that has to be done to figure out new FX and new things that haven't been done before, and especially for us at Weta, our interest is focused a lot on character animation, and that requires a lot of specialization and quite a large team working together to achieve all the aspects you need to create a character. Besides the motion capture and the animation, you have to understand how the skeleton works, how the muscles work, how the skin works, how the eyes work, how the cinematography works - there are so many aspects to it that it requires a large, dedicated team to do it, and I think that will be the case for a little while yet.

what's new on the cutting edge that you think won't be available to the hobbyists for a while in the form of plugins? what VFX techniques do you think could transfer to other industries?

Ooh, good question.

We write a lot of our own software, because of the scope & scale of what we need to do. So generally a lot of what we do is not available as plugins, and it would be up to other people to bring them to the level where people could use them at home, or on the hobby level.

Most of the ones that transfer to other industries by and large seem to be image-related? There's a lot of crossover between imaging technology and medical imaging - for learning about properties of skin, and hair, for example. So there's some crossover there, mostly with biology and medical imaging.

How is Peter Jackson like in person?

Peter's great in person. He's really collaborative, really open to ideas. And he's got a LOT of energy. And I think one of the things that motivates Peter, which I think applies to everyone at Weta, is that you don't give up on the film until you have put every ounce of effort you possibly can into it, to try to make it as best as possible before it goes out the door.

Hi Joe!

Congrats on another Academy Award nomination.

If you could name and one VFX shot or sequence of shots from any movie as your favourite, what would that be?

You know, it's always hard to think back to where you are - the most recent one is always the one that sticks out in your mind, and in DAWN OF THE PLANET OF THE APES the one that sticks out is where our director Matt Reeves had to push the camera really close, right into Caesar's eyes, because it gives you a sense of what Caesar is thinking, and it allows you to think about what is going to be happening in future stories.

What's your role in Welta Digital ?

My title is "Senior visual effects supervisor" - so I am responsible for all of the creative, artistic and technical direction of the company. That entails looking at new projects, trying to determine what we want the FX to be (in other words, what will the characters look like and how will they perform) and what new science or technology do we need to develop to bring them to life onscreen.

It's a good job.

What's your favorite pre-cgi movie? Have there been any movies that you thought, "Wow, I wish we thought of that?"

There's 2 that really stand out, pre-CGI - they would have to be 2001 and STAR WARS. Because 2001 was just visually stunning, with its portrayal of space and how you could use FX to take you INTO space, and STAR WARS just applied that with a whole idea of an action movie sequence that just really opened up a whole world of storytelling.

What's a great way for young aspiring film makers to network and develop useful connections?

MAKE FILMS.

The best thing you can do is work with other filmmakers and find out which part of it you enjoy the most, and how that fits in with how everybody else likes to do their part. It's almost like forming a band - you want to try out as many things as possible, and see what clicks.

What are your future projects?

Future projects? Well, right now we are working on the second Maze Runner film, with FOX. We have the extended cut of HOBBIT going on right now. We have a film called SPECTRAL that we're working on. And we're doing pieces of other films as well, but those are probably the ones that you would see coming up the most immediately.

Hi Joe,

Pardon my ignorance but this is actually a perfectly timed AMA since I was watching Titanic yesterday and marveling on how far we've come in FX since then.

So say for a crowded battle scene like you'd see in Battle of the Five Armies or the LOTR movies. How many actual actors are there in a shot? And how responsive are the virtual actors to one another?

Hope this question makes sense. Thanks!

Mmhmm, yes! There are 3 ways that we create characters for battle scenes:

• One is live-action characters - so you would have, for example, in BATTLE, you would have your main characters shot on a green screen, probably no more than 5-6 at a time, because you tend to focus on the main action that you need, knowing that you're going to fill in details later
• As we did on DAWN OF THE PLANET OF THE APES, we used performance capture and we focus on the main actors in the same way - for example, in the scene at the end, the fight between Caesar and Koba at the end on the tower, that's choreographed with just the 2 main actors in mind, but on this case it's done on a virtual stage with a virtual camera (which is a technique we developed for AVATAR)
• To fill out the background (and that can be a combination of either more motion-capture or some automatically generated animation) and we mix the two together, depending on how many characters we need for the background - for example, in DAWN OF THE PLANET OF THE APES, we would have about 1,000 apes in the battle scenes, with some of those on horseback

So usually a mixture of those 3.

We have a software package that we wrote, back on LOTR, called Massive, that will allow armies of characters and each of the agents has a brain, eyes and ears, so they can respond to the action of the other agents around them, automatically. So it's like choreographing a crowd of extras. If we want more specific behavior in our crowds, our animators will use motion capture and animation to create exactly what we want, in the moments that we want, and then use animation to blend and connect all the actions together.

This is cool! Mr. Letteri, thanks for doing this! I see the credits at the end of films and there are so many people involved! I imagine it's a complicated process to get just a single shot sometimes- involving everyone from storyboard artists and actors to VFX artists and directors. Say there was a shot planned of Caesar holding a gun in the air (or something). Could you walk through the process that takes place for that to appear on screen?

Sure.

Caesar holding a gun in the air?

So there's 2 streams that need to happen. One is we have to design & build our Caesar model in the computer, so we have to get all the facial expressions working, we have to get all the skin & muscles working, we have to get all the fur working, we also have to understand all the material properties of everything - fur, eyes, teeth, fingernails, skin - and we have to understand how to apply lighting in a physically correct way, so we can use a computer to essentially digitally photograph him.

So that's the preliminary work.

Then, onstage, we have an actor (in this case, it would be Andy Serkis) in a performance capture suit, and he would be working with our director Matt Reeves, and they would be filming the scene, with our Director of Photography Michael Seresin, and they would be filming the scene as if everything in it were real. So Matt would be directing Andy, Andy would be holding a prop gun, and Michael would be photographing him and lighting him as if everything were real. Then we record all of Andy's motion, and we also record all the physical attributes of the set - we scan the set he's recording on, we rebuild it, we record all of MIchael's lighting, and we re-create all of it in the computer.

Then we apply the animation, and we re-light the scene, and we re-photograph it digitally, and we render it out on the computer, and then we take our computer elements, and we composite them in with the live action photography, paint out anything that shouldn't be there (like Andy in his performance capture suit), and once you put everything together, you have the final film image.

It's a long process, because a lot of the building of the characters - that preliminary process - takes months before you even start to film anything. Once you have things filmed, though, it's generally 3-5 months for every shot.

What visual effect that your team was not a part of do you find most impressive?

There's a lot of great work out there. One of the scenes that stands out to me was the Kitchen Quicksilver scene in X-MEN: DAYS OF FUTURE PAST because it was such a great use of visual FX to tell a story in a unique way.

Hi Joe! Thanks for taking time out of your day to do this. I have a few short questions. 1) What's your favorite type of pudding? 2) What is the best life lesson you have ever learned? 3) Who is your role model? Thanks again!

1.) Chocolate!

2.) Best life lesson I've ever learned? Hmm. Probably not to worry if something hasn't done before, just dig in and figure out how to do it.

3.) Ooh, that's a good question. You know, I'd probably have to say if I really thought about it, it would be somebody...like Galileo. Somebody who really saw something for the first time that nobody had seen before, and had to try to figure out what it meant in the face of overwhelming opposition saying "what you see can't be real."

If You would have chosen a different carrer patch what would it have been?

If I hadn't gone into VFX, I probably would have gone into sciences - probably some sort of physics or math.

are there any obscure movies that people in the industry regard as hidden classics? for example, the specular reflection effects in 'flight of the navigator' holds up pretty well (a lot better than the CG in lawnmower man anyway)

chuckles

Hmmm. Boy, I wish I had an answer for you. Can't think of anything. Maybe we'll come back to this!

Hi Joe! Congrats on NOT being a part of 50 shades of grey! How does it feel to produce content that actually contributes to society?

Well, I...have always been motivated by looking at scripts, and wanting to work on films where I felt the scripts could actually take you somewhere. And I've been very fortunate to do that. I remember when Jim Cameron sent me the script for AVATAR, and asked me if I'd be interested in working on it. And I was just so enthralled with the possibilities that it just was really worth the effort to figure out what we had to do to create a whole world like that.

And I've been very fortunate also with the PLANET OF THE APES series, because i love the original films, and what i loved about them is by telling the story in the form of apes, you really got to look at these problems of society in a new way. And it was actually John Kilkenny at FOX who sent me the script for RISE OF THE PLANET OF THE APES, saying that "we're interested in re-invigorating this series," and the first film had such an interesting story to tell - you know, about a scientist who is not evil, who is actually very benign and very caring, and through his best efforts to help save his father, unleashes this virus that essentially starts to wipe out humanity. But at the same time, he fostered this chimp, Caesar, who becomes intelligent and starts to carry on what we think of as the best qualities of humanity. And so I thought this was a great framework for telling stories that matter.

And I'm hoping that we get the chance to do more of them.

You mentioned that gollum from lotr and hobbit is your favourite character; how did the idea come about to use an actor for the portrayal of gollum rather than use cgi? And what did you think of Andy Serkis as gollum? I thought he was amazing, really brought the character to life!

Well, Gollum is a CGI character. What happened is that Andy was brought in, to record Gollum's voice. But Andy, being an actor himself, worked with Peter to come up with this idea of having him perform in front of the camera with the other actors.

And that brought a whole new level of drama to the performance, because that meant that the actors could work with Andy as if Gollum were really there with them.

And so we then recorded Andy's performance, and used that as a guide for creating the Gollum character, on top of the performance he gave on the set.

Now, when we did RISE OF THE PLANET OF THE APES, we made a technical breakthrough where we could use the performance capture at the same time that Andy was performing onset.

And so that really allowed us then to do Gollum the same way for Hobbit, which was fantastic because it really kept Andy's performance in the moment. And then we were able to take that one step further with DAWN OF THE PLANET OF THE APES, because then we were able to take all the recording gear to a remote location. So it's given us the ability to work with actors to create these characters anywhere in the world.

And obviously, a huge fan of Andy. He's a fantastic actor, and we've been fortunate to work with him all these years.

What is your favorite movie that has bad visual effects?

That's a good question. I'm not sure. Visual FX aren't that bad nowadays!

Congrats on the Oscar nomination! This is your 9th Oscar nomination and you've won 4, how does it feel? And what will you be wearing on the red carpet?

It feels great, every time! It's not the kind of thing that ever gets old.

And I'll be wearing a tux. The same as every year.

Armani tux, if you must know the designer.

What was your first job?

Well, my first job was creating computer graphics - they were essentially logos - for television stations and things like that, and for commercials. That's how computer graphics were being used a lot in the early days.

How was your experience working on the hobbit:battle of the five armies movie?

The experience working on those films was great, because we got to really spend another 3 years in Middle Earth, which is a fantastically open environment for us. We had the ability to not only walk through this fantastic landscape of Middle Earth, but help to create large parts of it, and as the series went on, we got to create more and more interesting characters. So it was great in the first film, to be able to bring Gollum back again, because he's a favorite character of mine. It was great to be able to create Smaug, because he's such a fantastic character. And, in the end, we created about 130 different characters for those 3 films.

what inspires you?

What inspires me is the idea that you can create new characters that allow you to tell stories in unique ways. So for example if you look at DAWN OF THE PLANET OF THE APES, it's a story of how fear leads people to war. But by being able to tell the story through the eyes of apes, it allows you to see it in a new light and to distance yourself from your existing prejudices. And so it lets you see the story in a new way. And so in order to make that happen, in order to create it, we have to put quite a lot of effort into learning about the cinematic techniques of storytelling - like lighting, and animation - but you also need to understand the underlying science that makes a character come alive.

Storytelling is ultimately what inspires me. That's what leads us to look at creating specific visual FX, and that's what leads us into putting effort into one particular aspect versus another. It all depends on what the film needs to tell the story.

Hello!

How did you get started in vfx? What motivated you? Was it hard to learn at first with the old technology and all?

Thanks for doing this AMA.

What got me interested in VFX was really computer graphics. Because I was studying math, and just got interested in the idea of using computers to understand difficult mathematic concepts. One of the concepts that was new at the time was fractals - trying to understand the kind of math that allows you to create natural shapes like clouds and mountains. And because i was working on how to do clouds, when I got studied on ILM, was the opening shot of STAR TREK 6, where I got to blow up the Klingon Moon Praxis with the "ring of fire" explosion. And then the next film I got to work on was JURASSIC PARK, and I learned how you could apply some of those same techniques to creating realistic looking dinosaurs. And from there, I started learning more and more about filmmaking, and characters in animation, and just kept applying what I learned as I went.

Where do you see yourself 5 years from now?

Boy, that's a good question. You know, I actually don't ever think that far ahead? Although let me just say, though, it's sort of a similar answer that i gave before - it all depends on the stories, and we get such great stories sent to us that that influences where we go and really what the future holds for us.

Thanks for taking time for this!

1) What had been your toughest challenges so far in the industry? If it is in the execution side, could you describe the technical aspect of how you and your team solved it?

2) What question are you secretly hoping to be asked? :D

1.) Well, in general the toughest challenges have been creating characters, because you have to be able to make them dramatically perform and hold up to real actors in a scene, and then you also have the added challenge of how do you make them look real. And I think for me Gollum was the breakthrough there, because not only did we come up with the idea of using performance capture and working with Andy Serkis, but we also created a technique called "subsurface scattering" that allowed us to create realistic skin for the first time. And the combination of those elements together really gave us a character like Gollum, who was believably alive. And that's been the basis for all the characters that we've created since then. We've learned how to do it better and better, but if you look at Caesar in DAWN OF THE PLANET OF THE APES, those same principles are there - we've just refined them and used them in greater detail.

2.) Don't have any in particular! Do you have any Victoria?

Hello Joe! Welcome to reddit, my question; do you think that no matter how good cgi looks on film, will it always look dated at some point in the future or do you think we will eventually reach a point where we won't be able to tell what's real or what's not on film?

I think we'll be able to reach a point where we won't be able to tell what's real and what's not. You will hit the limit of what you need every pixel to look like to do exactly what it would have done if you photographed that character. You know, as an example, we had some chimpanzees in the first DAWN at the beginning of the film where many people came up to me and thought we had photographed real chimps - but there were no real chimpanzees used in either of the PLANET OF THE APES films. They were all digital characters. For some kinds of characters, and some kinds of FX, I think we are already there - I think that even for experts in the business, it's sometimes hard to tell what's real and what's not.

My name is Lloyd Turner. What are your favorite hobbies?

P.S. I am a huge fan

I'm interested in astronomy and have been ever since I was a kid, so that's probably where I spend most of my time when I'm not doing things directly related to computer graphics.

Hey Joe! Is there any past movies you wished you could have been a part of? Or any upcoming movies you would like to be part of? (Star Wars, Star Trek etc).

You know, I was always a huge fan of STAR WARS, I go to work on one of the re-releases, so I don't feel like I have to go back in history. And as far as future STAR WARS, I'm really looking forward to it. And in this business, it's hard to take a fresh look at a film when you work on it, so what I'm really looking forward to seeing STAR WARS in the cinema knowing nothing about it.

Have you worked on any Schwarzenegger films? The producers of this year's The Legend of Conan say they are big Weta fans. Would you like to be involved with Arnold's next king Conan film? Are you a fan of this fantasy character?

Hahaha yes I was a fan of Conan the first time around, but no, I've never had the chance to work with Arnold (or Governor Schwarzenegger, I should say).

What got you into animation?

I was actually interested in computer graphics first - how do you use computers to make realistic looking images. One of the first films I got to work on was JURASSIC PARK, which also required learning about creatures and about animation, so I became very interested in character work, and so I got to create Gollum for LORD OF THE RINGS.

And so for me, that was my breakthrough moment, where I started to learn how to turn characters into actors.

[deleted]

I'm not on Twitter.