How Avatar: The Way of Water's VFX Were Made
Director: Funmi Sunmonu
Director of Photography: AJ Young
Editor(s): Sammy Cortino
Talent: Dan Barrett
Producer: Ashley Hall
Line Producer: Jen Santos
Associate Producer: Emebeit Beyene
Production Manager: Andressa Pelachi
Production Coordinator: Peter Brunette, Kevin Balash
Director of Talent: Lauren Mendoza
Camera Operator(s): Lucas Vilicich
Sound: Paul Cornett
Production Assistant(s): John Brodsky, Gee Depratt
Art Department: Leah Waters
Post Production Supervisor: Edward Taylor
Post Production Coordinator: Andrea Farr
Supervising Editor: Kameron Key VF
Assistant Editor: Billy Ward
Released on 03/03/2023
Facial animation for an animator, even with our tool set
would be a good three to four weeks.
If we don't get that right, then simulations break.
I'm Dan Barrett, and this is how we created
The Na'vi and the creatures in Avatar: The Way of Water.
Any animator before they start a shot,
they give us the ground and we're gonna animate
on that ground and you're not gonna change it.
But obviously with water being liquid,
we start with something that we think is gonna be, you know,
the correct surface, the thing
that we can reliably animate to
but things can change very quickly,
or it's really critical
that any interactions with a character
or a boat or a creature with the water,
it's really critical that we get the buoyancy right,
that we are inferring mass correctly.
We did have tools that would simulate
to the water, but yeah,
it's a really tricky thing.
It really did present challenges, especially that interface.
I think underwater, not so bad.
Obviously above water, you're okay,
but as soon as you get a creature or a character
perhaps even falling into the water,
there's a lot of work that needs to be done.
A lot of simulation, a lot of back
and forward between the effects team and the animation team.
Water is, I think, 800 times denser than air.
So the way that something travels
through the air versus what happens
to the travel of that object when it hits the water,
it's something that I think viewers are used to seeing,
If we don't get that right, then simulations break.
So, that's a real challenge.
You can also see that there are, you know, actors
are sitting on bucks and we are capturing their performances
and there are people
moving those bucks to try to infer the motion
that ultimately is gonna be seen in the shot.
But, really when it comes
down to it and the shots being created,
the creatures are gonna kind of do what
Jim may want 'em to do later.
So, one of the challenges is taking those performances
and then making them work on
a different bit of motion.
We've captured it on a buck that's doing this
and Jim may want it to do this.
He'd still want that performance.
So, the animators have to very carefully
use their skills to take that performance,
preserve that performance
but make sure that the body
of that character makes sense physically on the creature.
There's a bunch of work to be done there for the team.
[Actor] Woo. Woo.
Fun shot for an animator to work on.
And the Payakan's performance is based
on reference that we had of Humpback Wales breaching.
There was some performance capture here, but a lot
of this is the animator's work on what Low Act does.
What you're seeing in the breakdown is you're
seeing the various simulation engines.
Water's a really complicated thing.
There are sort
of multiple states that we identify.
The large mass of water when it's disturbed by a whale
and then you get a sort
of a smaller spray and then through to the little droplets.
So, what you can see here is it broken
up into those component parts.
These simulations, they take an awfully long time
and having them broken up,
it's slightly easy to have control over
the way that
the simulations would look.
The most challenging thing to animate in Avatar,
the thing that we consider to have
the most importance are the facial performances.
This scene here was one of the first that we worked on
but this was the first time we really tested
out the new system.
The face cameras that are sitting
on the booms, on their helmets.
There are two cameras, whereas
in the past we had a single camera.
The two cameras gave us the ability
to recreate the actor's faces in a 3D environment.
So that's what those cameras gave us the ability to do,
to reconstruct the faces.
And we used to use a thing called blend shapes
which was based on the surface of the face,
and we'd look at the face
and we'd infer what muscles were being fired.
But what we would do is create these shapes
in the face and we cared more about
the surface rather than the interior,
the muscles of the face.
In the new system, we had
a database of the expressions that these actors
were capable of or were most likely to use during the film.
We gathered those expressions in sessions that Jim had
with the actors where he sat them down
in a booth where they were facing an array of cameras
and he went through their entire role
in the film with them as they sat there.
Amazing to see someone like Zoe go through the emotional arc
of her performance in the film
all within a two hour session
in a chair.
That gave us all of the expressions
that she was most likely to use in the film.
And the new facial system took those expressions
and used them as a database
that it would search when it received the performance
from stage, which is what we are seeing here.
You can see these performances
and you can see how subtle they are.
There's so much range and so many of the performances
in this film, the new system did a very, very good job
of capturing all of that nuance
and giving the animators a fantastic starting point.
Animators would then look at the reference cameras
that we're seeing here.
In the reference cameras,
you can really read emotion much more easily
than you can in a face cam.
So the animators would use
these reference cameras to get a sense of that emotion
and make sure that we are getting all of that subtlety.
And a lot of that is in the body as well.
With facial animation, it's as much about the head
as it is about the lips or the eyes.
I remember it took us a long time to get this shot of Jake
where it really just wasn't reading that well.
It was lacking so much in of the intensity
that Sam had in his performance.
And it took us a few goes looking at it
before we realized that his head was just a little off axis.
It was a couple of degrees off at most,
but it broke the entire shot
and it took us a wee while to figure that one out.
[Speaker] Avatars only go around [character squeaks]
[Avatar #1] Whoa,
[Avatar #2] Whoa, bro.
[Avatar #3] Uh huh.
[Avatar #4] Go monkey boy.
The performance capture suits that they wear,
they have dots on them and these dots can
either be reflectorized or they can be active,
so sometimes you'll use an active LED.
In this case, these are infrared
and these are passive reflectors that capture
all of the body data.
In that virtual space,
what you can see is, we'll also scan all the actors.
So, you know, we'll have a Sigourney scan.
We'll have a model of Sigourney
that has her exact proportions.
Those dots you see in that virtual space there,
they sit in the same world space
as you see those dots on the suit that Sigourney is wearing.
So, that's something that we use during
the solving process just to make sure the data has come
through cleanly and we'll take that then from there
and we'll push it through onto the characters.
10 seconds of facial animation for an animator,
even with our tool set would be a good
three to four weeks of careful animation.
And then beyond that,
there's everything else that needs to go into that shot.
There's simulation for clothes and hair,
the environments, the lighting, the renderings.
A shot or a sequence of that length,
you would expect to see that going through the pipeline
for a good six to eight months.
The system that we've developed
at Weta is called the APFS.
That is an acronym for anatomically plausible facial system.
Ropes. Ropes are hard and tails are hard.
We had to animate about 4,000 tails,
at least 4,000 tails on on Avatar: The Way of Water.
All of the Navi have tails.
The Kaya people, the forest people with
the more standard cattail
and then the Metkayina with their crocodile tails.
And so, the tales are always a particular challenge
So what does her heartbeat sound like?
A lot of the success of the production schedule
was all the planning that took place
with the team at Lightstorm before shooting began.
And one of the interesting things
about the way that this film was shot is Jim kind
of shot it twice.
He shot it with
all actors in their motion capture shoots
with the performance capture side of things.
And then a lot of it he would shoot again
in live action.
In the past, you might have someone like maybe me
on set who was hopefully holding a tennis ball
in the right spot
but the number of times that fails
and you are having to fix things,
nothing worse for a compositer here
than can you just change that eye line,
can you change those eyes on the plate?
It's a really hard thing to do.
Yeah, it was a real success.
My favorite creation, I think would be,
I like Quaritch.
I think the way that Stephen Lang played Quaritch,
I think that he's such a great character
and I think we see a little bit
of nuance in his character in this film.
[Producer] Only two shots
in the entire film don't contain visual effects.
Can you share with us which two shots
in the movie don't have visual effects?
Those shots didn't come across my desk,
so I haven't seen those shots.
Bullet Train Director David Leitch Breaks Down Action Sequences from His Movies
'Stranger Things' Auditions and How the Cast Landed Their Roles
How 'Elvis' Costume Designer Catherine Martin Transformed Austin Butler Into Elvis
How Stranger Things' SFX Artists Created Vecna
How 'Spider-Man: No Way Home' Visual Effects Were Made
How 'Dune' Composer Hans Zimmer Created the Oscar-Winning Score
How Tattoos Are Designed For Movies & TV
How 'The Lion King' Has Captivated Broadway For 25 Years
How Stop-Motion Animators Created Guillermo del Toro's Pinocchio
How Avatar: The Way of Water's VFX Were Made
How 'Asteroid City' Production Designer Creates the Worlds of Wes Anderson
'Jury Duty' Auditions and How the Cast Landed Their Roles
How 'Wednesday' Costume Designer Created Jenna Ortega's Looks
How 'The Last of Us' SFX Artists Created the Infected