Avengers: Endgame VFX Breakdown

ILM, Weta Digital, DNEG, Cinesite, Framestore, Digital Domain, Rise VFX, Lola VFX, Cantina Creative, Capital T, Technicolor VFX & Territory Studio worked on VFX for Avengers: Endgame

Josh Brolin as part the Masquerade pipeline at Digital Domain


ILM created the Smart Hulk character using new technology (Framestore used this model) ILM team responsible for 80% of Smart Hulk shots. Overall, ILM did 540 shots in the film Include half of the End Battle, the new time travel suits, Iron Man, Iron Patriot, Rescue, New Asgard, the Quantum Realm, the New York sequence with the the Hulks, Ant-Man, and Strange’s magic (the water tornado and Winds of Watoomb).

ILM, Weta Digital and Digital Domain worked on the complex sequence in the movie. Digital Domain and Weta Digital provided the complex and subtle performances of Thanos in Endgame. Digital Domain maintained much of the same pipeline as they had with Avengers: Infinity War, but with more Machine Learning.

Weta Digital did most of the final act, with the exception of Captain Marvel destroying the Alien ships, and the water, dam magic sequence.

Framestore delivered more than 300 shots for the movie. Framestore was tasked with breathing life into a ‘Smart Hulk,’ a never-seen-before Bruce Banner / Hulk hybrid. Balancing the human performance of Banner and the seismic power of the Hulk required a deft mix of cutting-edge tech and Framestore’s world-renowned animation, and the result is a striking fully-CG character whose unique performance is loaded with human nuance, emotion and comedy. To kick-start this process, a small team at Framestore were tasked with developing their machine-learning technology to take source footage of Mark Ruffalo and create a solve from that to match it in animation; ‘it was a way of convincing the client that we had the software that could do this quickly,’ explains Robert Allman, CG Supervisor. The result was a first pass of animation that could be used by the filmmakers to decide on the edit and begin the process of refining and adding face-shapes to use in the rig. Keyframe animation was then deployed for over 60 Smart Hulk shots, with head cam footage from the shoot used for reference. This performance was key for the animators on the film, ‘it was absolutely about channeling Bruce Banner,’ explains Max Solomon, Animation Supervisor. ‘His face is very expressive. Every nuance, every little twitch and eye dart, slight curl of the lip and muscle tension needed to be translated.’

Smart Hulk’s facial build was particularly challenging, as VFX Supervisor Stuart Penn explains: ‘There was a lot of work involved as their face shapes are so different – Hulk is massive, his mouth proportionally bigger, his eyes are sunken with deeper sockets. Tiny changes to his face had a huge effect on the performance.’ A new subset of facial shapes was built to enable the animators to achieve more complex and nuanced expression and give them more control in the animation.

The studio were also tasked with animating Rocket. The wise-talking raccoon returned to Framestore, who originated his character for Marvel Studios’ Guardians of the Galaxy, before upgrading him for Vol. 2. A little greyer around the muzzle, he required careful tweaking. Says Allman, ‘changes had to be subtle as small tweaks could make him look like a completely different character or even animal.’ Rocket was treated to a new flight suit, complete with a dapper red scarf and goggles, ‘there was a lot of detail to the costume,’ says Penn, ‘building the threads, adding a fine groom, right down to the texture of the bobbles of his scarf.’

The Avengers wear Quantum suits made of high-tech armoured cloth and carbon fibre which they don to travel through the quantum realm. The actors had been shot in their individual outfits, so Framestore built digi-doubles before replacing the costumes on the ten characters made up of an array of body types and proportions; ranging from Rocket and Black Widow to Nebula and Hulk. Based on concepts provided by Marvel Studios, the artists built a prototype suit for Captain America that was then adapted for each Avenger. These suits then had to manifest over their original outfits using the nanobot technology Framestore developed for Iron Man’s bleeding-edge Infinity War suit. Their helmets, based on Ant Man’s, also needed to be built in CG and manifest in the same way over the actor’s face. ‘The helmet is a snug but mechanical design,’ adds Allman. ‘We used the nanobot tech to give the impression that it comes out of nowhere.’

The team also had the chance to use their creativity to create a series of mesmerising quantum effects. Ant Man’s ‘Quantum van,’ which first features at the end of Marvel Studios’ Ant-Man and The Wasp, is revisited in Endgame as the Avengers test how it works. Says Penn, ‘we had to match the original effect and then amp it up as there’s a lot more going on; we had a lot more shots and see it from different angles.’ Framestore built the whole model; based on scans from the original, combining comp, lighting and FX to achieve the look. 2D LookDev Lead Tomas Lefebve used Nuke to get as close to the original look as possible before breaking it apart to decide on which elements needed 3D FX. Says Enrik Pavdeja, Compositing Supervisor, ‘we worked up lots of optical FX in compositing; heat distortions, colour breakup, aberrations and even our own blue plasma. We passed these pre-comps to lighting to create the interactive quantum plasma before FX.’

This effect is magnified with the build of the Quantum Gate, the Avengers’ time travel portal that features in 22 spellbinding shots. ‘We had a lot of freedom to work with the brief – in terms of the opticals, colour and luminosity’, says Allman. The gate has a similar look to the van but needed to be much more geometry-based, integrating with the shrinking Avengers and FX. Says Pavdeja, ‘A lot of the view of the characters was from the top down and we had to show them not only shrinking but falling down in 3D space – quite a difficult thing to do in 2D. We started to employ a lot more optical effects; time echoes, a few flares thrown in there, atmospherics, smoke, heat distortions,  aberrations and some lens-tilt to achieve a marriage between foreground and background.’

Complex CG environments were also required, with Framestore crafting much of the Avengers hangar base, replacing green screen shot on location of the interior space, as well as exterior views from the windows. The team then turned to an establishing shot of an environment which they knew intimately: Asgard. Having destroyed it in Marvel Studios’ Thor: Ragnarok, Framestore were required to piece it back together. ‘Our Asgard was synced into FX and the destruction we did for Ragnarok so the challenge was working out which version of what would be the one to use,’ says Allman. The sequence originally began with a shot of  Odin’s tower from over the mountains, which the team worked on for six months before a decision to switch to the more traditional front view point was made. Says Penn, ‘miraculously we just switched the camera, rendered it and were surprised to discover that it actually worked. It also made for a much better shot.’

From bringing Smart Hulk into the MCU, revisiting Rocket, building environments and futuristic FX, the film presented a variety of different challenges that the team relished.

Cinesite has delivered 264 shots. It is the sixth Marvel Studios film worked on by Cinesite, which previously completed 215 shots for Avengers: Infinity War.  Both films were directed by Anthony and Joe Russo, with visual effects supervised by Dan DeLeeuw and produced by Jen Underdahl.  Cinesite also worked closely with associate visual effects supervisor Mårten Larsson.

Endgame begins by exploring the emotional impact of the devastating events of Infinity War, where half of life in the universe was destroyed with a single snap of Thanos’ fingers.  Five years later, the remaining Avengers assemble once more to undo Thanos’ actions and restore order to the universe.

A large portion of Cinesite’s work was based around the Avengers revisiting scenes from previous films as they attempt to retrieve the infinity stones.  Cinesite VFX supervisor Simon Stanley-Clamp says, “Our work referenced three previous Marvel films; Captain America: The First Avenger (2011), Marvel’s The Avengers (2012) and Guardians of the Galaxy (2014), so the aesthetics were an important challenge; we had to match the look and tone of the originals faithfully.” 

Cinesite recreated an important sequence from the beginning of Guardians of the Galaxy, set on Morag.  The initial shots, where the Guardians’ M-Ship escape pod lands on the planet surface, were entirely CG, matching shots from the original film.  As Nebula and Rhodey watch Peter Quill (Chris Pratt) singing and dancing, the green screen set piece was extended 100% by adding columns, dripping water and foliage to the environment.  The planet surface is remote and barren yet beautiful, with ruined architectural features seen against a low sunset.  The final shot required extensive input from FX, lighting, animation, texturing and compositing.   

Cinesite’s most challenging sequence is set in a battle towards the climax of the film.  Following the destruction of the Avengers base, Hawkeye is pursued by Outriders in underground tunnels as he searches for the Gauntlet in the rubble.  The terrifying creatures, first seen in the epic battles of Black Panther, move in multiple ways: running, climbing, crawling and leaping.  Adding to the animation challenge, they do so on eight limbs.

Choreography of the Outriders was key to making the exciting chase sequence work.  Although Hawkeye is always one step ahead, the Outriders move at a blistering speed.  To make the pace and positioning believable, the creatures zigzag through the tunnels, drifting up the walls almost like skateboarders in their haste to get past each other and move forward in the limited space. 

Cinesite’s team also created the iconic sequence where Tony Stark is lost in space with Nebula.  From exterior shots showing the isolation of the ship to blackening Nebula’s eyes, recreating her robotic arm and the exterior space environments with a gaseous nebula visible from the cockpit.

In total, Cinesite’s team numbered around 160, including support, and work took eight months to complete.

Cinesite partner company TRIXTER recently completed work for Marvel Studios on Captain Marvel, the film has grossed over $1.1 billion worldwide, making it the first female-led superhero film to pass the billion-dollar mark.

Related Post: Avengers: Infinity War Visual Effects