Screen Graphics Pipeline

For Frozen 2, the emphasis was on artistic control, rather than mass, procedural population. For the “Show Yourself” musical sequence, we needed a tool that would allow us to display all the memories that would appear within the ice walls of the Ahtohallan glacier. Because this sequence was highly designed, we needed full control over which sets of images were seen per shot in order to support the narrative of the lyrics of the song, move the story forward, and convey the emotional impact. The screen graphics pipeline developed for Ralph Breaks the Internet was the perfect solution.

On the authoring side, we were able to take advantage of dpix, our proprietary repository and review tool for images and movies submitted by artists. Because dpix already provides extensive search functionality, we were able to extend it by simply adding additional metadata to any clips submitted as screen graphics.

Using this metadata, artists can quickly and easily browse and search dpix for the exact graphics they need for a particular screen.

We also needed to change the way these textures were consumed in our materials pipeline. In the past, textures were consumed in our materials using a map module.

If we just used the existing map module, we could end up with hundreds of user variables in a material, which would get unwieldy very quickly.

Using the results of these map calls through SeExpr embedded expressions would be cumbersome. Instead we added a new ‘SequenceMap’ module for dealing with sequences of image clips. This uses projection UVs typically authored as a separate set of Ptex maps, which is standard and consistent with existing map modules.

The new parameters in this SequenceMap are a JSON config file which contains the set of clips of varying lengths, and a frame expression which evaluates to give the position within those clips at the current render time.

This was then tied into a new user interface to allow an artist to quickly and easily populate and then individually edit graphics on screens in their shot. We provided a “drag-and-drop” plugin that allows artists to simply select screen geometry in Maya, choose the screen graphics clip they wanted in dpix, and drag it directly into the scene. All of the material binding and SequenceMap setup is taken care of for them, letting the artists focus on the composition and placement of the screens rather than the details of material parameters. An additional editor makes it easy to adjust the clip order, number of frames to play, and control any frame offset.

One of the key differences between the screens in Ralph Breaks the Internet and the “Show Yourself” sequence in Frozen 2 was the need for our characters to respond and react to the images on the screens, rather than being an ambient background element. Elsa's duet with the spirit of her mother Iduna provides the cathartic peak for the song, and required us to develop additional tools for animators to be able to quickly update the screen graphics with work-in-progress publishes in order to match eye lines and finesse the timing of Elsa's and Iduna's acting.

The additional fidelity required for Elsa to interactively "perform" against the screen also required us to adjust the maximum resolution for the screen graphics inside of Maya—previously it had been capped as a performance optimization in order to accommodate the hundreds or thousands of screens that might be in the background on Ralph Breaks the Internet.

The screen graphics toolset is now an invaluable fixture at Walt Disney Animation Studios, enabling us to quickly and easily add pre-generated imagery into any scene.