Cellphone Commercial: 4-Point Tracking in mocha AE

Topics covered in this After Effects tutorial:

BorisFX Mocha for Planar Tracking, Drawing the Outer & Planar Surfaces, Tracking & Refining in Mocha, Compositing the Screen, Luma Mattes vs. Alpha Mattes, Manually Tracking in AE

Exercise Preview

Exercise Overview

Now that we’ve removed the blue screen from Janet’s phone, it’s ready for a more inspiring background. Let’s give her phone a makeover by compositing the Cellphone logo onto it. To do so, we’ll need to use motion tracking to define which area will be covered by this graphic.

In a previous exercise, we tracked a single point on a moving item. However, this time we need to ensure that all four corners of Janet’s phone get tracked. To ensure we get a great result even when the phone wobbles or rotates, we’ll do our 4-point tracking in mocha AE, an application that installs with After Effects.

Previewing What You’ll Make in This Exercise

1. On the Desktop, navigate to Class Files > After Effects Level 2 Class > Cellphone > Finished Clips and double–click Cellphone Phone Screen Replacement.mov.

2. Play the video and notice that this is the same footage of the woman’s hand holding a phone, but with the Cellphone logo instead of the boring blue.

3. You should still have yourname-Cellphone.aep open in After Effects. If you closed it, re-open it now. We recommend you finish the previous exercises (3A–3B) before starting this one. If you haven’t finished them, do the following:

   • Open Cellphone-Ready for 4-Point Tracking.aep (from the Cellphone > Finished AE Projects folder).
   • Save the file as yourname-Cellphone.aep in the Cellphone folder.

Using mocha AE for Planar Tracking

After Effects is not natively great at 4-point tracking. Fortunately it comes with an application called mocha AE, which can help us out. mocha AE specializes in planar tracking, which takes advantage of the fact that there are a myriad of flat surfaces—or planes—all around us. It tracks surfaces instead of points. Instead of looking for the bull’s eye like a point tracker, mocha AE is focused on the movement of surfaces.

1. If you do not have Cellphone-MAIN open in the Timeline already, open it now.

2. Double–click the [3-Janet-CUphone] layer.

3. In your Composition viewing panel, change your Resolution to Full. Tracking at less than Full resolution will have imprecise results.

4. Select the hands layer.

5. Go to Animation > Track in BorisFX Mocha. This will add the Mocha effect to your layer.

6. In your Effects Control window, locate the Mocha AE CC effect, and click the Mocha icon, to launch the software.

7. If a Registration window pops up, click Register Later.

8. If a window appears asking if you’d like Mocha to collect anonymous usage data, click Collect.

9. Now we’re in mocha AE. Before we get started, let’s get acquainted with the workflow for tracking in mocha AE:

   • First we need to draw the mask that tells the tracker where to look. This is a box that fully encompasses the tracked area, plus more (similar to the outer box in After Effects’ native Tracker).
   • Then we need to draw an inner shape around the area we want to track. This defines the plane (exact shape and angle) of the surface that will be tracked, which is much more precise than After Effects can do.
   • Then we track the footage. We can either track it all at once with a single button click, or go frame-by-frame if we need to be more precise.
   • Finally we check the tracking. It’s not always perfect, so it’s often necessary to adjust individual frames as needed (by adding keyframes).

Drawing the Outer & Planar Surfaces

1. Let’s make sure we’re in the right workspace for this part of the tracking. At the upper left-hand corner of the interface to the right of the Tools, click the dropdown and make sure it’s set to Essentials.

2. We’ll start by using a tool called Create Rectangular X-Spline Layer. An X-Spline is basically a fancy word for a mask where the points are outside of the mask shape. In the Tools panel at the top, select the Create Rectangular X-Spline Layer tool.

3. Draw a box around the entire phone, including her fingers. It’s OK if it doesn’t include the whole phone, as we’ll adjust its shape next.

4. In the Upper Left-hand corner of the interface, notice something called Layer 1 has been created in the Layers Window.

5. To delete a terrible X-Spline, simply click the name of the layer and hit the delete key on your keyboard.

6. To adjust your X-spline points, simply choose the Pick Tool (which looks like the Selection Tool in AE). Move your points so that they surround the phone and some of the space outside it. You can change the position of the spline points by dragging on them (at the base of the blue handles).

   NOTE: Dragging on the blue box at the end of the handles will adjust the roundedness of the corners.

7. We can use keystrokes to temporarily switch tools. These are different than the ones in After Effects, so be aware NOT to press the Spacebar while doing the following:

   • To access the Zoom tool, hold down the Z key and drag upwards on the phone to zoom in (dragging down zooms out).
   • The Pan tool works like After Effects’ Hand tool. Hold down the X key and drag on the screen to pan around.

8. To the left of the main panel and below the Track section, click the Show planar surface button .

9. A blue box will appear over the phone. Switch to the Zoom tool or hold down Z and drag up to zoom closer to the top edge of the blue screen.

10. Switch to the Hand tool or hold down X to pan around and move the view.

11. Select the Pick tool if it’s not already. This works like After Effects’ Selection tool.

12. Drag the corners of the blue inner box to match them to the corners of the blue screen, the area we want to replace. Keep in mind that it’s OK to extend a little extra, but it’s not OK to have too little coverage.

Doing the Tracking & Refining Frames with Misalignment

The next step is to track the footage. We can either track all the footage now, or go frame-by-frame. In some frames, Janet’s hand obscures at least one of the phone corners. This can diminish the level of confidence in the tracking, even in fancy planar trackers like mocha.

1. Locate the “Track Motion Options” in the “Essentials” window, and turn “on” Perspective if it’s not already.

2. At the bottom of the viewing panel, locate the Timeline (which looks like a ruler) and the playhead within it.
   Make sure the playhead is at the start of the Timeline before you start tracking.

3. Then click Track Forwards button .

4. When the blue box around the bottom of the blue screen is misaligned hit the STOP TRACK button (square).

5. We need to fix this. Go back to the beginning by dragging the white playhead at the bottom of the main screen to the green arrow:

   

6. Realign the planar surface (blue box), and you’ll see that this is near-impossible! We can’t actually see the lower 2 corners of the cell-phone screen because Janet’s hand is obscuring them.

7. Since we’re having a problem with the bottom tracking, instead of tracking this whole area with our mask, let’s see what happens when we just track the area around the top two corners. As shown below, drag up the bottom left and bottom right spline points (leave the planar surface/blue box in place):

   

8. Let’s see what happens when we track all the frames. At the bottom right of the main panel, to the right of Track, click the Track Forwards button .

9. Notice it looks pretty good! Some of the frames could use adjustment so it aligns to the bottom corners, but we can take care of that next by adding keyframes.

10. Go back to the beginning by dragging the white playhead to the green arrow.

11. Zoom in on the bottom of the phone screen by holding Z and dragging upwards.

12. If needed, use the Pick tool to fix the alignment of the points.

13. Hold Z to zoom out again so you can see the whole phone.

14. To adjust the track, we’ll need to switch workspaces. At the upper left-hand corner of the interface to the right of the Tools, click the dropdown currently set to Essentials, to Classic.

NOTE: The Classic interface has a lot more ability to customize and adjust your track for those “less than perfect” tracking shots. Which is most of them.

Adjusting the Track

Just like in After Effects, we only need to add keyframes at key moments. This means the next step is to identify the frames where the planar surface is most misaligned (or “suck” frames, as we call them). Adding just a few should be enough to fix all the frames where the automatic tracking drifts away from the corners.

1. On a sheet of paper or blank text document on your computer, and make a list of frame numbers where your track drifts the most. Depending on how you drew your X-spline and adjusted your Planar Surface (blue box with corners), your “suck frame” numbers may be different than the ones we’ve listed.

2. Look at the area below the timeline bar. Locate the Current Frame value (the middle number at the bottom left). While looking at this area, drag the white playhead a few frames forward to notice the Current Frame number change.

3. Drag the white playhead to frame 96 or so, around when the planar surface starts to drift.

4. At the bottom left area with the three Parameters tabs, click the AdjustTrack tab. This is where we add keyframes to fix areas of footage that’s already been tracked.

5. Within this panel, click the Set Master All button.

6. Go to frame 99. You can either drag the playhead or double–click the current value, enter the number, and press Return/Enter.

7. To check the alignment, it may help to use a grid. On the left side of the screen, find the Layer Properties panel. In the menu next to Insert Clip, choose Grid8x8.

8. Zoom in so you can see the bottom corners of the phone screen up close.

9. With the Pick tool adjust any corners that are inaccurate, using the enlarged views in the corner. Notice that green arrow keyframes appear on the timeline.

10. In the Layer Properties panel on the left, next to Insert Clip set the menu to None.

11. Go to frame 118 and adjust the corners again.

12. Especially with these kinds of subtle movements with the phone, we want to be as accurate as possible. Notice that the keyframes that are appearing are for each corner. It’s not all the corners at once, as you might be used to from After Effects.

13. Take your time to go through frame-by-frame from the beginning, zooming as needed and using the Pick tool to move the spline points.

    TIP: To adjust an existing keyframe, you need to have the playhead directly over it. You can tell you’re directly on a keyframe by looking at the bottom right under the timeline and seeing the Delete keyframe icon is not grayed out.

14. Once you’re happy with your track, go to File > Save Project.

15. Close out of Mocha AE by clicking the red button in the upper left-hand corner of the interface,
    or by going to File > Exit Mocha AE.

Compositing the Cellphone Screen

1. Return to After Effects and make sure you’re in the 3-Janet-CUphone tab.

2. Select the [ExclamationPt-HDsize] layer (the logo comp) and go to its in point.

3. This is what we want to track on the phone screen, so turn on its eye .

4. Anytime you’re tracking anything in After Effects using mocha, you need it to fill the frame of whatever comp you’re working in. To see what size we need to match, press Cmd–K (Mac) or Ctrl–K (Windows) to see that it’s 1920 x 1080 px.

5. Click Cancel to close the window.

6. In the Timeline, double–click [ExclamationPt-HDsize] to open it in a new tab.

7. Select ExclamationPt-Portrait.

8. Go to Layer > Transform > Fit to Comp. The thing that fits to comp needs to be the ratio of whatever the screen is. In this case, our phone screen’s aspect ratio is 9:16.

9. Close the ExclamationPt-HDsize tab to return to the 3-Janet-CUphone tab.

10. Select the hands layer and go to its Effect Controls window.

11. In the Mocha AE CC effect, swivel open the Tracking Data section and click the Create Track Data button.

12. In the window that opens, click the sprocket on for the layer you wish to get tracking data from.

13. Back in Effect Controls, set the Export option dropdown to Corner Pin (Supports Motion Blur).

14. Then set the Layer Export To dropdown to 1-ExclamationPt-HDsize and click the Apply Export button.

15. With [ExclamationPt-HDsize] selected, press U. Notice that it gave us one keyframe for every frame of the screen tracking.

16. Feel free to scrub through with the playhead to see that the tracking alignment is pretty good.

    If Your Tracking on the Phone Screen Doesn’t Work

    If for some reason, the tracking you did doesn’t look quite right, you may choose to use the keyframe data we’ve provided for you.

    1. We need to delete the [ExclamationPt-HDsize] layer’s keyframes before we can replace them, so make sure they are showing.
    2. Select Corner Pin and press Delete (you may need to deselect first).
    3. Click the stopwatches next to Position, Scale, and Rotation to delete their keyframes.
    4. Close up the layer, reopen it, and to the right of Transform click Reset.
    5. Go to the Desktop, navigate to Class Files > After Effects Level 2 Class > Cellphone > mocha, and double–click 3-Janet-CUphone-Noble.txt.
    6. Select all the text, then copy it.
    7. Close the file and return to After Effects.
    8. Make sure the playhead is at the beginning of the Timeline.
    9. With the [ExclamationPt-HDsize] layer selected, paste the data.
    10. Scrub through with the playhead and the keying should look good.

17. Notice that Janet’s hands are being covered by the logo when they hover over the screen. We can fix this by applying the phone matte.

18. Drag the phone matte layer up to be the topmost layer.

19. Show the phone matte by clicking its eye .

20. Drag through with the playhead to see that most of the layer (including the hand) is black, and only the phone is white.

21. Notice that this layer is already being used as a Luma Matte for the [ExclamationPt-HDsize] layer, as you can see in that layer’s TrkMat column.

    Luma Mattes vs. Alpha Mattes

    The alpha mattes we’re familiar with apply the layer’s alpha transparency data to the layer below it. Luma mattes work similarly, except After Effects uses the layer’s luminescence (brightness and darkness) to matte the layer below. The lighter the layer, the more the layer below will show. White fully reveals the matted layer, and black completely hides it (similar to a layer mask in Photoshop).

22. Hide/show the phone matte layer as many times as needed to see how After Effects used its luminescence data to reveal only the part of the magenta phone screen that’s not covered by the hand. Make sure the layer is hidden before you move on.

23. Drag through with the playhead to see that Janet’s hands are no longer hidden by the logo. This works because our phone matte was white wherever the phone was and we want the tracked screen to only show through the white of the phone matte.

More Tracking with mocha

We’ll use mocha to do more tracking, but this time we’ll encounter some of its limits.

1. Switch to the Cellphone-MAIN Timeline tab.

2. Double–click the [2-Janet-phone] layer.

3. Scrub through the footage. The phone in the previous shot had a little wobble, but this phone is a full blown contortionist! Janet moves her phone a large distance, and it rotates and changes perspective.

4. At the bottom of your Composition panel, change your Resolution to Full.

5. Select the janet layer, the one we want to track.

6. Move your playhead to the beginning of the Timeline.

7. These free range corners don’t want to be pinned down, but it’s doable. Let’s see
   what mocha AE can do! Go to Animation > Track in BorisFX Mocha to add the effect to the layer.

8. Click the Mocha icon in the Effects Control window to launch it.

9. If a Registration window pops up, click Register Later.

10. In mocha, hold down the Z key and drag upwards on the phone to zoom in.

11. To move the view, hold down the X key and drag.

12. In the Tools panel at the top, select the Create Rectangular X-Spline Layer tool.

13. Once again, we want to draw a box around the entire phone. After drawing a rough box, adjust the corners as shown:

    

14. If you haven’t already, click on the first point again to close the shape.

15. In the Tools panel, make sure the Pick tool is selected.

16. Above the main panel, click the Show planar surface button .

17. Drag the corners of the blue inner box (planar surface) to match the corners of the blue phone screen, the area we want to be replaced. Use the enlarged view that appears in the top-left corner to help you perfect alignment. Keep in mind that it’s OK to extend a little extra, but it’s not OK to have too little coverage.

18. With a spastic target like this, who knows what Escher-like result we’d get if we tracked it all now… This is a clear cut example of when we want to be super methodical and track frame-by-frame. At the bottom right of the main panel, to the right of Track, click the Track To Next Frame button . (If you don’t see it, switch your workspace to Classic)

    Keep clicking until you start to see the blue tracking box not aligning properly with the screen, probably around frame 86. At this point, when the path becomes misaligned, we’ll just stop and do the remaining tracking in After Effects. When you have footage with lots of movement like this, tracking by hand will have better results than poorly drawn automatic tracking.

19. We’ll use mocha’s AdjustTrack features to help us refine the good part we have. Towards the bottom left of the mocha app, click the AdjustTrack tab.

20. Go back to the first frame and if needed, realign anything that needs adjusting.

21. Click the Set Master All button to make this the master frame.

22. Our “suck” frames are toward the end, so go to frame 84.

23. Use the Pick tool to realign the blue inner box with the screen.

24. Move the playhead to 86 and realign again.

25. Go to File > Save Project. Exit Mocha by going to File > Exit Mocha.

26. Back in After Effects, select the janet layer and go to its Effect Controls window.

27. In the Mocha AE CC effect, swivel open the Tracking Data section and click the Create Track Data button.

28. In the window that opens, click the sprocket ON for the layer you wish to get tracking data from.

29. Back in Effect Controls, set the Export Option dropdown to Corner Pin (Support Motion Blur).

30. Then set the Layer Export To dropdown to tmo-HDsize and click the Apply Export button.

Continuing Tracking in After Effects

1. Return to After Effects and make sure you are in the 2-Janet-phone tab.

2. Select the [tmo–HDsize] layer, show it , and go to its in point.

3. Paste the tracking we did in mocha.

4. Press U to show the keyframes for the Corner Pin effect we just pasted.

5. Press K to keyframe navigate and preview the tracking frame-by-frame.

6. Stop when you get to the frame where the screen alignment goes off track. This should be around 2;28.

7. Drag a selection box around all the keyframes at 2;28 and after 2;28 (or wherever the alignment goes off track) and delete them.

8. With [tmo–HDsize] selected, press Shift–T.

9. Reduce the Opacity of the screen to 50%. Now we can see everything else better.

10. Select the Corner Pin effect.

11. Zoom in close to the phone on the Composition panel (make sure the Resolution is set to Full).

12. Make sure the playhead is set to 2;28 (or wherever you stopped and deleted keyframes), and the Selection tool is selected.

13. Drag the beige points around the Cellphone graphic to align it with the phone screen. Be careful NOT to drag the entire graphic, as that will change its Position!

    

14. As you do this, notice that new keyframes are being added for the corner pins.

15. Move forward one frame at a time, and realign the corner pins to match the screen.

16. When you’re done, set Opacity back to 100%.

17. Click in a blank area of the Timeline to deselect all.

18. Scrub through with the playhead. You may notice that at certain points, there’s a bit of edge showing through the screen. This is happening because there’s a bit of glow around the edge. Fortunately in this case, because the background is simple, we can fix this pretty easily.

    If You Don’t Like Your Results

    If you notice problems other than the screen glow, you can use the provided keyframe data. To replace your keyframes with ours, delete the Corner Pin effect and make sure you’re at the [tmo–HDsize] layer’s in point. Then copy and paste from mocha > 2-Janet-phone-Noble.txt.

Getting Rid of the Extra Edges Around the Screen

1. Remember when we did our keying, we did a separate key for the screen? That’s going to come in real handy here. Select the phone matte layer and show it .

2. Solo it and go to its in point.

3. To make this layer better for use as a luma matte, apply the Levels effect.

4. In the Effect Controls panel, look underneath the Histogram (the graph, not under the black-to-white gradient!) Drag the black (leftmost) slider to the right until the screen is as black as it can get.

5. Drag the white (rightmost) slider slightly to the left.

6. Un-solo the phone matte layer by clicking in its Solo column .

7. Press Cmd–Y (Mac) or Ctrl–Y (Windows) to create a new solid.

8. We want this color to be the exact same color as the Cellphone background. In the Solid Settings dialog, click the eyedropper next to the color swatch.

9. With the eyedropper, click on the magenta Cellphone background.

10. Name it glowBG, make sure it’s comp size, and click OK.

11. Drag glowBG beneath the phone matte layer.

12. If you don’t see the TrkMat column, click the Toggle Switches / Modes button.

13. Black areas of the matte will be hidden in the matted layer. We want to show the phone screen, so we need an inverted matte. For the glowBG layer, click the menu in the TrkMat column and change it to Luma Inverted Matte “phone matte”.

14. Move the playhead to 2;29. This is where the keying for the phone matte ends.

15. Shorten the end of the glowBG layer by pressing Opt–] (Mac) or Alt–] (Windows).

16. Scrub through with the playhead and check out the logo on the phone screen.

17. If your phone screen isn’t glowing enough, go into the phone matte layer’s Keylight settings within the Effects Controls panel. Adjust the Screen Shrink/Grow, reducing it to anywhere between –0.5 and –1.5 per your taste. You can also increase the Screen Softness property to make the glowBG blur a bit.

18. Collapse the layers in the Timeline and close the tabs other than Cellphone-MAIN.

19. Save the project and leave it open. We’ll continue working on it in the next exercise.