I guess doing the lipsyncing as the last step in the production of a movie is not what you would call ‘common practice’. It is not even ‘good practice’. But then, that’s just another first, of the many others encountered during production 🙂
After the recording session with Julia I managed to clean up the takes and provide Phil and Mikkel with a bunch of solid raw takes. Julia you did a fabulous job. Thank you !
While being swamped with work, Phil was able to do a very first draft of the song, mixing Julia’s voice (properly postprocessed) with orchestration. This first version is already very beautiful. Thinking back at the moment when I actually wrote the passage in the script, I never imagined to have such a touching song in my movie.
Again, this was one of those magic moments I encountered throughout the production, when individual pieces come together to form something new and this ‘new’ is more then the raw sum of those parts.
Well, anyway, this ‘finished’ song provided me now with the last missing part to finalize the visual part of the movie, which is the lipsyncing.
And it came to no big surprise that some lessons were learned, … again.
First, the rigging of the mouth sucked horribly in regard to create plausible mouth shapes for the song. But changing the rig at this point was out of the question, so I had to tweak a lot. Having a song with long held notes helped definitely here 🙂
Next, some decisions I made for speed improvements came now back to me to bite me very hard. For directional reasons and to fit the images better to the music, I edited the song sequence in such a way, that I used parts of some shots twice with different points of the song. Regarding rendertime this was a good decision, but, as you can imagine, trying to get this to work with lipsyncing is a big fail.
For one shot I was lucky, as Ara is only seen from relatively far away, and the overlap is at long held notes of the same vocal. Phew !
For the other shot, which is a closeup of Ara’s face, I had no chance to cheat in this way. I cursed the editor and tried to cheat anyway, until I noticed that in the edit I still had 14 unused frames at the end of the shot, which allowed me the shift the overlapping part by exactly this amount. And since the overlap was only 10 frames I was saved. Double Phew !
Another interesting thing to do, was not only to match the lip and mouth movements, but to also use the facial controls to match the emotional state of the song. I did some self observations how my lids, brows and forehead moved when trying to sing the song myself (it is a good thing nobody was in hearing range …). It is always an enlightening experience to observe what small movements of the brows can do the a face’s expression.
The last two nights I managed to finish this final task as well and currently my render machine is humming along and creates the final renders. These will be finished on Sunday and then the visual aspect is truly finished and I am eagerly waiting for the sound and music department to fill in the missing pieces.
It’s soooo close now …
