Project 3 Face Morphying

Part 1. Facial keypoints and triangulation

I utilized the ginput function to select the keypoints of my face. Based on a specific order, for example, from nose to mouse and from eyes to ears, I orderly selected different keypoints on my face and add four bounder points at each angles of the image. Then I utilized the scipy.spatial.Delaunay to help me create triangulation based on selected keypoints.

The result is listed below.

Part 2. Affine warp between triangles

An affine warp is a type of linear mapping method used in image processing that allows for a variety of geometric transformations including rotation, scaling, translation, and shearing. The main advantage of using an affine warp is that it preserves lines and parallelism in images, which makes it particularly useful for geometric corrections in image editing and computer vision applications.

By using np.linalg.solve function to solve the linear matrix between target triangle points and source points, I get a affinity matrix, which projects from target to source. Then I calculated the inversive affine matrix and applied it to the source points, which help me perfectly warp the image.

The raw images and warp results are listed below.
‍

Part 3. Morphing sequence

Part 4. Average face

By using an annotated dataset on https://fei.edu.br/~cet/facedatabase.html , I averaged the keypoints of each face and use it as the target to morph the images. Here is the result of average face of 200 different faces, and the top four images is part of the morphed results and bottom are the initial images.

Then I tried to morph my own face into average face and morph other's face in my face. The following is the results.

Part 5. Caricatures: Extrapolating from the mean

I calculated the distance between my face's keypoints and mean face's keypoints and times it by 2.0, then add it back to my face's keypoints.

Bells and Whistles

I created a music video for Star Trek captains by using face morphing.