Project 3: Face Morphing

In order to morph one image to another, Ineed to mark some reference points on both images that correspond to each other, so that certain parts of the first image know which direction to deform to into the second image. The point-marking is accomplished using the web interface linked here.

In order to visually effectively interpolate between the two images, Ialso compute a triangle mesh out of the given set of vertices for both images using Delauney triangulation, with the intent that Ideform each triangle face from one image to the next.

There needs to be a 1-to-1 correspondence with the mesh in the first image with the second so that each point specified within the first image has a "final destination" or a position to interpolate to in the second image.

Below are two example images with the point/vertex positions marked at key facial features (for example, around the eyes, contours of the face, mouth, etc.), as well as a visualization of both images' mesh representation. The middle bottom figure displays both images' meshes overlayed with the mesh generated by interpolating halfway between the two images.

In order to compute the midway image's colors, Ineed to consider several things:

1. The midway mesh's triangle faces, and their transformation matrix between each individual triangle in both of the original images' meshes.
   • Let's denote the transformation matrix between an arbitrary coordinate in the midway image with that in image 1 as A_midto1 and a coordinate in the midway image with that in image 2 as A_midto2. Then:
     \(image1coordinates = A_{midpto1} * midwaycoordinates \)
     \(image2coordinates = A_{midpto2} * midwaycoordinates \)
2. The color of each triangle in the interpolated mesh by sampling the original image.
   • I get a list of all coordinates within a given triangle face in the mesh, and use the formulations in (1) to obtain the coordinates in image 1 to sample a color from.
   • I used bilinear interpolation sampling to avoid any aliasing artifacts when sampling pixels in the original images.
   • Irepeat this procedure for all triangles in the mesh in order to fill in all the colors in the midway image
3. Cross dissolving both images
   • For this section, Iwant to have our image be 50% Rebecca and 50% George, so I add the two transformed midway image buffers and average the color channels per pixel.

Here are the results for 50% cross dissolve and 50% deformation:

We generalized Part 2's algorithm to include different levels for mesh deformation and cross dissolution. We render out 45 frames of an image morphing into another while incrementing the cross dissolve and the deformation by 1/45 each time. The general linear interpolation method for cross dissolution and deformation is

Results are shown below:

I used the FEI face database to calculate and display the average Brazilian face. First, I computed the position of the points of the average face, and morphed each image within the dataset to those points. Finally, I averaged the images together into one. Results are as follows:

Here are some sample images from the dataset that were morphed into the shape of the average face:

Here is the result of morphing the average Brazilian face to my face shape, and my face to the average Brazilian face shape.

I look SO brazilian.

We can create caricatures of images by cranking the linear interpolation percentage past 100% (or below 0%, depending which way you want to deform). Here are the results for making my face EXTRA Brazilian, by 130%, 160%, 200%:

We use the same idea of facial morphing with a photo of my boyfriend and Megamind:

Here's a small music video of multiple faces of my cat: