This documentation is for the old version. Go to the latest Graphics Mill docs

JPEG-Optimized Red-Eye Removal Technique

When removing red-eye effect from JPEG files, all changes are also affect only small part of image - red-eyed pupils. That's why you may wonder whether it is possible to avoid JPEG file recompression using the same technique as described in the Applying Lossless JPEG Transforms topic.

Red-Eye Removal can be easily used in conjunction with LosslessJpegTransform class to recompress only fixed pupils instead of entire photo. This topic discusses how to do it.

General Approach

Let's assume that the we process the following image:

When you apply RedEyeRemoval transform, the algorithm builds a mask which contains so-called blobs, i.e. elements of image which are recognized as red eyes. Each blob is described by its bounding rectangle and center point.

To get mask, use the RedEyeRemoval.Mask property. It exposes the Blobs property which returns one or two blobs (depending on the mode).

This figure demonstrates blobs which will be found for this photo (red rectangles):

After we get coordinates of portions of image which are modified, we can easily apply lossless JPEG patching. In general words, algorithm will be the following:

Load JPEG file as Bitmap.
Apply red-eye removal (e.g. like described in the Semiautomatic Red-Eye Removal or Manual Red-Eye Removal topic).
Crop out all blobs.
Patch the original JPEG file with these blobs.

The only problem you may have is a limitation of JPEG patching technique - coordinates of the patch should be aligned to the size of JPEG samples. However it can be easily achieved with a help of the LosslessJpegTransform.AlignToSampleSize method. This way you will recompress slighly larger section of the image, however it is still more acceptable than recompression of entire photo. On the figure below - red rectangles are original blobs and blue rectangles are aligned blobs:

Red-eyed photo with highlighted original (red) and aligned (blue) blobs.

Code Example

Now let's see how it can be implemented.

First of all, let's define some variables which will keep blobs and other data:

Visual Basic

Dim blobRectList As ArrayList
Dim sourceFileName As String
Dim resultFileName As String
Dim sourceBitmap As Aurigma.GraphicsMill.Bitmap

ArrayList blobRectList;
string sourceFileName;
string resultFileName;
Aurigma.GraphicsMill.Bitmap sourceBitmap;

When the user selects to load the image, we initialize these variables (it is assumed that you define file names yourself):

Visual Basic

sourceBitmap = New Aurigma.GraphicsMill.Bitmap(sourceFileName)
blobRectList = New System.Collections.ArrayList

sourceBitmap = new Aurigma.GraphicsMill.Bitmap(sourceFileName);
blobRectList = new System.Collections.ArrayList();

At last, these methods will do all the job:

Visual Basic

Private Sub RemoveRedEyeSemiautomatically(ByVal face As System.Drawing.RectangleF)

    Dim redEyeRemoval As New Aurigma.GraphicsMill.Transforms.RedEyeRemoval
    redEyeRemoval.Mode = Aurigma.GraphicsMill.Transforms.RedEyeRemovalMode.Semiautomatic
    redEyeRemoval.FaceRegion = face
    redEyeRemoval.ApplyTransform(sourceBitmap)

    AddBlobs(redEyeRemoval.FaceRegion.Location, redEyeRemoval.Mask.Blobs)

End Sub

Private Sub RemoveRedEyeManually(ByVal face As System.Drawing.RectangleF, _
 ByVal eye As System.Drawing.PointF)

    Dim redEyeRemoval As New Aurigma.GraphicsMill.Transforms.RedEyeRemoval
    redEyeRemoval.Mode = Aurigma.GraphicsMill.Transforms.RedEyeRemovalMode.Manual
    redEyeRemoval.FaceRegion = face
    redEyeRemoval.EyePoint = eye
    redEyeRemoval.ApplyTransform(sourceBitmap)

    AddBlobs(redEyeRemoval.FaceRegion.Location, redEyeRemoval.Mask.Blobs)

End Sub

Private Sub AddBlobs(ByVal location As System.Drawing.PointF, _
 ByVal blobs As Aurigma.GraphicsMill.Transforms.RedEyeBlob())

    ' Blobs are returned coordinates of the face rectangle. 
    ' Convert them to coordinate of the bitmap.
    Dim l As System.Drawing.Point = System.Drawing.Point.Round(location)
    For i As Integer = 0 To blobs.Length - 1
        Dim r As System.Drawing.Rectangle = blobs(i).BoundingRectangle
        r.Offset(l)

        ' Add the blob to the list.
        blobRectList.Add(r)
    Next

End Sub

Private Sub SavePatchedJpeg()
    Dim inputStream As System.IO.MemoryStream = Nothing
    Dim outputStream As System.IO.MemoryStream = Nothing

    For i As Integer = 0 To blobRectList.Count - 1

        Dim losslessJpeg As Aurigma.GraphicsMill.Codecs.LosslessJpegTransform

        ' Since the LosslessJpegTransform allows to patch only with one bitmap per time,
        ' when patching the image for the first time, we will be load the image from file. 
        ' Otherwise we load it from memory stream which stores the result of the previous patching.
        If i = 0 Then
            losslessJpeg = New Aurigma.GraphicsMill.Codecs.LosslessJpegTransform(sourceFileName)
        Else
            losslessJpeg = New Aurigma.GraphicsMill.Codecs.LosslessJpegTransform(inputStream)
        End If

        ' Align blob rectangle to JPEG sample size. 
        Dim blobRect As System.Drawing.Rectangle = _
         losslessJpeg.AlignToSampleSize(blobRectList(i), _
         Aurigma.GraphicsMill.Codecs.JpegAlignToSampleSizeMode.Patch)

        ' Get the bitmap for the patched area.
        Dim blobBitmap As New Aurigma.GraphicsMill.Bitmap
        Dim crop As New Aurigma.GraphicsMill.Transforms.Crop
        crop.Rectangle = System.Drawing.RectangleF.op_Implicit(blobRect)
        crop.ApplyTransform(BitmapViewer1.Bitmap, blobBitmap)

        ' If we patch the last blob, save result to file. Otherwise save to a 
        ' memory so that we could patch next blobs without temporary files.
        If i < blobRectList.Count - 1 Then
            outputStream = New System.IO.MemoryStream
            losslessJpeg.WritePatched(outputStream, blobRect.Location, blobBitmap)
        Else
            losslessJpeg.WritePatched(resultFileName, blobRect.Location, blobBitmap)
        End If

        losslessJpeg.Close()

        If i < blobRectList.Count - 1 Then
            inputStream = outputStream
        End If

        ' Clean resources up.
        losslessJpeg.Dispose()
        blobBitmap.Dispose()

    Next
End Sub

private void RemoveRedEyeSemiautomatically(System.Drawing.RectangleF face)
{
    Aurigma.GraphicsMill.Transforms.RedEyeRemoval redEyeRemoval = 
        new Aurigma.GraphicsMill.Transforms.RedEyeRemoval();
    redEyeRemoval.Mode = Aurigma.GraphicsMill.Transforms.RedEyeRemovalMode.Semiautomatic;
    redEyeRemoval.FaceRegion = face;
    redEyeRemoval.ApplyTransform(sourceBitmap);

    AddBlobs(redEyeRemoval.FaceRegion.Location, redEyeRemoval.Mask.Blobs);
}

private void RemoveRedEyeManually(System.Drawing.RectangleF face, System.Drawing.PointF eye)
{
    Aurigma.GraphicsMill.Transforms.RedEyeRemoval redEyeRemoval = 
        new Aurigma.GraphicsMill.Transforms.RedEyeRemoval();
    redEyeRemoval.Mode = Aurigma.GraphicsMill.Transforms.RedEyeRemovalMode.Manual;
    redEyeRemoval.FaceRegion = face;
    redEyeRemoval.EyePoint = eye;
    redEyeRemoval.ApplyTransform(sourceBitmap);

    AddBlobs(redEyeRemoval.FaceRegion.Location, redEyeRemoval.Mask.Blobs);
}

private void AddBlobs(System.Drawing.PointF location, 
    Aurigma.GraphicsMill.Transforms.RedEyeBlob[] blobs)
{
    // Blobs are returned coordinates of the face rectangle. 
    // Convert them to coordinate of the bitmap.
    System.Drawing.Point l = System.Drawing.Point.Round(location);
    for (int i = 0; i < blobs.Length; i++)
    {
        System.Drawing.Rectangle r = blobs[i].BoundingRectangle;
        r.Offset(l);

        //Add the blob to the list.
        blobRectList.Add(r);
    }
}

private void SavePatchedJpeg()
{
    System.IO.MemoryStream inputStream = null;
    System.IO.MemoryStream outputStream = null;

    for (int i = 0; i < blobRectList.Count; i++)
    {
        Aurigma.GraphicsMill.Codecs.LosslessJpegTransform losslessJpeg;

        // Since the LosslessJpegTransform allows to patch only with one bitmap per time,
        // when patching the image for the first time, we will be load the image from file. 
        // Otherwise we load it from memory stream which stores the result of the previous patching.
        if (i == 0)
        {
            losslessJpeg = new Aurigma.GraphicsMill.Codecs.LosslessJpegTransform(sourceFileName);
        }
        else
        {
            losslessJpeg = new Aurigma.GraphicsMill.Codecs.LosslessJpegTransform(inputStream);
        }

        // Align blob rectangle to JPEG sample size. 
        System.Drawing.Rectangle blobRect = losslessJpeg.AlignToSampleSize((Rectangle)blobRectList[i],
            Aurigma.GraphicsMill.Codecs.JpegAlignToSampleSizeMode.Patch);

        // Get the bitmap for the patched area.
        Aurigma.GraphicsMill.Bitmap blobBitmap = new Aurigma.GraphicsMill.Bitmap();
        Aurigma.GraphicsMill.Transforms.Crop crop = new Aurigma.GraphicsMill.Transforms.Crop();
        crop.Rectangle = blobRect;
        crop.ApplyTransform(BitmapViewer1.Bitmap, blobBitmap);

        // If we patch the last blob, save result to file. Otherwise save to a 
        // memory so that we could patch next blobs without temporary files.
        if (i < blobRectList.Count - 1)
        {
            outputStream = new System.IO.MemoryStream();
            losslessJpeg.WritePatched(outputStream, blobRect.Location, blobBitmap);
        }
        else
        {
            losslessJpeg.WritePatched(resultFileName, blobRect.Location, blobBitmap);
        }

        losslessJpeg.Close();

        if (i < blobRectList.Count - 1)
        {
            inputStream = outputStream;
        }

        // Clean resources up.
        losslessJpeg.Dispose();
        blobBitmap.Dispose();

    }
}

Let's give some comments to this code. RemoveRedEyeSemiautomatically and RemoveRedEyeMaually should be called when the user selects a face and run semiautomatic or manual red-eye removal algorithm respectively. Both the methods applies the removal effect and append result blobs by calling the AddBlobs method. It just puts a rectangle corresponding to the updated part of image to the blobRectList variable.

When the user is satisfied with a result, the SavePatchedJpeg method should be called. It uses these blobs to patch the original JPEG file using the approach described above.

JPEG-Optimized Red-Eye Removal Technique

General Approach

Code Example

See Also

Manual