Rotate, flip and transform

edit

Quarter turns and mirrors move pixels without changing them. Rotations by any angle, skews and transformations compute new pixels with a resampling filter, like a resize. All of them apply to every frame and to the poster frame.

Quarter turns and mirrors

Rotate(RotateMode) turns the image clockwise by 90, 180 or 270 degrees, and Flip(FlipMode) mirrors it horizontally or vertically:

public static void RotateAndFlip(string inputPath, string outputPath)
{
    using var image = Image.Load(inputPath);

    // Make the stored pixels upright first: a camera stores how to turn the photo in its EXIF orientation
    image.AutoOrient();

    // A clockwise quarter turn, then a mirror: every pixel is moved, none is resampled
    image.Rotate(RotateMode.Rotate90);
    image.Flip(FlipMode.Horizontal);
    image.Save(outputPath);
}

These are exact pixel permutations: every sample, 16-bit low bits, floating-point values and alpha included, is copied unchanged. A quarter turn swaps the width and the height, and the axes of the pixel aspect ratio. Flip also exists for one frame.

Apply the EXIF orientation

A camera or a phone often stores the pixels as the sensor read them, with an EXIF orientation that tells viewers how to turn them. AutoOrient applies that orientation to the pixels (all eight EXIF orientations: mirrors, rotations and the diagonal reflections), then sets it to ExifOrientation.TopLeft. It does nothing when the image is already upright.

The other operations work on the stored pixels and keep the orientation as it is. Call AutoOrient first, before a crop, a resize or a rotation, so that they apply to what the viewer sees: a rotation of a photo that still has its orientation would be turned again by the viewers. See also Handle phone photos and Metadata and orientation.

Rotate by any angle

Rotate(degrees, options) turns the image around its center. A positive angle turns clockwise. By default, the canvas grows to hold the whole turned image, and the corners it gains get TransformOptions.BackgroundColor:

public static void Straighten(string inputPath, string outputPath, double degrees)
{
    using var image = Image.Load(inputPath);

    // The canvas grows to hold the whole turned image. A JPEG image has no alpha: the corners need an opaque color
    image.Rotate(degrees, new TransformOptions { Filter = ResamplingFilter.Lanczos3, BackgroundColor = new Rgba64(65535, 65535, 65535, 65535) });
    image.Save(outputPath);
}

A rotation by a multiple of 90 degrees is the exact permutation of Rotate(RotateMode), and a rotation by a multiple of 360 degrees leaves the image unchanged.

Skews and affine transformations

Skew(degreesX, degreesY) leans the vertical lines of the image by one angle and its horizontal lines by another one, around its center. Transform applies an AffineTransform: any combination of translations, scalings, rotations and skews:

public static void SkewAndTransform(string inputPath, string skewedPath, string transformedPath)
{
    using var image = Image.Load(inputPath);

    // The vertical lines lean by 20 degrees. The canvas grows to hold the whole image, and the area the image does
    // not cover is transparent: a pixel format without alpha needs an opaque TransformOptions.BackgroundColor
    using (var skewed = image.Clone())
    {
        skewed.Skew(20, 0);
        skewed.Save(skewedPath);
    }

    // Half the size, then a 45-degree turn: the transformations are combined first, so the pixels are resampled once
    var transform = AffineTransform.CreateScale(0.5, 0.5).Then(AffineTransform.CreateRotation(45));
    image.Transform(transform, new TransformOptions { Filter = ResamplingFilter.Lanczos3 });
    image.Save(transformedPath);
}

AffineTransform has the factories CreateTranslation, CreateScale, CreateRotation and CreateSkew, and Then combines two transformations: the first one is applied, then the next one. Combine the transformations before calling Transform, so that the pixels are resampled once. The coordinates are those of the stored pixels: the image covers [0, width] x [0, height].

Correct a perspective

CorrectPerspective straightens a document, a whiteboard or a painting photographed at an angle: give the four corners of the quadrilateral in the photo, in order around it, and they become the corners of the image. Without a size, the result is as wide as the longer of the top and bottom edges and as tall as the longer of the left and right edges:

public static Size StraightenDocument(string inputPath, string outputPath, PointD topLeft, PointD topRight, PointD bottomRight, PointD bottomLeft)
{
    using var photo = Image.Load(inputPath);

    // The corners of the page in the photo, in order around it, become the corners of the image: the result is as
    // wide as the longer of the top and bottom edges, and as tall as the longer of the left and right edges. A JPEG
    // photo has no alpha: a corner outside the photo shows an opaque background
    photo.CorrectPerspective(topLeft, topRight, bottomRight, bottomLeft, options: new TransformOptions { BackgroundColor = new Rgba64(65535, 65535, 65535, 65535) });
    photo.Save(outputPath);
    return photo.Size;
}

The corners are in pixel coordinates, where the image covers [0, width] x [0, height]: the top-left corner of the top-left pixel is (0, 0) and its center (0.5, 0.5). The filter, the background color (for corners outside the photo) and the working space come from TransformOptions; its canvas settings are not used.

Transform also applies a ProjectiveTransform, the image seen in perspective. ProjectiveTransform.CreateFromQuadrilateral maps a rectangle to four points, such as the corners of a page in a photo; its inverse maps the page back to a rectangle:

public static void FlattenPage(string inputPath, string outputPath, PointD topLeft, PointD topRight, PointD bottomRight, PointD bottomLeft, Size pageSize)
{
    using var image = Image.Load(inputPath);

    // The page seen in perspective, inverted: from the photo to the page
    var page = new RectangleD(0, 0, pageSize.Width, pageSize.Height);
    var pageToPhoto = ProjectiveTransform.CreateFromQuadrilateral(page, topLeft, topRight, bottomRight, bottomLeft);
    if (!pageToPhoto.TryInvert(out var photoToPage))
        throw new ArgumentException("The corners do not delimit a page.", nameof(topLeft));

    // The canvas is the page: what the photo shows around it is cut
    image.Transform(photoToPage, new TransformOptions { CanvasMode = TransformCanvasMode.Explicit, CanvasSize = pageSize, BackgroundColor = new Rgba64(65535, 65535, 65535, 65535) });
    image.Save(outputPath);
}

An AffineTransform converts implicitly to a ProjectiveTransform.

Straighten a scanned document

A page scanned or photographed at a slight angle has slanted lines of text. Deskew finds their angle and rotates the image back around its center, keeping its size and filling the uncovered corners with white, like paper:

public static double StraightenScan(string inputPath, string outputPath)
{
    using var scan = Image.Load(inputPath);

    // The angle of the lines of text is estimated, and the page rotated back; the corners it uncovers are white
    var skew = scan.Deskew(new SkewEstimationOptions { MaxAngle = 10 });
    scan.Save(outputPath);
    return skew;
}

EstimateSkew returns the angle without changing anything: positive when the lines go down to the right, so Rotate(-angle) straightens them. The angle is the one of the projection profiles: the dark pixels (or the light ones, for light text on a dark background) are counted along the lines of each candidate angle, and the angle where the lines of text fall in the fewest rows wins. SkewEstimationOptions sets the largest angle searched (15 degrees by default) and the precision (0.05 degree). The estimate needs parallel lines of ink, such as text or a table; a page without contrast gives 0, and leaves the image unchanged. Pass TransformOptions to choose another filter, background or canvas.

Estimate a homography from matched points

When the corners of a page are not known exactly, or when two photos of the same flat scene are matched point by point (corners found by DetectCorners, for example), ProjectiveTransform.EstimateHomography computes the projective transformation that maps the points of one onto the points of the other, by least squares, from at least four pairs. Some matches are usually wrong: EstimateHomographyRobust keeps the homography most of them agree on (RANSAC), and tells which pairs are inliers:

public static HomographyEstimate AlignToReference(string photoPath, string outputPath, PointD[] photoPoints, PointD[] referencePoints, Size referenceSize)
{
    using var photo = Image.Load(photoPath);

    // The points of the photo and the points of the reference they match, some of them wrong: RANSAC keeps the
    // homography most of them agree on
    var estimate = ProjectiveTransform.EstimateHomographyRobust(photoPoints, referencePoints, new HomographyRansacOptions { Threshold = 2 });

    // The photo seen from the point of view of the reference
    photo.Transform(estimate.Transform, new TransformOptions
    {
        CanvasMode = TransformCanvasMode.Explicit,
        CanvasSize = referenceSize,
        BackgroundColor = new Rgba64(0, 0, 0, 65535),
    });
    photo.Save(outputPath);
    return estimate;
}
  • Points. Four pairs with no three points on a line give the exact homography; more pairs are averaged. Points on a line determine none, and throw an ArgumentException.
  • Robust estimate. HomographyRansacOptions sets the distance under which a pair is an inlier (3 pixels by default), the number of random samples (1,000) and the seed: the same seed gives the same result everywhere. The HomographyEstimate has the homography, refitted on its inliers, the inliers, and their error.
  • Warp. Pass the homography to Transform with TransformCanvasMode.Explicit: the canvas then has the coordinates of the destination points.
  • Matrix layout. The matrix maps row vectors ([x, y, 1] on the left, the offsets in M31 and M32). Transpose it for an API that maps column vectors. The estimate is the least-squares solution over the inliers, without a further refinement of the distances between the points and their images.

Transform options

TransformOptionsDefaultMeaning
CanvasModeExpandExpand makes the smallest canvas that holds the whole transformed image, with the image centered in it. Keep keeps the size of the image, and Explicit uses CanvasSize: with these two, Transform applies the transformation as given and cuts what lands outside the canvas, and Rotate and Skew center the image on the canvas
CanvasSize-The size of the canvas with Explicit
FilterBicubicThe resampling filter, as for resizing. NearestNeighbor copies pixels without creating new colors
BackgroundColorTransparent blackThe color beyond the edges of the image, into which the edges blend, and of the canvas the image does not cover
WorkingSpaceEncodedEncoded or LinearSrgb, as for resizing

A pixel format without alpha (Rgb24, Gray8, Gray16) needs an opaque BackgroundColor when the result shows it: otherwise the operation throws an UnsupportedImageFeatureException. A transformation that flattens the image to a line, or a perspective whose horizon crosses the image with an Expand canvas, throws an ArgumentException. The canvas is checked against the resource limits before anything is allocated, and the image is unchanged when an operation fails.

Cursor hotspots

The hotspot of a cursor frame (Frames[i].Metadata.Hotspot) follows the pixel it points at: quarter turns, mirrors and AutoOrient move it exactly, and rotations by any angle, skews and transformations move it to the pixel that contains the image of the center of its pixel. When that pixel leaves the canvas, the operation throws an UnsupportedImageFeatureException and leaves the image unchanged: set the hotspot to null or to a pixel that is kept first. See ICO and CUR.