Overview:
Graphic works often require a video sample to fit in an irregular shape. Such an operation might, for example, have to account for perspective distortions, like in the case where a video needs to fit in a quadrangle that is part of a 3D scene captured from a specific view angle. But in the more general case, for artistic reasons or whenever the exact transformation is unknown, a rectangular video clip may need to fit in a shape defined by a smooth contour.
Challenge:
So for an input of a video file and a target image annotated with a two-dimensional contour, a method is needed that would precisely transform the video edges (rectangle) to the given
contour, then interpolate the rectangular canvas to the irregular canvas enclosed by the
contour.
Off-the-shelf software packages like MS Powerpoint or Adobe Premiere provide tools that are most often very inaccurate and hence insufficient.
Approach:
Coding a piecewise-linear transformer takes two major steps to implement. The first step is an interactive canvas adjustment tool, that allows the user to quickly ensure that the underlying canvas transformation is proper. The second step incorporates an affine warper that would transform each frame, patch-by-patch, and render the video into the target image.
We have found MATLAB® to be a very practical platform for such tasks. Prototyping time is minimal, and documentation of the so many built-in functions including those in the Image Processing Toolbox is one of its kind.
Moreover, there are built in capabilities in MATLAB that even allow for graphic editing to be done in a swift and robust manner. Polyline drawing and editing was found to provide the right tool to maneuver the canvas, mentioned as step one above.
Results:
A cute little tool written in MATLAB is capable of making video-based artwork, such that given video samples squeeze into irregular shapes. The 'video reshaper' is making a piecewise-linear warping of a rectangular canvas into one defined by a free-hand contour.