Two images of the same scene that differ only in exposure are related by a transformation on their pixel values.  This is mathematically denoted by g(x) = s(f(x)).

The transformation s is typically nonlinear.  It is derived from the camera's response function r that dynamically compresses the range of possible pixel values.  That is, a pixel value in image f results from integrating light at each sensor element q(x) over the camera's spectral response yielding  f(x) = r(q(x)).  If this integration time is longer or shorter in image g, a proportional amount of light kq(x) results in pixel value g(x) = r(kq(x)).  By solving for q(x) in the relation for f and plugging that into the relation for g yields a so-called comparametric relationship between images:

g(x) = r(kr^-1(f(x))

with s( ) = r(kr^-1( )).

Seminal work in this area has been conducted by MIT graduate Dr. Steve Mann, now with the University of Toronto.  Dr. Mann has established several parametric models which quantify the exposure difference between two images.  A comparametric model which seems to well approximate exposure differences in most digital cameras is the "preferred" model established by Mann.  This relates the pixels of image f and g via

Please note that parameters a and c are intrinsic camera parameters that need only be determined once.  After they are found (see reference 1 below), they are fixed for that camera.  Thus we have a mapping only dependent on k.

IUL Contributions

Our contribution to range registration is seeded in the use of piecewise linear approximations to both the comparametric relation and the comparametric function.  The inspiration for our piecewise linear modeling stemmed from wanting to develop an approach that was friendly to the joint domain and range registration process.  

Contributions

1. Initially, we used a piecewise linear approximation to the comparametric data.  This was then indirectly related to any one of Mann's parameterized comparametric models.  Dr. Mann seemed to like the piecewise linear idea and referred to the paper describing the range registration concept as an "excellent paper".

2. Next, we noted that the piecewise linear approximation we were employing was underconstrained.  The published paper gave the constraints and conditions that resulted in better performance without practically increasing computation cost.

3. Third, and we believe our most significant contribution to range registration, was to cleverly piecewise linearly model the camera response function r such that it could be estimated from data samples of its corresponding comparametric relation.  The elegance of this approach is that the constrained function r could be solved in an unconstrained manner and it does so in a semiparametric way so that it could arbitrarily closely model r without the number of data points affecting the complexity of the model.  The paper describes this approach in detail.

In this third contribution, we can mathematically express the relationship by 

g_PL(x) = r_PL(kr_PL^-1(f(x))

where note that because the piecewise linearly approximated camera response function r_PL, and its inverse r_PL^-1 are both piecewise linear, their functional composition results in g_PL also being piecewise linear.

This type of modeling allows us to capture nuances in the comparametric relation that are difficult to specify when establishing parameterized models.  Though we have in general observed good range registration performance with Mann's parameterized models, the semiparametric approach developed at the IUL allows for extra accuracy when needed.  The comparagram examples below illustrate this.  On the left is a comparametric function estimate using Mann's "preferred" model and on the right is the semiparametric piecewise linear estimate developed at the IUL.

Please refer to the Books example for a better demonstration of the range registration performance achieved.

¹ The paper in question is S. Mann, "Comparametric Equations with Practical Applications in Quantigraphic Image Processing," IEEE Trans. on Image Processing, Vol. 9, No. 8, pp. 1389-1406, 2000.

This page was last updated May 15, 2006 .