I decided to jot down some notes made from some algorithms that I previously worked on. These are mostly high level descriptions providing some intuition on how things work.
- Is a method of producing a high quality LDR (low dynamic range) image from a series of LDR images taken at different exposures (bracketed exposure)
- It does not require a conversion step to HDR, e.g. like tone mapping that is needed for a display device to show the HDR image properly
- Global operators for tone mapping – Apply some scaling to the entire image to convert to LDR
- Local operators – Do different things in different regions of the image to preserve detail
- Exposure Fusion uses 3 quality measures — namely ‘contrast’, ‘saturation’ and ‘well-exposedness’
- For contrast – Apply Laplacian Filter
- For saturation – Compute the standard deviation within R,G,B channels
- For well-exposedness – We want to keep pixels which are neither over-exposed (close to 1) nor under-exposed (close to 0) , so we want to keep values close to 0.5 using a Gaussian curve
- We combine the above info into a weight map using multiplication
- If we just apply the weight map to each of the images in the bracketed exposure there will very sharp transitions and undesirable halos etc.
- To counter that, we blend images at multiple resolutions using a pyramid method
- First create a Laplacian pyramid of the images
- Then create a Gaussian pyramid of the weight maps
- Combine at each level
- Finally you end up with one pyramid – Now repeat the sequence of ‘Up sample, blur, combine’ until you get the final image
Panorama Stitching with Ransac Homography:
- Use SURF/SIFT feature detector to get the keypoints for both the images
- Use Flann Matcher to get the matches
- Find the min and max distance between keypoints
- Take matches whose distance is less than some threshold (3 * min distance)
- Find homography using these set of matched points
- A matrix H which transforms vector a to vector b (aH = b, H = b * inverse(a))
- vector a & b contain the 4 points respectively
- use singular value decomposition to solve for this matrix
- Use this matrix to find out how many good matched points are transformed successfully, check ‘inliers’ that are below a threshold value in terms of distance
- Find the matrix that produces the best set of inliers, that is the homography matrix to use
- warp the image using this matrix
An eigenvector v of a linear transformation T is a non-zero vector that, when T is applied to it, does not change direction
- Edge detection filter that accounts for gradients in both the x and y directions.
- 2 filters are used, namely:
Laplacian of Gaussian:
Laplace filter is extremely sensitive to noise, so smoothening it with a Gaussian filter first is good.
Reinhard Tonemapping Operator:
- Summation of log luminance and average
- operator = L ( 1 + L) — small luminance is scaled by 1 and large ones are penalized.
- Has 2 operators – global and local
- Using a simple Gaussian filter causes halos due to large intensity differences.
- To counter this, we applied a penalty on the intensity difference.
- So we start with a spatial Guassian, then a penalty Gaussian on the intensity difference.
- This is done by creating patches of the image around a certain pixel.
- The spatial Gaussian is taken of the patch, and the penalty Gaussian is calculated using the intensity difference between the rest of the patch and the current pixel.
Set of operations:
- Input is the RGB values of radiance, possibly in radiance format
- Compute the intensity (I) at each pixel. E.g. use a weighted average: 1/61*(R*20 + G*40 + B)
- Compute the chrominance channels: R/I, G/I, B/I
- Compute the log of the intensity: L = log2(I)
- Apply bilateral filter: B = bf(L)
- Compute the detail layer: D = L-B – this should contain the edge information
- Apply an offset and a scale to the base: B’ = (B-o)*s
- Reconstruct the log intensity: O = exp(B’+D)
- Put the colors back: R’,G’,B’ = O*(R/I, G/I, B/I)
- Apply a gamma compression
Scale Invariant Feature Transform (SIFT):
- Construct a scale space
- each octave is for one size and then many levels of blur within each octave
- LoG approximation
- within octave subtract pairs to get LoG
- Find keypoints
- find maxima/minima
- find subpixel maxima/minima
- check if point is min or max among the neighbors around and on top and bottom layer in scale space
- Filter keypoints
- Keypoint orientation
- flat both gradients are small, edge – one is big other is small, corner – both are big
- gradient magnitudes and orientation – using a histogram – 360 deg into 36 bins
- Feature generation
- 16×16 window around a keypoint, 4*4 squares within this – calc grad and ori magnitudes within each 4×4 and add to a histogram
Fast library for Nearest neighbor search – k nearest neighbors