Colorizer

Images of the Russian Empire

Colorizing the Prokudin-Gorskii photo collection

Shown above is an image (right) of combined color channels from Sergei Mikhailovich Prokudin-Gorskii's RGB glass plate negatives (left), which were purchased in 1948 by the Library of Congress.

Project Overview

The goal of this project is to apply image processing techniques to digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible. A simple python script splits the digitized glass plates into 3 images, representing the Red, Green, and Blue image color channels. These images are then combined and algorithmically aligned to form a single, sharp RGB color image.

Image processing techniques:

  • Alignment metrics (image pyramid)
  • Edge detection with Sobel filter
  • Automatic cropping
  • Automatic contrasting

Approach

Single-Scale Implementation

For low-resolution images of approximately 300x300 pixels, I performed an exhaustive search over a window [-15,15] of possible displacements. Using an image matching metric, each displacement was scored using normalized cross-correlation (NCC), which is simply a dot product between two normalized vectors: (image1./||image1|| and image2./||image2||). The red/green color channels with the best scores were selected.

Results

Under each image I list the offsets calculated by the algorithm for the green and red channels.

Cathedral G:(12, 3), R:(5, 2)

Nativity G:(3, 1), R:(7, 0)

Settlers G:(7, 0), R:(14, -1)

Monastery G:(3, 2), R:(-3, 2)

Pyramid Alignment

Exhaustive search becomes prohibitively expensive if the pixel displacement is too large (which is the case for high-resolution glass plate scans), which are approximately 3000x3000 pixels. For a faster search procedure, I implemented an image pyramid, which represents the image at multiple scales (scaled by factors of 2). The processing is done sequentially starting from the coarsest scale (smallest image) and going down the pyramid, updating the estimate through recursive calls to the single-scale implementation.

Approach:

  • Downscale image by a factor of 20
  • Search over possible displacements in a search window
  • Score each displacement with the original blue color channel using NCC
  • Update image with the calculated displacement, multiplied by the scaling factor
  • Recursively call function, halving the search window and scaling factor

Results

River G:(52, -22), R:(107, -55)

Cabins G:(59, 13), R:(131, 18)

City G:(6, 24), R:(22, 32)

Harvesters G:(59, 16), R:(123, 13)

Three Generations G:(53, 14), R:(112, 11)

Train G:(42, 5), R:(86, 32)

Turkmen G:(56, 21), R:(116, 28)

Village G:(64, 12), R:(137, 22)

Self-Portait G:(78, 29), R:(176, 37)

Icon G:(41, 17), R:(89, 23)

Lady G:(51, 9), R:(111, 12)

Emir G:(49, 24), R:(78, -712)

Sobel Filter

In the case of Emir of Bukhara, the images to be matched do not actually have the same brightness values (they are different color channels), likely due to the dominant presence of the color blue, so using NCC produces a blurred result. An alternative apporach is to apply edge detection with the Sobel operator on the original image, and then align the edges using the NCC metric. The operator uses two 3x3 kernels which are convolved with the original image to calculate approximations of the derivatives, and approximate the image gradient.

Applying the Sobel filter to the Emir of Bukhara image yields the following result:

Emir_filtered G:(49, 24), R:(105, 41)

Automatic Contrasting

Automatic contrasting was applied after the combined color channels were generated through the pyramid algorithm. The approach included calculating the median image pixel intensity, multiplying the image matrix by some factor, shifting the image by the difference between the new and original medians, and clipping values outside of a bound [-1, 1]. The result is an image with more contrast, where the luminance or color is greater.

Results

Automatic contrasting applied (right)

Automatic contrasting applied (right)

Acknowledgements

  • CS 194-26 Course Staff at UC Berkeley
  • Professor Alexei (Alyosha) Efros