Color Differential Geometry or the Spatial Color Model

Introduction

This webpage gives an overview of color scalespace or the spatial color model as it can be used in digital image processing, from digital microscopy and High Content Screening (HCS) to medical imaging and astronomy.
Although this article is limited to 2D, the spatial color approach can be used for 3D and even 4D image processing. The basic principle is "what you describe is what you get".
For an introduction on differential geometry see differential geometry. The spatial color model and linear and non linear scalespace can be combined to select shapes with specific colors.

Spatial color model

E, El and Ell versus lambda
The graph shows the wavelength lambda (l) x-axis and the color derivatives, up to the second order.
E (intensity) is the 0th order derivative, El is the 1st order derivative to f and Ell is the 2nd order derivative to f (f, frequency).
The spectrum is probed with a Gaussian, centered on 520 nm and with sigma 55.0.
Note: the color derivatives are calculated relative to frequency, but plotted to wavelength for convenience.

The 0th order (E) derivative measures the total weight of the light spectrum and corresponds closely to the luminositiy curve as defined by the CIE 164 standard (standard human observer).
The 1st order derivative (El) of the spectrum measures yellow-ish minus cyan-ish and the 2nd order derivative (Ell) measures green-ish minus magenta-ish.

Spatial color model derivatives

Introduction of spatial extent in the Gaussian color model yields a local Taylor expansion at wavelength lambda0 and position x0.
The spatial Gaussian color model measures up to the 2nd order in the spectral axis, the coefficients of the Taylor expansion of the Gaussian smoothed spatio-spectral energy distribution.

Spatial color probe

Each measurement of a color signal has a spatial (x,y) as well as a spectral (lambda) resolution (scale, sigma). One probes an energy density volume in a three-dimensional spatio-spectral space, where the size of the probe is determined by the observation scale sigmax, sigmay and sigmalambda.
The probe for spatial color is constructed by probing the product of the spatial and the spectral space with a Gaussian aperture.

Behavior of the derivatives

Color Scalespace

The following images illustrate the result of using and combining the derivatives in order to select certain colors. The sigma used for the calculations is 1.

Color Grid First order color derivative El Second order color derivative Ell
RGB color grid showing the primary colors red, green and blue and their combinations. Yellow being a mix of red and green, cyan being a mix of green and blue and magenta being a mix of red and blue.
The images show a color grid and its first (El) an second order (Ell) color derivative, the Gaussian derivative was calculated with a sigma of 1.
The derivatives are divided by the original image E for normalisation of the first order color derivative (El/E) and of the second order derivative (Ell/E).
The normalisation makes the result independent of the color intensity.

Color Grid Positive first order color derivative El/E Negative first order color derivative El/E
The second image on the left shows where the normalised first order color derivative (El/E) is positive (white), which selects the colors red, yellow and green. The image on the right shows where the normalised first order color derivative (El/E) is negative (white), which selects the colors cyan, blue and magenta.

Color Grid Positive second order color derivative Ell/E Negative second order color derivative Ell/E
The second image on the left shows where the normalised second order color derivative (Ell/E) is positive (white), which selects the colors red, blue and magenta. The image on the right shows where the normalised second order color derivative (Ell/E) is negative (white), which selects the colors yellow, green and cyan.

Color Grid Ell added to El Ell added to El is positive
The second image on the left shows the result of adding El to Ell. The image on the right shows where the result of this addition is positive (white), which selects the colors red, yellow and magenta.

Color Grid Ell subtracted form El Ell subtracted form El is positive
The second image on the left shows the result of subtracting Ell from El, the image on the right shows where the result of this subtraction is positive (white) which selects the colors yellow, green and cyan.

El divided by Ell Positive El divided by Ell Negative El divided by Ell
The images show the result of dividing El by Ell and the regions in which the result is positive (second left, white), which selects the colors red and cyan. The image on the right shows where the result of this subtraction is negative, which selects the colors yellow, green, blue and magenta. El/Ell or Ell/El as such describe the hue.

Color Scale Space and Linear Scalespace combined

Colored blobs
By combining the color scalespace with linear scalespace it is possible to selectively detect colored spots.

The following formulas, which are a combination of linear scalespace and color scalespace will detect colored elliptic patches on a darker background. The detection is independent of the intensity of the colored spot over a whide range of intensities. Other formulas for color detection are possible, this list only gives a basic overview. Scale, sigma, is 2.0 in all cases.
Other combinations of color scalespace and linear scalespace are also possible, eg. line detection.

Red:
Lww<0 and LvvLww-Lvw2>0 and El>0 and Ell>0

Yellow and Green:
Lww<0 and LvvLww-Lvw2>0 and El>0 and Ell<0

Green:
Lww<0 and LvvLww-Lvw2>0 and El>0 and Ell<0 El+Ell<0

Yellow:
Lww<0 and LvvLww-Lvw2>0 and El>0 and Ell<0 El+Ell<0

Cyan:
Lww<0 and LvvLww-Lvw2>0 and El<0 and Ell<0

Magenta and Blue:
Lww<0 and LvvLww-Lvw2>0 and El<0 and Ell>0

Magenta:
Lww<0 and LvvLww-Lvw2>0 and El<0 and Ell>0 and El+Ell>0"

Blue:
Lww<0 and LvvLww-Lvw2>0 and El<0 and Ell>0 and El+Ell<0"


Red, Yellow and Green:
Lww<0 and LvvLww-Lvw2>0 and El>0

Blue, Cyan and Magenta:
Lww<0 and LvvLww-Lvw2>0 and El<0

Red, Blue and Magenta:
Lww<0 and LvvLww-Lvw2>0 and Ell>0

Green, Yellow and Cyan:
Lww<0 and LvvLww-Lvw2>0 and Ell<0


Red, Yellow and Magenta:
Lww<0 and LvvLww-Lvw2>0 and El+Ell>0

Blue, Green and Cyan:
Lww<0 and LvvLww-Lvw2>0 and El+Ell<0

2D color edge detection

Toy Color Image Color edge detection
Color Canny edge detection, sigma (scale) of the Gaussian in this case was 2.
The color canny edge detector was used with shadow and highlight invariance (H), and no non maxima suppression.

The main advantage of the El, Ell color space is the color differentiatibility, as shown here.

Several invariants for the detection of color edges are defined.
The following color invariants are currently available:
H = shadow and highlight invariance (White)
N = illumination color and shadow invariance (Colored)
C = shadow invariance (WhiteMatte)
W = intensity normalization (WhiteUni)
E = not invariant for anything (NoInvar)

Sensitivity of the invariants in table form:
Geometry
shadow
Highlight Illumination
profile
Illumination
intensity
Illumination
color
Dominant color (Hue)H----+
Object colorN-+---
C-+--+
Normalised colorW+++-+
Spectral colorE++ +++

where '+' denotes sensitivity, and '-' invariance.

The invariants are orderable on both degree of invariance and (opposite) dicriminatory power:
Discriminative power E > W > C > N > H as opposed to their invariance.

Hints and tips:
For bright-field microscopy, N is the best choice for color edge detection, since it has high discriminative power, but is invariant to illumination distribution (unequal background) and illumination color.

Application of spatial color model on microscopy

Detecting edges in mouse tail section

Skin section 4000K Skin section 4000K edges
Skin section 2600K Skin section 2600K edges
Influence of illumination color temperature on edge strength, scale (sigma) is 3.0 .
Mouse tail, skin tissue section illuminated by a halogen bulb at 4000K (top) and 2600K (bottom) color temperature.

Mouse tail section stained with hematoxylin and eosin.
By chosing the right invariant Canny color edge detector (N, illumination color and shadow invariance), the detected color edges are very robust (invariant) to changes in illumination color tempearture.

Separating colors in fluosphere preparation

TetraSpeck beads TetraSpeck beads - red
TetraSpeck beads - green TetraSpeck beads blue TetraSpeck beads orange
Separation of four colors of fluosphere beads.

Four color separation of fluospheres by solving a system of differential equations for each color, sigma is 1.0. In this case the spatial color is being used as the sole discriminator without additional spatial constraints, i.e. selection is on spatial color only.

Four separate exposures of three TetraSpeck beads (TetraSpecktm microspheres, 4.0 um, fluorescent blue/green/orange/dark red) photographed using optical filter sets appropriate for DAPI, fluorescein, rhodamine and Texas Red dye. The stage was shifted after each exposure.
The same beads appear blue, green, orange or red, depending on the filters used.

Selecting color in fluorescent stained nuclei

Micronuclei test Micronuclei test - selection
Detection of fluorescent blue stained nuclei.

Detection of fluorescent blue stained nuclei, sigma is 2.0. In this case the spatial color is being used as the sole discriminator without additional spatial constraints, i.e. selection is on spatial color only.

References

For additonal information you may also contact my former colleague Jan-Mark Geusebroek (UvA) or Prof. Bart M. ter Haar Romeny (T.U. Eindhoven).

See also

Acknowledgments

I am indebted, for their pioneering work on automated digital microscopy and High Content Screening (HCS) (1988-2001), to my former colleagues at Janssen Pharmaceutica (1997-2001), such as Frans Cornelissen, Hugo Geerts, Jan-Mark Geusebroek, Roger Nuyens, Rony Nuydens, Luk Ver Donck and their colleagues.

Many thanks also to the pioneers of Nanovid microscopy at Janssen Pharmaceutica, Marc De Brabander, Jan De Mey, Hugo Geerts, Marc Moeremans, Rony Nuydens and their colleagues. I also want to thank all those scientists who have helped me with general information and articles.


The author of this Webpage is Peter Van Osta.