Typical image sensors like the ones we use in digital cameras are comprised of many individual photosensors, all of which capture light. These photosensors are natively able to capture the intensity of light but not its wavelength (color). As a result, image sensors are typically overlaid with something called a "color filter array" or "color filter mosaic." This overlay consists of many tiny filters that cover the known pixels and allow them to render color information.
The digital image's processor can decode the color of an area by essentially averaging the color data from the various interpolated color filters and the relative brightness registered by the pixels. One of the most common filter arrangements at work in modern devices is called the Bayer filter.
Bryce Bayer and the Bayer Filter
The Bayer filter was invented in 1974 by Bryce Bayer, an employee of Eastman Kodak. You may sometimes hear Bayer's arrangement of microfilters referred to as RGGB. That's because the arrangement uses a proportion of two green filter elements (GG) for each red (R) and blue (B) filter element. The entire array is spread over a 2x2 block of pixels, and each microfilter covers one-quarter of a pixel, like so:
G R G R
B G B G
G R G R
B G B G
Bayer Filter Design and the Demosaicing Process
Did you know that the human retina is naturally more sensitive to green light in daytime vision? Bayer used this knowledge when he selected his filter proportions ― which favor green light ― in an attempt to mimic our visual perception. He proposed another version of the filter with cyan, magenta, and yellow as the color set, but the necessary dyes didn't exist at the time, so it wasn’t created until much later. The CMY version has a higher quantum efficiency, and you can find it in some newer digital cameras.
Each pixel receives input from all three primary colors, but they are not capable of outputting complete wavelength information since each pixel records only one of the three. Thus, to move from the "Bayer pattern" image to a full-color image, both camera firmware and software can use various algorithms to understand the full color values of each pixel. This process is called demosaicing.
The simplest demosaicing algorithms average the input of nearby pixels to obtain an idea of the full color. Here's an example:
- A pixel recording green may be flanked by two pixels recording blue and two recording red.
- Together, these five total pixels provide information to estimate the full color values for the green.
- Similarly, this complete color value contributes to an estimate of the green value for the blues and reds surrounding it.
This demosaicing technique works well in large areas of constant color or smooth changes, but in high-contrast areas where colors change abruptly, it may result in loss of detail, which can lead to color bleeding and other color artifacts like zippering. More sophisticated algorithms make complex sets of assumptions about the way the color values correlate or about the content of the image to render the colors more accurately.
Summary
The Bayer filter, named for its inventor Bryce Bayer, is a microfilter overlay for image sensors that allows photosensors (which normally only record light intensity) to record light wavelength as well. The Bayer filter is the most common of such filters, and we find it in use in nearly all modern digital cameras. This filter uses a mosaic pattern of two parts green, one part red, and one part blue to interpret the color information arriving at the sensor. Once recorded, digital algorithms are applied to interpolate or "demosaic" the resulting Bayer pattern and turn it into full-fledged color data for the image.