The Perception of Colors

In our last post on [»] absorbance spectroscopy, we measured a few copper (II) chloride solutions and noticed they each had their respective colors, due to solvatation effect. The measured spectra are given in Figure 1 with approximate color as they appear to the user. In that post, we discussed a lot about how molecules absorb light and how transition metals produce colored solution by absorbing light in the visible range. However, we did not discuss the fundamental concept of “what are colors exactly?”. That’s exactly what we will do in this post.

Interestingly, the notion of color is not so well described in conventional textbooks on either general optics or spectroscopy despite many renowned scientists experimented with them, such as James Clerk Maxwell, Newton or Thomas Young and Hermann von Helmholtz. While most people will credit Newton for his discovery that white light is composed of a rainbow of colors, much less know that he also used multiple prism systems to recombine different monochromatic light to reproduce colors.

In this post, we address the question of colors and, more specifically, we study how it is possible to reproduce on a computer display a color patch from measured spectra only. Some experimental results are already shown in Figure 2 which compare a photography of the cuvette holding the liquids of Figure 1 with a color patch reproduced directly from their spectra. Note the good agreement between the reproduced patch and the photography. An open-source computer program is given at the end of the post to convert spectra from the [∞] SpectrumAnalyzer program to colored patch and to synthetic color spectra.

When we shine white light through each of the solutions of Figure 2, we obtain a filtered spectrum that depends on the absorption of each of the solutions. This filtered light contains many photons of different wavelengths which ultimately hit our retina and produce an electric signal that is interpreted by our brain as color. And while the process from which photons are converted into an electric signal is relatively well understood, the cognitive process of how the brain interprets these electric signals is much less known. What is certain is that a “normal” (non-color-blind) human beings will perceive red, green, blue and yellow colors with variations between them. Not all variations exist though and while it is possible to talk about a greenish-yellow, it is not possible to talk about a greenish-red. Red is considered as opposite to green and yellow is opposite to blue. Mixing them will result in a decrease of chromaticity and, in the good proportion, to pure achromatic response (i.e. gray). In fact, Maxwell used the opposition process to quantify the color mixing phenomena.

The process that occurs in our retina is a bit complicated as colored light is first decomposed into electric stimuli through three types of photopigments cells called cones. At contrario to what many people think, the eye is not exactly sensitive to red, green and blue light but to something slightly different (in particular in the red region). The spectral sensitivities of the retina as standardized by the Commision Internationale de l’Eclairage (abbreviated CIE) are given in Figure 3. They are referred to as the LMS sensitivities for long (L), medium (M) and short (S) wavelengths. It is worth noting that our brain, however, does not see colors in terms of LMS because these signals are further processed by deeper cells in the retina to yield an opponent-color model in the fashion of the green/red and yellow/blue trend.

Another important piece of information about the LMS spectra of Figure 3 is that they are not the sensitivity of the photopigments themselves. The eye has a yellowing effect on the incoming light, which increases with age, and a thin layer of blood also tints the light before it reaches the photopigments. Also, not everyone has the same photopigments as there exists two different genetic protein variation for each cone type. Historically, these curves were not obtained directly but as a derivation from experimental data involving both people with normal retina and people lacking either L, M and S photopigments (color-blind).

When the CIE standardized the human color response process in 1931, most of the incentive was to achieve consistency in industrial color reproduction – not necessarily to characterize the human physiological process itself. They based their 2° standard observer data on the experiments of Guild and Wright to produce the standardized spectral sensitivities of Figure 4 which are still used today and can be downloaded on the [∞] CIE website. Further work refined these results to include things like larger fields of view but the 1931 standard is the most widely used despite the fact that the original data collected by Guild and Wright used different experimental protocol and included a total of only 17 people in total.

Each spectrum, such as those of Figure 1, can be decomposed into a tristimulus value using either the LMS or the XYZ data using the inner product of the spectra and the XYZ functions (here using the XYZ spectral functions):

where s(λ) is the spectrum to be decomposed and x(λ), y(λ) and z(λ) are the standardized spectra of Figure 3.

It derives from this representation that any linear combination of the x(λ), y(λ) and z(λ) is also valid:

with similar expressions for Y’ and Z’.

The LMS functions are nothing but a linear combination of XYZ functions and vice-versa. The Luther-Yves conditions state that any linear combination of the XYZ functions produces a physiologically accurate set of tristimulus values. Going from one to the other can be done using a matrixial representation:

where

and

There exist other sets of functions, such as the RGB functions, but the most used are the XYZ ones. Historically, the RGB were computed first but had unpleasant mathematical properties for computations at that time, so the CIE looked for a linear combination of the RGB functions that would have convenient mathematical properties, leading to the XYZ set.

Any color, light or dark, can therefore be plotted in a 3-dimensional space made of X, Y and Z tristimulus. Lightness is proportional to how large the XYZ vector is and its color, or chromaticity, is given by the direction of the vector. Since this post is about color, we will focus on the direction of the vector and not on how large or small it is.

A convenient way to study the direction of the tristimulus vector in the XYZ space is to find its intersection with a plane of equation X+Y+Z=1. An example is given in Figure 5 with the XYZ tristimulus space, the plane passing through the points (1,0,0), (0,1,0) and (0,0,1) and two experimental vectors exp and n intersecting this plane. Figure 5 also displays a curvy colored surface to which we will focus our attention on in the rest of this post.

First, it is important to mention that not all XYZ points in the XYZ space exist. For instance, if we look at Figure 4, we see that it is impossible to have a photon triggering response in the x(λ) sensitivity curve but not in the y(λ) sensitivity curve. The realm of possible, physical, colors is bounded by the spectral locus and the purple line shown in Figure 5.

The spectral locus is the path taken by pure monochromatic rays ranging from violet to red. The purple line closes the graph from violet to red and has the property that no single monochromatic ray can ever reproduce one of its colors. Note that although the two terms violet and purple are often wrongly used interchangeably, violet is a real monochromatic color whereas the various shades of purple are not. The misconception probably originates from the fact that purples are used to mimic violet in computer displays.

One of the advantages of computing the intersection of the vector with the X+Y+Z=1 plane is that it is possible to reduce it to a 2-dimensional problem through the xy notation:

Since we lost the magnitude of the vector, the Y component is often appended to the (x,y) coordinates yielding the (x,y,Y) coordinates system which is another representation of the (X,Y,Z) tuple. These notations are often shortened as xyY just like we shortened (X,Y,Z) to XYZ when talking about a vector or space itself.

It is possible to go from xyY to XYZ using the following equations:

Many other formula like these ones can be found on the remarkable website of [∞] Bruce Lindbloom.

Up to now, we discussed a lot about how to compute tristimulus values from a given spectrum but we did not address the process of color reproduction itself. Remember that our goal is to produce a colored patch on a computer display that looks like the real color, just based on its spectrum measured with a tool like our [»] 350-700 nm spectrometer. To understand the color reproduction process, we need to address one specific property of the stimulus decomposition process which is called metamerism.

Because we are transforming an information-rich spectrum into a 3-dimensional tristimulus value, some information about the spectrum is lost. It is therefore possible to have two different spectra s(λ) and s’(λ) that have the same XYZ tristimulus value:

Such a pair of spectra, s(λ) and s’(λ), is said to be metameric in reference to [∞] isomers in chemistry.

A striking example of metamerism in everyday life is approximated with white light. If you go shopping at your nearest convenience store, you will find different lamp technologies that all roughly produce the same color (white) despite the fact that they have very different emission properties which supposedly mimic natural light. This phenomenon has such large consequences that the CIE also published standard illuminants spectra with some shown in Figure 6 (FLx are fluorescent lights, D65 is broad daylight and LDx are LEDs light). If you compute the XYZ values for the three spectra in Figure 6 you will obtain very close results. Note that the spectra of Figure 6 are not exactly metameres because they don’t give exactly the absolute same XYZ values, but they’re close to it.

A second important property that is required to understand color reproduction is the additivity of XYZ stimuli. Just like a linear combination of the x(λ), y(λ) and z(λ) are a valid color space, linear combinations of spectra s_i(λ) are also possible:

with

and

The color reproduction process consists in finding values of the weights ω_i given a set of spectra s_i(λ) such that its tristimulus XYZ match the one of an experimental spectrum s’(λ). We call each of the s_i(λ) functions a primary color of the color reproduction system. In television and display systems, these primary colors are typically LEDs whose intensities can be modified on-demand by controlling the amount of current going through them, which then plays the role of the weight factor.

Nearly all colors can be reproduced that way. Let’s dig into the process of mixing different primary colors more in-depth using the xyY color space.

Assume we have two primary colors whose spectra are s₁(λ) and s₂(λ) and have the coordinates (x₁,y₁) and (x₂,y₂) in the xyY plane. Given the weights ω₁ and ω₂ we can compute the coordinate (x,y) of the mix s(λ)= ω₁s₁(λ)+ ω₂s₂(λ).

To ease up computations we introduce the pseudo-weights, w^’_i, and the total pseudo-weight, Ω,

We can now compute the coordinate x using the definition (a similar work is done for y):

or

with

which is nothing but the equation of a line going from (x₁,y₁) to (x₂,y₂).

This is illustrated in Figure 7. It shows that, using two primary colors, we can adjust their respective weights (e.g. by tuning the intensity of the emitter) to yield any color that fits on the line between the two primaries.

When using three primary colors, we obtain a subset of the color space that has the shape of a triangle. It’s the typical process used by television and other displays to reproduce colors. The process generalizes to any convex hull made using N primaries although, to my knowledge, only [∞] Sharp tried to implement such technology with not so much success.

Using our [»] 350-700 nm spectrometer, I measured the three primaries of my Samsung SyncMaster T24B350 display. The results are shown in Figure 8 and roughly correspond to red, green and blue light although I have strictly no idea why there is a small bump in the blue channel near 510 nm. Note that my spectrometer is not calibrated in intensity and does not compensate for the varying quantum efficiency of the silicon sensor so the results are only approximately correct. Nonetheless, it is good enough for the educational purposes of this post.

From the spectra of Figure 8, I computed the XYZ tristimulus values and then the xy chromaticity components of each of the colors labeled here (x_r,y_r), (x_g,y_g) and (x_b,y_b) for the red, green and blue primaries respectively. The primaries are reported on a xyY chromaticity chart in Figure 9. Any colors lying inside of the triangle can be reproduced by this display by adjusting the intensities (weights) of the primaries. This addressable sub-space of the complete physiological color space is called the gamut of the display, borrowed from the Latin word describing the full set of notes in music.

It is immediately apparent in Figure 9 that a lot of colors cannot be reproduced by my Samsung display (all colors outside of the display gamut). In particular, no points in the spectral locus or purple line can be reproduced at all. This is the reason a picture of a rainbow on your phone or computer is never as vivid as the original phenomenon captured by the human eye.

The limited gamut of the Samsung display directly comes from the spectral broadness of its primaries which make their xy component go away from the spectral locus of pure monochromatic colors. Not all displays have such a narrow gamut, for instance my Acer AL2623W has a much larger addressable color space. A display with a larger color sub-space is often described as wide gamut although not all wide gamut displays are made equals. Some cinema projectors use lasers to produce primaries on the spectral locus and increase their gamut, but these systems are not scalable to consumer-level displays. A lot of hope is put into [∞] quantum dots which are probably going to be the future of consumer electronics for wide gamut displays and home cinema.

A consequence of the color reproduction process is that, by definition, the complete color space can only be reconstructed by using an infinite number of primaries arranged on the spectral locus and forming a convex hull equal to the initial, full, physiological color space. In practice, because human eye cannot resolve nearby colors, about 100 primaries would already be enough in this process. This is still impractical, and we have to deal with the fact that no current system can reproduce all the colors that nature (physics) can offer to the human eye.

Let’s go back to our color reproduction goal. From Figure 1, we can compute XYZ components that we then project into the xyY chromaticity space. The question is now how to compute the weights of the primaries of Figure 9 so that we can address the display to reproduce the color patch that we want.

We can rewrite our XYZ stimulus in matrix form as

where the (X_i,Y_i,Z_i) can be obtained from the (x_i,y_i) notation and arbitrarily setting Y_i to 1.

In this process, it is also customary to scale the matrix such that the neutral point (X_n,Y_n,Z_n) corresponds to ω_r=ω_g=ω_b=1. This neutral point is what we would call white although the notion of white is vague because our brain can accommodate to change in illuminants. For the analysis of light sources, CIE illuminant E is used with coordinates (1,1,1) which gives (¹/₃,¹/₃) in the xy space.

The scaling coefficients are found using

leading to the final system

and the reverse operation

Using this latter matrix allowed me to display a colored patch for the spectra of Figure 9 that is accurate to a human observer provided it is within the display gamut. Points outside of the gamut are not accurately rendered because they fall beyond the representation capabilities of the display and no combination of his primaries can ever render the color accurately. Different methods exist to deal with the case of out-of-gamut points such as clipping the point to the device gamut boundaries or scaling the complete scene color space range to fit within the device gamut. The former method is said to be colorimetric and the latter perceptual. Colorimetric is best in our case and is the technique I used to render all the chromaticity diagrams here, but perceptual is better to deal with images such as in photography.

So, are we done? Not yet.

Until now, the patches are displayed correctly on my Samsung display, but what about yours, or what about my Acer display that has completely different primaries?

We would have to repeat the process using each display primaries coordinates to find the proper weights based on the xy coordinates obtained from the spectra. In practice, this could be achieved by having each display aware of its own conversion matrix and provide XYZ tristimulus values to the user in the form of images and let the display do the job. That’s almost how it’s done today, but a touch differently (quite a lot in fact!).

Historically, people didn’t care too much about accurate color reproduction and everybody was using CRT display anyway that roughly behaved the same way. So instead of packing XYZ data into images, people directly packed the weights known as sRGB values for the red, green and blue primaries of the CRT displays. So, some kind of standard primaries were needed to perform the computations from XYZ to sRGB space. The computer industry chose the primaries of the Rec 709 standard which reproduced quite well the CRT primaries. Figure 10 shows the sRGB gamut compared to the measured gamut of my Samsung display. You can see it’s relatively similar but not exactly the same. It’s relatively typical for modern, low-end, computer displays.

To reproduce accurate colors from a sRGB image, the computer first translates the sRGB data into XYZ data (note: it’s actually in CIELab format – a non-linear derivative of CIE XYZ space – but which goes beyond the scope of this post) and the XYZ data is then translated into proper weights for the display being used. This requires the computer to know about the gamut of the display which is performed using either a default profiling information given by the display manufacturer (e.g. Samsung, Acer, Sharp…) or much better, by profiling your exact display using a colorimeter (a kind of tiny spectrometer). Most photographers and nearly all professional colorists profile their display frequently to get the most accurate colors possible. By the way, not all programs do this translation process and those who do are said to be color-aware. Devices that are not color aware just send the raw sRGB data to the display which may completely distort the color rendering depending on their actual primaries.

sRGB was good enough for the emergent Internet stuff but falls short for serious jobs because of its relatively limited gamut. Newer color spaces were invented such as AdobeRGB, DCI-P3, Rec 2020 or ProPhotoRGB which all have extended gamuts. Some of these color spaces are physically realizable, such as AdobeRGB and DCI-P3) and some are just theoretical because they require imaginary primaries and have the sole purpose of proposing to store data in a larger gamut space. An example of such sub-spaces is the ProPhotoRGB one. Rec 2020 falls a bit in-between because it’s the new cinema standard designed to replace Rec 709 and is technically feasible but doesn’t have any implementation yet (i.e. you cannot buy yet a Rec 2020 display – at least at the time of this writing). Much hope is placed around q-dots to make Rec 2020 accessible for consumer devices.

Note that historical formats like .bmp and .gif didn’t care too much about accurate color reproduction and therefore have no indication on the color space they’re using. Newer formats like .png and .jpeg have the option to embed information about the color space they are using. When no color space information is specified or exists, they’re usually processed as being in sRGB space. I say usually because, in some cases, programs will not perform any attempt to convert the data to the user display gamut and will directly use the weights as-is, yielding incorrect color rendering.

The color patch of Figure 2 have been explicitly encoded with a sRGB profile to display as best as possible on your own display. It doesn’t mean however that the colors are physically accurate because the second and third patches fall outside of the sRGB gamut. I could have encoded the image in a larger color space such as AdobeRGB or DCI-P3 which would have probably rendered well on your iPhone or iMac, but at the risk that people with less expensive devices would have a buggous color rendering because their browser would not be color-aware and would distort the extended gamut to fit into the device specific primaries. By sticking to the sRGB color space, you maximize the compatibility of your images with the rest of the Internet – so it was a safe choice but with the disadvantage of producing less vivid colors and many people choose to stick to that solution. Nonetheless, the conversion between spectra to color patch can be considered as functional as can be judged on Figure 2.

More can be done once you can convert spectral information to sRGB color. For instance, Figure 11 shows both a colored patch and a simulated colored spectral image like we would see if we had looked at the spectrum through a prism or using a color camera. You can download an open-source Matlab program to produce the results of Figure 11 [∞] here under a GPLv3 license. The program is compatible with the .csv files generated by our SpectrumAnalyzer program and is provided with a few sample files of both Raman spectra and spectra from the [»] 350-700 nm spectrometer.

We therefore have achieved our goal of reproducing colored patch from raw spectra but a lot of information has been left aside such as how to compare colors and the concept of just-noticeable-differences (jnd), uniform colors spaces such as CIELAB or CIELUV, how illuminants affects metamerism and how cameras perceive the world (hint: they do not respect 100% of the Lüther-Yves conditions!). If you’d like to dig into these concepts, I can recommend the books “Colorimetry: Fundamentals and Applications” by Otha and Robertson and “Standard Colorimetry: Definitions, Algorithms and Software” by Claudio Oleari (the latter is a bit more challenging to read). If you’d like to understand more about ICC profiles and gamut management in color-aware programs, you can look at secondhand versions of “Understanding Color Management” by Abhay Sharma, former member of the ICC consortium.

I hope this post has been useful for you :) don’t hesitate to share your comments on the [∞] community board to let me know!

I would also like to give a big thanks to Young, Naif, Sebastian, James, Lilith, Alex, Stephen, Jesse, Jon, Cory, Sivaraman, Karel, Themulticaster, Tayyab, Samy, David, Marcel, Kirk, Michael, Shaun, Dennis, M and Natan who have supported this post through [∞] Patreon. I also take the occasion to invite you to donate through Patreon, even as little as $1. I cannot stress it more, you can really help me to post more content and make more experiments!