aesthetic appeal

The Retinex algorithm for beautifying pictures

/ spqr / Leave a comment

There are likely thousands of different algorithms out in the ether to “enhance” images. Many are just “improvements” of existing algorithms, and offer a “better” algorithm – better in the eyes of the beholder of course. Few are tested in any extensive manner, for that would require subjective, qualitative experiments. Retinex is a strange little algorithm, and like so many “enhancement” algorithms is often plagued by being described in a too “mathy” manner. The term Retinex was coined by Edwin Land [2] to describe the theoretical need for three independent colour channels to describe colour constancy. The word was a contraction or “retina”, and “cortex”. There is an exceptional article [3] on the colour theory written by McCann which can be found here.

The Retinex theory was introduced by Land and McCann [1] in 1971 and is based on the assumption of a Mondrian world, referring to the paintings by the dutch painter Piet Mondrian. Land and McCann argue that human color sensation appears to be independent of the amount of light, that is the measured intensity, coming from observed surfaces [1]. Therefore, Land and McCann suspect an underlying characteristic guiding human color sensation [1].

There are many differing algorithms for implementing Retinex. The algorithm illustrated here can be found in the image processing software ImageJ. This algorithm for Retinex is based on the multiscale retinex with colour restoration algorithm (MSRCR) – it combines colour constancy with local contrast enhancement. In reality it’s quite a complex little algorithm with four parameters, as shown in Figure 1.

Fig.1: ImageJ Retinex parameters
  • The Level specifies the distribution of the [Gaussian] blurring used in the algorithm.
    • Uniform treats all image intensities similarly.
    • Low enhances dark regions in the image.
    • High enhances bright regions in the image.
  • The Scale specifies the depth of the Retinex effect
    • The minimum value is 16, a value providing gross, unrefined filtering. The maximum value is 250. Optimal and default value is 240.
  • The Scale division specifies the number of iterations of the multiscale filter.
    • The minimum required is 3. Choosing 1 or 2 removes the multiscale characteristic and the algorithm defaults to a single scale Retinex filtering. A value that is too high tends to introduce noise in the image.
  • The Dynamic adjusts the colour of the result, with large valued producing less saturated images.
    • Extremely image dependent, and may require tweaking.

The thing with Retinex, like so many of its enhancement brethren is that the quality of the resulting image is largely dependent on the person viewing it. Consider the following, fairly innocuous picture of some clover blooms in a grassy cliff, with rock outcroppings below (Figure 2). There is a level of one-ness about the picture, i.e. perceptual attention is drawn to the purple flowers, the grass is secondary, and the rock, tertiary. There is very little in the way of contrast in this image.

clover in grass
Fig.2: A picture showing some clover blooms in a grassy meadow.

The algorithm is suppose to be able to do miraculous things, but that does involve a *lot* of tweaking the parameters. The best approach is actually to use the default parameters. Figure 3 shows Figure 2 processed with the default values shown in Figure 1. The image appears to have a lot more contrast in it, and in some cases features in the image have increased their acuity.

Fig.3: Retinex applied with default values.

I don’t find these processed images are all that useful when used by themselves, however averaging the image with the original produces an image with a more subdued contrast (see Figure 4), having features with increased sharpness.

Fig.4: Comparing the original with the averaged (Original and Fig.3)

What about the Low and High versions? Examples are shown below in Figures 5 and 6, for the Low and High settings respectively (with the other parameters used as default). The Low setting produces an image full of contrast in the low intensity regions.

Fig.5: Low
Fig.6: High

Retinex is quite a good algorithm for dealing with suppressing shadows in images, although even here there needs to be some serious post-processing in order to create an aesthetically pleasing. The picture in Figure 7 shows a severe shadow in a inner-city photograph of Bern (Switzerland). Using the Low setting, the shadow is suppressed (Figure 8), but the algorithm processes the whole image, so other details such as the sky are affected. That aside, it has restored the objects hidden in the shadow quite nicely.

Fig.7: Photograph with intense shadow
Fig.8: Shadow suppressed using “Low” setting in Retinex

In reality, Retinex acts like any other filter, and the results are only useful if they invoke some sense of aesthetic appeal. Getting the write aesthetic often involves quite a bit of parameter manipulation.

Further reading:

  1. Land, E.H., McCann, J.J., ” Lightness and retinex theory”, Journal of the Optical Society of America, 61(1), pp. 1-11 (1971).
  2. Land, E., “The Retinex,” American Scientist, 52, pp.247-264 (1964).
  3. McCann, J.J., “Retinex at 50: color theory and spatial algorithms, a review“, Journal of Electronic Imaging, 26(3), 031204 (2017)

The photography of Daidō Moriyama

/ spqr / Leave a comment

Daidō Moriyama was born in Ikeda, Osaka, Japan in 1938, and came to photography in the late 1950s. Moriyama studied photography under Takeji Iwamiya before moving to Tokyo in 1961 to work as an assistant to Eikoh Hosoe. In his early 20’s he bought a Canon 4SB and started photographing on the streets on Osaka. Moriyama was the quintessential street photographer focused on the snapshot. Moriyama likened snapshot photography to a cast net – “Your desire compels you to throw it out. You throw the net out, and snag whatever happens to come back – it’s like an ‘accidental moment’” [1]. Moriyama’s advice on street photography was literally “Get outside. It’s all about getting out and walking.” [1]

In the late 1960s Japan was characterized by street demonstrations protesting the Vietnam War and the continuing presence of the US in Japan. Moriyama joined a group of photographers, associated with the short-lived (3-issue) magazine Provoke (1968-69), which really dealt with elements of experimental photography. His most provocative work during the Provoke-era was the are-bure-boke style that illustrates a blazing immediacy. His photographic style is characterized by snapshots which are gritty, grainy black and white, out-of-focus, extreme contrast, Chiaroscuro (dark, harsh spotlighting, mysterious backgrounds). Moriyama is “drawn to black and white because monochrome has stronger elements of abstraction or symbolism, colour is something more vulgar…”.

“My approach is very simple — there is no artistry, I just shoot freely. For example, most of my snapshots I take from a moving car, or while running, without a finder, and in those instances I am taking the pictures more with my body than my eye… My photos are often out of focus, rough, streaky, warped etc. But if you think about I, a normal human being will in one day receive an infinite number of images, and some are focused upon, other are barely seen out of the corners of one’s eye.”

Moriyama is an interesting photographer, because he does not focus on the camera (or its make), instead shoots with anything, a camera is just a tool. He photographs mostly with compact cameras, because with street photography large cameras tend to make people feel uncomfortable. There were a number of cameras which followed the Canon 4SB, including a Nikon S2 with a 25/4, Rolleiflex, Minolta Autocord, Pentax Spotmatic, Minolta SR-2, Minolta SR-T 101 and Olympus Pen W. One of Moriyama’s favourite film camera’s was the Ricoh GR series, using a Ricoh GR1 with a fixed 28mm lens (which appeared in 1996) and sometimes a Ricoh GR21 for a wider field of view (21mm). Recently he was photographing with a Ricoh GR III.

“I’ve always said it doesn’t matter what kind of camera you’re using – a toy camera, a polaroid camera, or whatever – just as long as it does what a camera has to do. So what makes digital cameras any different?”

Yet Moriyama’s photos are made in the post-processing stage. He captures the snapshot on the street and then makes the photo in the darkroom (or in Silver Efex with digital). Post-processing usually involves pushing the blacks and whites, increasing contrast and adding grain. In his modern work it seems as though Moriyama photographs in colour, and converts to B&W in post-processing (see video below). It is no wonder that Moriyama is considered by some to be the godfather of street photography, saying himself that he is “addicted to cities“.

“[My] photos are often out of focus, rough, streaky, warped, etc. But if you think about it, a normal human being will in one day perceive an infinite number of images, and some of them are focused upon, others are barely seen out of the corner of one’s eye.”

For those interested, there are a number of short videos. The one below shows Moriyama in his studio and takes a walk around the atmospheric Shinjuku neighbourhood, his home from home in Tokyo. There is also a longer documentary called Daidō Moriyama: Near Equal, and one which showcases some of his photographs, Daido Moriyama – Godfather of Japanese Street Photography.

Artist Daido Moriyama – In Pictures | Tate (2012)

Further Reading:

Lightning strikes!

/ spqr / Leave a comment

Sometimes we tend to forget how exciting first achievements are. You get a good sense of these if you peruse vintage science journals from the late 1800s, many of which are available online as PDFs. When I was looking for an article from La Nature Revue Des Sciences recently from 1884, I came across another interesting article on the photography of lightning strikes by Gaston Tissandier (Vol.12, No.548., pp.118-119), entitled “Les Éclairs, Reproduits par la Photographie Instantanée“, or “The Flashes reproduced by instant photography”. The images show photographic prints of lightning taken by Mr. Robert Haensel of Reichenberg, Bohemia.

Photographs of lightning, taken on July 6th, 1883 at 10pm, when the sky was very dark

These photographs seem very simple, but are like pieces of artwork. They were acquired using silver-bromide gelatin plates, and activated by the lightning flashes themselves. Now the average duration of a flash of lightning is 0.1-0.2 seconds, so it says a lot about the sensitivity of film at the time. Haensel exposed 10 plates, of which four good negatives were produced. The photographs were reproduced for publication using the photogravure process.

This article was also published in The Popular Science Monthly, as, “Photographing a Streak of Lightning”, Vol. 24 pp.752-754 (April 1884). An earlier article appeared in The Photographic News, on January 4th, 1884 (London).