I have never stopped asking myself the same question: what is a mosaic system? A grid, a rule, one colour per cell. If that is all it is, it can borrow ideas from any other craft: Islamic tilework, weaving, tartan, lace, the stone of the Roman tessellarius. Every craft has its own way of filling a surface. I turn it into a setting.
This time I borrowed camouflage.
In two paragraphs
The word is French (camoufler, “to disguise”) and it became common in the First World War, when the aeroplane and the aerial photograph changed what it means to be seen. Painters were drafted into army units. The French set up a camouflage service; the British painted ships in the zigzags of dazzle, which do not hide a ship but confuse its heading. The woodland and tigerstripe patterns we know come from there.
Before all of that, though, there was a painter who had already thought it through from the side of nature.
Why Abbott
Abbott Handerson Thayer (1849–1921) was an American painter and bird watcher, and is often called the father of camouflage. His central idea, which he developed with his son Gerald in their 1909 book Concealing-Coloration in the Animal Kingdom, is countershading: animals are dark above and light below, so that light from the sky cancels the shadow and the body flattens out and disappears. Thayer did not see the animal. He saw the distribution of tone on it.
That is exactly the eye of a mosaic. A mosaic system does not “see” a face; it reads tones. So Thayer was the right figure for this experiment: the man who argued that concealment is the dissolving of tone should vanish in the same way. (Thayer also overreached: he claimed even the pink of flamingos was camouflage. But the core observation stayed.)
The video
Fourteen seconds. Every frame is a real render from Mozaix: no cross-fade, no mask, no opacity change. From one frame to the next, only a setting changes.
The setting doing the work is called Follow, which controls how closely the strokes obey the portrait. At 1.0 the face is built out of strokes. At 0 nothing of it is left, only pure camouflage. In between, stroke length and width, bend, and the palette all change together, the palette moving from the tones of the photograph to the colours of the earth.
The computer vision behind it
Three things, all classical and none of them magic:
- Palette. A k-means clustering of the photograph’s colours gives four representative tones, and each of the image’s four classes gets one.
- Tone. The photograph becomes a lightness map, ranked from light to dark. Dark classes ask for dark regions, light classes for light ones, and Follow sets how strongly that is enforced.
- Direction. We blur the tone and compute its gradient. The strokes turn along the contours of the tone, which means they follow the outlines of the face. As Follow falls, they let go of the contour and return to a common direction.
The hard part was an unglamorous problem: strokes are placed at random, so a small step in Follow re-rolled the whole pattern and the video flickered. I solved it by giving each stroke its own stream of random numbers, derived only from the seed. Follow merely decides whether, and how, a stroke is used. A small step now moves a few strokes, and the frames read as one object changing.
The same face in other camouflage
The film uses one of five camouflage styles in Mozaix, Directional Strokes. The others fill a surface in quite different ways, and each has presets named after real prints. Here is the same photograph of Thayer in three of them, with six of their presets. Nothing was retouched: every image is one render, in the film’s palette, which moves from the tones of the photograph to the colours of the earth as Follow falls.
Three of these presets keep the face on purpose. Fleck portrait gathers small clusters of dots on the face’s own darks and lights, Pixel portrait does the same with blocks snapped to a square lattice, like digital camouflage, and Engraving portrait swells thin bands where the face is dark and thins them where it is light. The other three are patterns first. Leibermuster runs large dots together into soft blotches, and the face is gone. Cubist piles squarish blocks at every angle, and the head can only be guessed. Razzle dazzle cuts the canvas into clashing panels of hard parallel bands, after the ships of the First World War: it does not hide a shape, it breaks it up, and here nothing of the portrait is left.
The same rule holds in all of them. The picture is still there, and a setting decides how much of it the pattern is allowed to show.
Mosaics made of photographs
Everything so far fills each piece of the pattern with a flat colour. Mozaix can also fill every piece with a photograph. For these I gathered about two thousand pictures of soldiers, American, British and Western European, all in the public domain or under Creative Commons licences, and let Mozaix choose which one goes where. The rule is the one every photo mosaic follows: a piece of the portrait that is dark gets a dark photograph, a pale piece gets a pale one. From a distance you read Thayer’s face; close up, every piece is a different soldier in a different place. The camouflage grid decides the shape of the pieces, and the photographs decide what each piece is made of.
A hybrid keeps both ideas. The pattern also decides which pieces become photographs and which stay paint: the darkest classes of the camouflage are filled with flat colour, and the rest with pictures. The paint holds the shape together, as in an ordinary camouflage print, and the photographs show through the lighter patches, so the face reads like a two-tone poster lit from inside by pictures.
Camouflage and computer vision share one question from opposite sides: what does an observer need to recognise a shape, and what is enough to lose it? A mosaic that can do both, with a single slider, seems to me a good way to ask it.
More on the underlying ideas: algorithmic mosaic art, and the history of mosaic algorithms.
