Where Art and Tech Click: Algorithmic Photography

Use the term “algorithmic photography” and you might draw a blank reaction from many people, but not from British digital artist Alex May. He, it appears, invented both the term and the technique.

Not to be confused with AI-generated images (although there is a connection) or with what at first glance might appear to be time-lapse photography, algorithmic photography, as May practices it, creatively extracts a still image from a digital video that he shoots.

That might sound like any old YouTube style screen grab. Of course, it’s not.

In an algorithmic still image, certain motion is presented as a single point in time, artistically rendered by colorful and impressionistic swirls against a moment’s static setting, which could be buildings and streets in a cityscape.

The artistry comes from the application of algorithms that May writes to capture colors as they change during what is typically a five-minute shot, and that also filter time and motion. In many cases, May uses algorithms that might well be useful to computer vision systems, but rather than creating a seeing machine, he is giving a new look to a comparatively old art form: the photograph.

“I’ve been using the term ‘algorithmic photography’ for a number of years now, to describe this process I came up with, which is a digital process but has lots of links into photography and the history of photography,” says May, who is based in Brighton on the U.K.’s south coast. “It’s a sort of balance between how we see the world through our own perspective and through technology.”

May has long lived at the intersection of art and technology, having started writing computer code in the early 1980s using assembly language, while always maintaining a keen interest in photography. He has an accomplished resume as a digital artist, having worked as a video jockey on the international club scene in the early 2000s, and later sculpting video works that have appeared in well-known art and science venues such as London’s Tate Modern museum and The Francis Crick Institute.

One of his inspirations is Eadweard Muybridge, the British photographer who, in the 1870s, invented techniques for higher shutter speeds that disproved the common notion (as depicted in illustrations of the time) that a galloping horse’s legs pointed outward; rather, they curled in, Muybridge showed via a series of still images that translated into motion.

“It struck me at a young age that this technology allows us to see beyond what our amazing physical attributes allow us to see in our search for knowledge and how things work,” he noted.

Through the use of algorithms, May is now pushing still photography to capture motion, taking his hero Muybridge to a new level. “Photographs to me have always been really alien, because they capture this instant in the world, but we don’t experience the world like that, because we’re in motion,” May said. “Something about capturing motion in a photo captures something closer to the way we experience the world.”

Appropriate for someone whose work straddles art and technology, May’s subject matter sometimes focuses on the science world, where his work depicts not the inner workings of science itself, but rather what he sees as the artistry of practicing scientists.

For example, May has an ongoing residency at Vienna’s University of Natural Resources and Life Sciences, popularly known as BOKU, where he algorithmically photographs scientists working with yeast on an immunology project. In one photograph, which follows here, he applies colors and elongation to Dr. Özge Ata of BOKU University in order to capture her movement against the static everyday setting of lab furniture.

“It becomes this impressionistic representation of the science taking place,” May said. “These are real scientists, really doing science, and I’m using the algorithmic photography technique to capture that in an artistic way, to show the fluidity of their movements. It suggests a dance performance.”

May can just as readily find himself photographing a street scene or an urbanscape. In a fanciful rendition of the Philadelphia Museum of Modern Art, seen below, he uses braids of colors to show off the many runners who, within five minutes, have ascended the steps in the style of Rocky from the eponymous movie.

One of his favorites shows the well-known Brighton Pier in his U.K. home town (below), which juxtaposes the happiness of tourists with the menace of birds infamous for swooping down to snatch sandwiches and snacks from the hands of unsuspecting visitors.

“That picture for me captures the tension between the birds in the domain of the sky and the people in the domain of the ground,” said May. “There is literally tension in the air, with people having a nice time and birds circling around looking for every opportunity to grab a chip. It’s something quintessentially Brighton and touristy.” May shot the video with a small GoPro Hero 6 — typical of the type of portable video camera he uses — before applying algorithms for the

Keeping in mind the “photography” part of algorithmic photography, some time-honored challenges apply, such as getting good camera angles. As an avid urban photographer, May notes that gaining a view from above (in other words, climbing to a certain height) can be key. “But in some cities, it’s very difficult to get off of street level, unless you’re actually going into a building, in which case you need to have some reason to be there.”

New York City’s elevated High Line park in lower Manhattan was a valuable assistant as May photographed the movement of iconic yellow taxis from that vantage point, shown below.

“It was nice to get some verticality,” he said.

With so many algorithms involved, one might assume artificial intelligence is part of May’s toolbox. But so far, it has played only an experimental role. May is working with AI to help separate out people and objects that he then renders creatively. One problem has been that AI, as studies have shown, does not always reliable identify people, and can miss individuals due to racial and other biases trained into datasets.

“There’s a lot of work in this area, looking at what’s called fairness, accountability, and transparency,” noted Sanmay Das, professor of computer science at George Mason University in Fairfax, VA. Das points to the Gender Shades study connected to the Massachusetts Institute of Technology’s Media Lab showing that, in the words of the study, “commercial AI systems significantly mis-gender women and darker skinned individuals.”

Das noted, “A lot of that comes down to bias in the underlying data sets that they’re trained on.”

May is well aware of these issues as he starts working with AI, a technology he refers to as a “collaborator” because it includes the use of datasets trained by humans.

Up until now, pre-AI, “I’ve had control, because I’ve written the code myself,” May said. “I’m now moving into AI where I’m introducing trained data sets that I didn’t train. So I’m having to invite a collaborator, which is a sort of first in this process. If there are biases in there, they will come through. I have no control over that.”

AI is something that any practitioner in the incipient field of algorithmic photography will have to contemplate. So far, relatively few artists seem to be pursuing algorithmic photography—at least using that name. British technology blogger Peter Krantz has written about his project in the field. Paris-based multimedia artist Sabina Covarrubias also is doing work that she identifies as algorithmic photography.

May himself will get a chance to examine bias and other aspects of AI in his next endeavor: the AI Hokusai Arttech Research Project, an online initiative in which invited artists such as May will use AI to generate artwork. The project takes its name from Japanese artist Katsushika Hokusai, known for his 19th century innovations in printmaking.

Whatever emerges—or doesn’t—from AI, May will continue practicing the truism of his art.

“The image can only be created algorithmically,” he says. “Photography as we know it can only collect light. It’s an accumulation of light picked up by the sensor. Whereas the algorithmic photo is picking out specific data coming from the image.”

Could it be that the person with the most data in the end not only wins, but is also the most creative?

Mark Halper is a freelance journalist based near Bristol, England. He covers everything from media moguls to subatomic particles.