The day an algorithm started painting…

I’ve been reading and writing a lot lately on the future of work, specifically about how many jobs may no longer exist in the future because of automation. This means that algorithms and robots might be able to do the work more efficiently and cheaply than any human alternative. I’ve started to see some examples of this in the news. One was a robot that was able to inspect bridge infrastructure and alert someone if it found the concrete pillar of a bridge was crumbling. The robot boat could travel up and down a river, inspecting the underwater structures in ways that had been previously handled by human divers. There are other examples of robots that use magnets to cling to a bridge, allowing them to crawl over the structure and inspect it for defects. Both of these are great examples of robots replacing tedious and potentially dangerous jobs that are currently being done by people.

For many people researching the future of work this isn’t such an issue. It makes sense that these kinds of jobs will become a hybrid of people and machines working together to solve a problem. In some cases it might mean a machine can completely take over the job.

There has always been a sense of security that the more “human” jobs would be free from any encroachment from machines. Those were the more complex jobs that required creativity and complex problem solving. It seemed like a good partnership, machines could handle the tedious tasks and we could tackle the real problems of the world.

Then I heard about “The Next Rembrandt” project. The project analyzed all of Rebrandt’s paintings using 3D scanning that could captured the brushstrokes and the painting itself,  pixel by pixel. When the team was done it asked the algorithm to take what it knew about a real Rembrandt:


Credit: H. O. Havemeyer Collection, Bequest of Mrs. H. O. Havemeyer, 1929

and used it to create a completely new Rembrandt using 3D printing to replicate the colour and brushstroke texture of an original painting. It then told the algorithm to take what it had learned and paint something. They gave it some parameters, the program should painting a portrait of a 30 to 40 year old Caucasian male with facial hair, he should be wearing dark clothing and have a collar around his neck and a hat on his head. The program then took all this information and returned this:


Credit: The Next Rembrandt

It seems I was a bit smug about what machines wouldn’t be able to tackle in the future. Art has traditionally been considered an exclusively human domain. On the surface it would seem that that domain has been breached. In truth, it hasn’t happened yet. The machines that created this painting were carefully directed by human programmers and designers. It didn’t decide that it needed to become a painter because of some intrinsic need, it was following orders. It managed to do this in an original and unpredictable way which is most important take away from this project. If humans are able to design these kinds of machines we are going to be able to experiment and innovate in much more interesting and accelerated ways. The real thrill from this kind of experiment is projected it forward to the next challenge. Imagine an algorithm being feed decades of diagnostic information on cancer, all of the available pharmaceutical and medical techniques available and then being told “cure cancer”. Initially it won’t be that expansive, more likely the command will be “cure my cancer” but you can start to see how these tools can begin to actually change the way we tackle problems.