How does PIX18 work?
|
PIX18 has two components - think of them as a "mind" and a "body". The mind is an Artificial Intelligence software module capable of conceiving of a virtual painting. A virtual painting is a collection of paint strokes in the AI's memory. There can be hundreds, thousands, and sometimes tens of thousands of strokes in any single virtual painting, each stroke comprising a unique color, strength, speed, and brush motion. The physical body of the robot, typically a conventional gantry or articulated arm, then executes the virtual painting onto a canvas or panel. The physical robot typically requires a human to initially load a canvas, a palette of paints (as specified by the AI), and a brush. From then on, the robot runs automatically, picking up paint with the brush, painting on the canvas, mixing paints, and cleaning the brush as needed. A single painting can take up to 48 hours to paint.
|
Time-lapse view of a painting
|
How does the AI conceive of painting?
Like a human artist, PIX18 can draw inspiration from various sources. In the most direct case, PIX18 will interpret an existing photo or collection of photos, or a live camera feed. This is similar to an artist painting directly from a photo or from a live view, such as still life, a sitting model, or a scenic vista. In other cases, the robot can create an image entirely from scratch, from its imagination and past experiences (see below).
|
Once a inspiring image is in hand, the AI attempts to find a collection of strokes that would best represent that image. There are two internal processes that work side by side to accomplish this -- think of them as a creator AI and a critic AI. The creator proposes a painting, and the critic gives positive or negative feedback. Based on the feedback, the creator makes modifications - amplifying certain aspects and attenuating others - and asks for critique again. The creator and the critic can go back and forth hundreds of times per second. Starting from what appears to be almost a random painting and nonsensical feedback, the painting will gradually evolve. Like two amateurs playing chess against each other and improving in the process, the creator and the critic both improve over time, in a process known as "Adversarial Learning". All these iterations happens inside the computer. It can take anywhere from a few minutes to a few days on a cloud computer, depending on the complexity and size of the painting, Once the virtual painting is complete, it is handed of to the physical robot to execute.
|
Internal evolution of a virtual painting representing two overlapping circles
|
How does the critic AI form an opinion? In the simplest case, the critic will determine the merit of a candidate painting offered by the creator AI based on how well the candidate painting matches a given photo or live view. In a more abstract case, the critic AI will determine the value of a candidate painting based on how recognizable it is: Does it look like a human, for example, or is it recognizable as a cat. This depends on the experience the critic AI has obtained by seeing other humans, cats, or objects of interest. In even more abstract cases, the critic AI has no specific target at all. The question is simply whether the critic's deep neural network gets "excited" in some way by the offered painting - for example, do its neurons spike in an unusual pattern. Finally, some of the most abstract creator AI can stroll around the world using Google StreetView or learn what is happening around the world on Google News, acquire its own experiences and impressions, and draw inspiration from there. At that point, there is no telling what the robot will get excited about and what it will choose to paint.
More reading
Carlos Aguilar, Hod Lipson (2008) "A robotic system for interpreting images into painted artwork", GA2008, 11th Generative Art Conference