Do computers understand art?

A team of scientists from the University of Girona and the Max Planck Institute in Gera number of has shown that some mathematical algorithms provide clues about the artistic style of a painting. The composition of colours or certain aesthetic measurements can already be quantified by a computer, but machines are still far from being able to interpret art in the way that people do.

How does one place an artwork in a particular artistic period? This is the question raised by researchers from the Laboratory of Graphics and Image in the University of Girona and the Max Planck Institute for Biological Cybernetics, in Gera number of. The scientists have shown that certain artificial vision algorithms mean a computer can be programmed to "understand" an image and differentiate between artistic styles based on low-level pictorial information. Human classification strategies, however, include medium and high-level concepts.

Low-level pictorial information encompasses aspects such as brush thickness, the type of material and the composition of the palette of colours. Medium-level information differentiates between certain objects and scenes appearing in a picture, as well as the type of painting (landscape, portrait, still life, etc.). High-level information takes into account the historical context and knowledge of the artists and artistic trends.

"It will never be possible to precisely determine mathematically an artistic period nor to measure the human response to a work of art, but we can look for trends", Miquel Feixas, one of the authors of the study, reported in the journal Computers and Graphics, tells SINC.

The scientists analysed various artificial vision algorithms used to classify art, and observed that certain aesthetic measurements (calculating "the order" of the image based on analysing pixels and colour distribution), as well as the composition and diversity of the palette of colours, can be useful.

The team also worked with people with little knowledge of art, showing them more than 500 paintings done by artists from 11 artistic periods. The participants were "surprisingly good" at linking the artworks with their corresponding artistic period, showing the high capacity of human perception.

Beyond the implications for philosophy and art, the researchers want to apply their research in developing image viewing and analysis tools, classifying and searching for collections in museums, creating public informative and entertainment equipment, and in order to better understand the interactions between people, computers and works of art.


Beauty, order and complexity
The earliest work of this kind was done in 1933, when the mathematician George D. Birkhoff tried to formalise the notion of beauty with an aesthetic measurement defined as the relationship between order and complexity. After this, the philosopher Max Bense converted this into a measurement of information based on entropy (disorder or diversity).

As per Bense, the creative process is a selective process ("to create is to select"), within a range of elements (a palette of colours, sounds, phonemes, etc.). The creative process can be seen as channel for transmitting information between the palette and the artist and the objects or features of an image. This concept provides a powerful tool for analysing composition and the visual attention ("saliency") of a painting.