In a context where the Covid crisis severely impacts culture, it is more important than ever to build visitor loyalty. I had proposed different ways to improve the customer experience in museums, and today I suggest discovering an initiative of the Royal Museums of Fine Arts in Brussels.
An exhibition dedicated to the Belgian artist Pierre Alechinsky has just opened and gives a very nice overview of his plethoric production. On this occasion, an interactive device using artificial intelligence allows visitors to draw on a touch screen and see, live, which works of Alechinsky come closest.
The prototype was developed by the Numediart department of the University of Mons in Belgium. Its director, Thierry Dutoit, explains the genesis of the project and its technical aspects in the following article.
How was the collaboration with the Royal Museums of Fine Arts born?
Thierry Dutoit: At UMONS, we have been working on artificial intelligence for museum exhibitions for many years. We have, among other things, interacted a lot with the museums of Mons in the framework of Mons2015 (European cultural capital in 2015).
In the framework of a European research project (i-MOTION) that associated us with the University of Basel, we investigated the possibilities of AI to search very quickly for a particular video (one minute) in an extensive video database (several hundred hours). This is a significant problem for documentary research.
We tried to solve this problem by proposing to the user to draw a scene from the video and use the AI to find the video clips that “look like” the sketch. We very quickly outperformed humans, and in 2017, we were the winners of an international competition with our system.
We quickly realized that the same technique could be used to search for images… We had already tested the idea at the Artothèque (Mons) during a museum night. When Karine Lasaracina (MRBAB) contacted us in December, looking for a system that would allow the visitor to be involved in the museum process, we proposed to adapt our solution to Alechinsky’s works.
How was the contribution of the device to the visitor experience envisaged?
Thierry Dutoit: The idea of the interaction with the visitor is to call upon his memory of the exhibition. We request him to draw a work that he remembers to see if the AI can understand his sketch and to put it in adequation with the original work. The visitor’s ability to make a sketch is therefore also put to the test. Finally, as the device sends back not only one result (the closest according to the AI) but about fifteen similar works, the visitor is invited to question himself on the pictorial link between these works.
How does the system work?
Thierry Dutoit: The device consists of a large touch screen connected to a computer that carries AI developed for the exhibition. The system begins to propose works from the first stroke of the user’s sketch, and the works’ proposals are refined as one completes the drawing. It was important for this type of use that the entire system be rapid to respond.
What were the technical difficulties you had to face?
Thierry Dutoit: It was necessary to make the AI “digest” all the scanned works of the painter.
Given the use of the system in an unattended museum setting, it was necessary to configure the system so that a hacker could not easily break the application or hijack the use of the computer.
Finally, it was necessary to make the system robust to a power failure: the computer automatically detects when the power is turned off and restarts on its own (without human intervention) if it is within the museum’s opening hours.
Tags: recommendation algorithms