Do neural networks dream of visual illusions?
This is a question studied by researchers from the Department of Information and Communication Technologies, led by Marcelo Bertalmío, together with Jesús Malo, a researcher at the University of Valencia.
The convolutional neural network is a type of artificial neural network where neurons are organized in receptive areas similar to neurons in the visual cortex of a biological brain. Today, convolutional neural networks (CNNs) are found in many autonomous systems (e.g., face detection and recognition, autonomous vehicles, etc.). This type of network is very effective in many artificial vision tasks, such as image segmentation and classification, along with many other applications.
The convolutional networks were inspired by the behavior of the human visual system, mainly the basic structure consisting of the chaining of linear and then non-linear composite modules. Research published in the advanced online edition of the journal Approach Research it examines the phenomenon of visual illusions in network bends in comparison to their impact on human vision. A study carried out by Alexander Gómez Vila, Adrian Martín, Javier Vázquez-Corral and Marcelo Bertalmío, members of the Department of Information and Communication Technologies (DTIC), with the participation of researcher Jesús Malo from the University of Valencia.
“Because of CNN’s connection to our visual system, in this article we wanted to see how convoluted networks have similar problems to our visual system. Therefore, we focused on visual illusions. Visual illusions are images that perceive how our brains really are differently, ”explains Gómez Vila, the first author of the study.
In the study, the authors trained CNN to perform human perspectives for simple tasks such as denoising and unlocking. What they have seen is that these CNNs trained in these experimental conditions are “deceived” by visual illusions of brightness and color in the same way that visual illusions deceive human beings.
Moreover, Gómez Villa explains, “for our work, we also examine when such illusions generate unexpected responses on the network, but do not match human perception,” which is a different optical illusion than CNN would perceive in humans.
The results of this study are consistent with a long-held hypothesis that low-level visual illusions are considered a byproduct of optimization for natural environments (which humans see in their daily lives). Meanwhile, these results highlight the limitations and differences between the human visual system and CNN’s artificial neural networks.
Reference: “Color illusions also deceive CNN for low-level visual tasks: analysis and implications” A. Gomez-Villa, A. Martín, J. Vazquez-Corral, M. Bertalmío, and J. Malok, September 4, 2020. Approach Research.
DOI: 10.1016 / j.visres.2020.07.010