PhD thesis - Visual attention modeling

University of La Rochelle
(2007-2010)

During my PhD thesis (Perreira Da Silva, 2010), I focused on building links between the fields of computational modeling of visual attention, computer vision, and the more generic concept of adaptation. To this end, I developed a computational model of attention dedicated to adaptive vision. This model allows interaction with a vision system so that it adapts its processing according to the “interest” of each element of the scene, i.e. what we call saliency.

The model responds to constraints of speed, adaptation, extensibility and dynamics, resulting from an analysis of existing computational visual attention models and their applications. We thus proposed a hybrid approach, taking advantage of the benefits of hierarchical attention models (speed, extensibility) and distributed models (plausibility, management of dynamics). This original approach makes it possible to generate an attentional focal point at each time step without using a saliency map or an explicit feedback inhibition mechanism. This new computational model of real-time visual attention is based on a dynamic system of prey / predator equations, which is well suited for the trade-off between non-deterministic attentional behavior and properties of stability, reproducibility, and responsiveness (Perreira Da Silva et al., 2009; Perreira Da Silva & Courboulay, 2012; Courboulay & Perreira Da Silva, 2012; Perreira Da Silva et al., 2011; Perreira Da Silva et al., 2010).

Presentation of the visual attention model.

The analyses of the experiments carried out during the thesis are very positive: despite the chaotic behavior of the prey / predator equations, this system has interesting properties of stability, reproducibility, and reactivity, while allowing rapid, efficient and plausible exploration of the scene. These properties have allowed us to tackle different types of applications such as assessing the complexity of images, dynamic cropping or attentional segmentation of images and videos. The potential of this model remains to be exploited in many other areas, such as human-machine interaction, scene analysis or multimedia indexing.

My PhD thesis is written in French, but a full description of the model was published in a book chapter (Perreira Da Silva & Courboulay, 2012). The source code of the model is not yet available, but as a binary executable can be downloaded.

You will find below a few videos of some applications (Perreira Da Silva et al., 2010; Perreira Da Silva et al., 2012) of this model (video segmentation, dynamic image discovery, dynamic re-purposing). One of the applications not illustrated in the following videos is the evaluation of images or videos complexity using visual attention (Perreira Da Silva & Courboulay, 2011; Perreira Da Silva et al., 2011). This was one of the first publications on this topic.

Detection of areas of interest in a road traffic video.
Visual attention based dynamics images discovery.
Visual attention based video re-purposing.

A few more videos are available here.

  1. Perreira Da Silva et al., 2012
    Implementation and evaluation of a computational model of attention for computer vision , Matthieu Perreira Da Silva and Vincent Courboulay , 2012
  2. Perreira Da Silva et al., 2012
    Real time dynamic image re-targeting based on a dynamic visual attention model , Matthieu Perreira Da Silva, Vincent Courboulay and Patrick Le Callet , In 2012 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2012 PDF
  3. Courboulay et al., 2012
    Real-time computational attention model for dynamic scenes analysis: from implementation to evaluation. , Vincent Courboulay and Matthieu Perreira Da Silva , In SPIE Optics, Photonics and Digital Technologies for Multimedia Applications - Visual attention, 2012 PDF
  4. Perreira Da Silva et al., 2012
    Implémentation et évaluation d’un modèle d’attention pour la vision adaptative , Matthieu Perreira Da Silva and Vincent Courboulay , Traitement du Signal, 2012 PDF
  5. Perreira Da Silva et al., 2011
    Une nouvelle mesure de complexité pour les images basée sur l’attention visuelle , Matthieu Perreira Da Silva and Vincent Courboulay , In XXIIIème coloque GRETSI - Bordeaux, 2011 PDF
  6. Perreira Da Silva et al., 2011
    Image complexity measure based on visual attention , Matthieu Perreira Da Silva, Vincent Courboulay and Pascal Estraillier , In IEEE International Conference on Image Processing - ICIP, 2011 PDF
  7. Perreira Da Silva et al., 2011
    Objective validation of a dynamical and plausible computational model of visual attention , Matthieu Perreira Da Silva, Vincent Courboulay and Pascal Estraillier , In IEEE European workshop on visual information processing, 2011 PDF
  8. Perreira Da Silva, 2010
    Modèle computationnel d’attention pour la vision adaptative , Matthieu Perreira Da Silva , 2010 PDF
  9. Perreira Da Silva et al., 2010
    Evaluation of preys / predators systems for visual attention simulation , Matthieu Perreira Da Silva, Vincent Courboulay, Armelle Prigent and Pascal Estraillier , In VISAPP 2010, 2010 PDF
  10. Perreira Da Silva et al., 2010
    Système temps réel de simulation d’attention visuelle : application aux images et séquences d’images , Matthieu Perreira Da Silva, Vincent Courboulay, Armelle Prigent and Pascal Estraillier , In RFIA 2010, 2010 PDF
  11. Perreira Da Silva et al., 2009
    Attention visuelle et systèmes proies / prédateurs , Matthieu Perreira Da Silva, Vincent Courboulay, Armelle Prigent and Pascal Estraillier , In XXIIe Colloque GRETSI - Traitement du Signal et des Image, 2009 PDF