Home >

CLASS - Cognitive-Level Annotation using Latent Statistical Structure

CLASS was a 3.5 year targeted research project that ran from January 2006 to June 2009. It was funded by the European Union Information Society Technologies unit E5 – Cognition.

Scientific Goals

The main goal of CLASS was to develop a basic cognitive ability for use in intelligent content analysis: the automatic discovery of content categories and attributes from unstructured content streams. The demonstrators focused on object recognition and scene analysis in images and video with accompanying text streams. Autonomous learning made recognition more adaptive and allowed more general classes and much larger and more varied data sets to be handled.

Technically, the work combined latent structure models and semi-supervised learning methods from machine learning with advanced visual descriptors from computer vision and state-of-the-art text analysis techniques. Three levels of abstraction were studied: new individuals (specific people, objects, scenes, actions); new object classes and attributes; and hierarchical categories and relations between entities.

The basic research results were illustrated by three proof-of-concept demonstrators: an Image Interrogator that interactively answers simple user-defined queries about image content; a Video Commentator that automatically creates textual descriptions of the action and content of situation comedy videos for visually impaired users; and a News Digester that combines television news stories with captions from several sources to create a textual and visual digest of them. New functionality was added to each demonstrator every year, so there were three versions of each demonstrator.


CLASS was an interdisciplinary project, combining eight leading European research teams in visual recognition, text understanding & summarization, and machine learning:


The publicly available versions of the CLASS demonstrators can be found on the demonstrator page.


CLASS produced a large number of scientific publications. Most of these are available for download from the publications page.

A feature on CLASS research appeared in the June 2009 the ICT Results service, and Vittorio Ferrari gave a talk covering some of our work on visual attributes at the ICT CogSys 2010 workshop.

The proceedings of a workshop associated with CLASS - the IJCAI-09 Workshop on Cross-Media Information Access and Mining (CIAM 2009) are also available.

Tutorials and Talks

A few of the tutorials and talks produced during CLASS are available on the tutorials page.

Datasets and Software

Some of the datasets and software associated with CLASS are available on the datasets page.


Home page: http://class.inrialpes.fr. CLASS flyer in PDF and MSWord formats.

Contact: Bill Triggs (coordinator), Bill.Triggs@imag.fr, phone +33 4 7651 4553
Laboratoire Jean Kuntzmann, 51 rue des Mathematiques, 38402 Saint Martin d'Heres, Grenoble, France