OMG-Emotion Recognition Challenge – Final Results

Erandol The final results of the 2018 OMG-Emotion Recognition Challenge are out:??

The leaderboard will be permanently stored on our website, and it will provide a quick access to the results, the links to the formal descriptions and code repository for each solution.

This will help to disseminate knowledge generated by the challenge even further and will improve the reproducibility of your solutions.

The solutions used different modalities (ranging from unimodal audio
and vision to multimodal audio, vision, and text), and thus provide us
with a very complex evaluation scenario. We then decided to separate the
results into one for valence and one for arousal.

For arousal, the best results came from the GammaLab team. Their three
submissions are our top 3 CCC arousal, followed by the three submissions
from the audEERING team, and the two submissions from the HKUST-NISL2018

For valence,?? the GammaLab team stays still in first (with their three
submissions), followed by the two submissions of ADSC team and the three
submissions from the iBug team.

It is very interesting to note that the winning teams used a combination
of unimodal and multimodal solutions.

We will keep this leaderboard intact for our 2018 challenge and will
create a general leaderboard, later on, so the results of the challenge
will remain on our website as it is.

Congratulations to you all!

Like it? Share it!
  • 6

The OMG-Emotion Recognition Challenge



The One-Minute Gradual-Emotion Recognition (OMG-Emotion)
held in partnership with the WCCI/IJCNN 2018 in Rio de Janeiro, Brazil.

I. Aim and Scope

Our One-Minute-Gradual Emotion Dataset (OMG-Emotion Dataset) is composed of 420 relatively long emotion videos with an average length of 1
minute, collected from a variety of Youtube channels. The videos were
selected automatically based on specific search terms related to the
term “monologue”. Using monologue videos allowed for different
emotional behaviors to be presented in one context and that changes
gradually over time. Videos were separated into clips based on
utterances and each utterance was annotated by at least five
independent subjects using the Amazon Mechanical Turk tool. To maintain
the contextual information for each video, each annotator watched the
clips of a video in sequence and had to annotate each video using an
arousal/valence scale and a categorical emotion based on the universal
emotions from Ekman.

We release the dataset with the gold standard for arousal and valence as
well the individual annotations for each reviewer, which can help the
development of different models. We will calculate the final Congruence
Correlation Coefficient against the gold standard for each video. We
also distribute the transcripts of what was spoken in each of the
videos, as the contextual information is important to determine gradual
emotional change through the utterances. The participants are encouraged
to use crossmodal information in their models, as the videos were
labeled by humans without distinction of any modality. We also will let
available to the participating teams a set of scripts which will help them
to pre-process the dataset and evaluate their model during in the
training phase.

We encourage the use of neural-computation models based on deep
learning, self-organization, and recurrent neural networks, just to
mention some of them, as they present the state-of-the-art performance in
such tasks.

II. How to Participate

To participate, please send us an email to with the title “OMG-Emotion Recognition
Team Registration”. This e-mail must contain the following information:
Team Name
Team Members

Each team can have a maximum of 5 participants. You will receive from us
the access to the dataset and all the important information about how to
train and evaluate your models.
For the final submission, each team will have to send us a .csv file
containing the final arousal/valence values for each of the utterances
on the test dataset. We also request a link to a GitHub repository where
your solution must be stored, and a link to an ArXiv paper with 4-6
pages describing your model and results. The best papers will be invited
to submit their detailed research to a journal yet to be specified.
Also, the best participating teams will hold an oral presentation about
their solution during the WCCI/IJCNN 2018 conference.

III. Important Dates

Publishing of training and validation data with annotations: March 14,
Publishing of the test data, and an opening of the online submission:
April 11, 2018.
Closing of the submission portal: April 13, 2018.
Announcement of the winner through the submission portal: April 18, 2018.

IV. Organization

Pablo Barros, University of Hamburg, Germany
Egor Lakomkin, University of Hamburg, Germany
Henrique Siqueira, Hamburg University, Germany
Alexander Sutherland, Hamburg University, Germany
Stefan Wermter, Hamburg University, Germany

Like it? Share it!
  • 2