Protocol

Data Collection Procedure

Movie Selection

We select 10 popular romantic and family movies which have been in theater and received high IMDb ratings (not lower than 7.1), including About Time, Me Before You, The Theory of Everything, The Fault in Our Stars, The Spectacular Now, Lady Bird, Wonder, The Blind Side, Gifted, and The Judge.

Interaction-Oriented Excerpts

Each movie clip was excerpted based on a complete interpersonal interaction, which is the basic semantic unit for comprehending and experiencing a story plot. We define an interaction-oriented movie
clip by explicit criteria.

Data Annotation

Character Relationship

Each annotation of character relationship was annotated by explicit definition.

  1. Character Relationship Type
    • Romance, Family, and Friendship.
  2. Character Relationship Status
    • The label indicates whether characters get along well or not, ranging from 1 to 5. Higher scores represents characters share better interpersonal relationship.

Audience Emotion

Each annotation of audience emotion was annotated by 8 annotators recruited from a university.

  1. Perceived Emotions
    • Perceived valence was annotated in 5 levels; perceived arousal was annotated in 3 levels.
  2. Induced Emotions
    • Both induced valence and induced arousal were annotated in 5 levels.
  3. Degree of Being Moved
    • The label, which was annotated in 5 levels, indicates the audience’s degree of being moved after watching a movie clip.

Multimodal Features

Audio

  1. eGeMAPS
  2. VGGish

Visual

  1. Facial Action Units
  2. Pre-trained VGG19 with batch normalization

Text

  1. Pre-trained BERT