Skip to content

Latest commit

 

History

33 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MigrationNarrate

MigrationNarrate: A Dataset for Detection of Migration Narratives in YouTube Videos

MigrationNarrate is a dataset for studying migration narratives in UK-related YouTube videos. It contains 1,115 human-annotated videos labelled using a hierarchical taxonomy of migration narratives, together with a larger set of 4,425 automatically filtered unlabelled videos.

The taxonomy consists of 12 super-narratives and 53 fine-grained narrative labels. Of these, 51 fine-grained narratives are represented in the human-annotated dataset.

The dataset was introduced in:

Fatima Haouari, Carolina Scarton, and Kalina Bontcheva. MigrationNarrate: A Dataset for Detection of Migration Narratives in YouTube Videos. Accepted to the Main Conference of EMNLP 2026. arXiv:2608.20984


Dataset Overview

MigrationNarrate was created to support research on the detection and analysis of migration narratives in video-based online discourse.

The original collection contains short YouTube videos related to migration in the UK, published between 1 January 2024 and 30 September 2025. Videos longer than three minutes were excluded.

The collection and filtering pipeline resulted in:

Dataset Videos Unique channels Avg. duration
Filtered corpus 5,540 2,081 90.46 sec
Human-annotated dataset 1,115 684 79.73 sec
Unlabelled subset 4,425

The 1,115 annotated videos are divided into:

Split Videos
Train 774
Dev 117
Test 224
Total 1,115

MigrationNarrate/
│
├── data/
│   ├── labelled_data.tsv
│   └── unlabelled_data.tsv
│
├── data_splits/
│   ├── train.tsv
│   ├── dev.tsv
│   └── test.tsv
│
├── taxonomy/
│   └── migration_narratives_taxonomy.tsv
│
├── scripts/
│   └── extract_transcripts.py
│
├── requirements.txt
└── README.md

data/labelled_data.tsv

Contains the identifiers and human annotations for the 1,115 annotated YouTube videos.

Field Description
videoId YouTube video identifier
super_narrative Human-annotated super-narrative
narrative Human-annotated fine-grained narrative

The annotation None indicates that, after human annotation/adjudication, no migration narrative was explicitly stated or strongly implied in the video.

data/unlabelled_data.tsv

Contains the identifiers of the 4,425 migration-related videos retained by the automatic filtering pipeline but not selected for human annotation.

These data can be useful for, for example, weakly supervised, semi-supervised, or further manual annotation experiments.

taxonomy/migration_narratives_taxonomy.tsv

Contains the hierarchical migration narrative taxonomy used for annotation, including the 12 super-narratives and 53 narrative labels and their definitions.

The taxonomy is based on the migration narrative taxonomy introduced in the European Commission Joint Research Centre (JRC) report Navigating migration narratives: Research insights and strategies for effective communication.


Data Collection

Videos were collected using two complementary strategies:

Channel- and playlist-based collection. Videos were collected from migration-related and broader political/news channels and playlists using the YouTube Data API v3.

Search-based collection. Additional videos were retrieved using migration-related search phrases in order to increase the diversity of sources represented in the corpus.

The initial collection was subsequently filtered to retain videos:

  • published between 1 January 2024 and 30 September 2025;
  • with a duration of three minutes or less;
  • identified as migration-related using semantic similarity filtering.
  • Pre-annotated with GPT5.1 to select a diverse sample.

After this process, 5,540 videos remained in the filtered corpus.

A diverse subset of 1,115 videos was then selected for human annotation.

For full details of the collection, filtering, sampling, and annotation procedures, please refer to the paper.


Annotations

MigrationNarrate uses a hierarchical annotation scheme.

Each video is first assigned a super-narrative and then, where applicable, a corresponding fine-grained narrative.

The annotation taxonomy contains:

  • 12 super-narratives
  • 53 fine-grained narratives

The human-annotated dataset contains instances belonging to 51 of the 53 fine-grained narrative categories, in addition to examples labelled None.

Each annotation batch was independently annotated by two annotators, with disagreements resolved through adjudication.


Obtaining the Videos

We do not redistribute the original YouTube videos or audio files.

Instead, the dataset provides YouTube video identifiers that can be used to locate videos that remain publicly available on YouTube.

Please note that YouTube videos may be removed, or become unavailable over time. Consequently, it may not always be possible to access all videos included in the original dataset.


Obtaining the Transcripts

We do not redistribute the generated video transcripts.

The transcripts used in the paper were generated locally from the video files using faster-whisper (large-v3).

Requirements

Install the required Python packages with:

pip install -r requirements.txt

The requirements.txt file contains the Python dependencies required by the transcription script, including faster-whisper and tqdm.

The script also requires FFmpeg to be installed and available on the system path.

Generating Transcripts

We provide a script for generating transcripts from locally available video files:

python scripts/extract_transcripts.py videos/

The videos/ directory should contain the videos as MP4 files, with each file named using its corresponding YouTube video ID:

videos/
├── VIDEO_ID_1.mp4
├── VIDEO_ID_2.mp4
└── VIDEO_ID_3.mp4

For example:

videos/abc123.mp4

For each video, the script generates a .txt transcript with the same video ID:

videos/abc123.mp4
videos/abc123.txt

The script automatically detects the language of each video and generates the transcript using faster-whisper large-v3.

Please note that reconstructed transcripts may differ slightly from those used in the original experiments depending on software/model versions and the availability or version of the underlying video.


Citation

If you use MigrationNarrate in your research, please cite:

@article{haouari2026migrationnarrate,
  title     = {MigrationNarrate: A Dataset for Detection of Migration Narratives in YouTube Videos},
  author    = {Haouari, Fatima and Scarton, Carolina and Bontcheva, Kalina},
  journal   = {arXiv preprint arXiv:2608.20984},
  year      = {2026}
}

The paper was accepted to the Main Conference of EMNLP 2026.


License

The MigrationNarrate dataset and annotations are released under the CC BY-NC-SA 4.0 license for non-commercial research use.


Contact

For questions about the dataset, please open an issue in this repository or contact the authors of the paper.


Acknowledgements

About

A Dataset for Detection of Migration Narratives in YouTube Videos

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages