MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmark

We present the first large-scale open-set benchmark for multilingual audio-video deepfake detection. It comes from a paper posted to arXiv on 23 October 2026. Our dataset comprises nearly 350 hours of real and fake videos across eight languages, with 59% of data being generated. For each language, the fake videos are generated with several distinct audio and video deepfake generation models, selected based on their quality, via visual inspection. We organize the training, validation and test splits such that only a subset of the chosen generative models and languages are available during training, thus creating several challenging open-set evaluation setups. We perform experiments with various pre-trained and fine-tuned deepfake detectors proposed in recent literature. Our results show that state-of-the-art detectors are not currently able to maintain their performance levels when tested in our open-set scenarios. We publicly release our data and code for non-commercial use at https://github.com/CroitoruAlin/MAVOS-DD.

Why it counts

We present the first large-scale open-set benchmark for multilingual audio-video deepfake detection. Our results show that state-of-the-art detectors are not currently able to maintain their performance levels when tested in our open-set scenarios.

Sources

Primary source: primary source

What is not known

This brief does not claim independent replication. Claims that appear only on X and not in the primary source stay unknown.