MCIF
MCIF (Multimodal Crosslingual Instruction Following) is a multilingual human-annotated benchmark based on scientific talks that is designed to evaluate instruction-following in crosslingual, multimodal settings over both short-...
Read Moreby Beatrice Savoldi | Jan 13, 2025 | Corpora | 0
mGeNTE (Multilingual Gender-Neutral Translation Evaluation) is a natural, multilingual corpus designed to benchmark gender-neutral language and automatic translation.mGente is built upon European Parliament speech data extracted...
Read Moreby Beomseok Lee | Aug 21, 2024 | Corpora | 0
Spoken Language Understanding (SLU) involves interpreting spoken input using Natural Language Processing (NLP). Voice assistants like Alexa and Siri are real-world examples of SLU applications. The core tasks in SLU include...
Read Moreby Mauro Cettolo | Apr 30, 2024 | Corpora | 0
Ready-to-use version for MT research purposes of the multilingual transcriptions of TED talks
Read Moreby Dennis Fucci | Oct 20, 2023 | Corpora | 0
Text corpora for Spanish, French, and Italian containing gendered words referring to the first-person speaker
Read Moreby Beatrice Savoldi | Oct 19, 2023 | Corpora | 0
The INclusive Evaluation Suite (INES) is a test set designed to assess MT systems ability to produce gender-inclusive translations for the German→English language pair. By design, each German source sentence in INES includes an...
Read Moreby Beatrice Savoldi | Oct 9, 2023 | Corpora | 0
GeNTE (Gender-Neutral Translation Evaluation) is a natural, bilingual corpus designed to benchmark the ability of machine translation systems to generate gender-neutral translations. Built from European Parliament speeches,...
Read Moreby Marco Gaido | Jul 7, 2023 | Corpora | 0
EC Short Clips is a test set dedicated to evaluate automatic subtitling systems.
Read Moreby Marco Gaido | Jul 7, 2023 | Corpora | 0
EuroParl Interviews is a test set dedicated to evaluate automatic subtitling systems.
Read Moreby Matteo Negri | Jun 1, 2023 | Corpora | 0
Multilingual benchmark built from European Parliament speeches and annotated with Named Entities and Terminology
Read Moreby Mauro Cettolo | May 30, 2023 | Corpora | 0
Annotation of dubbing segments based on the Heroes corpus
Read More
Our #PickOfTheWeek 📚: “High-Fidelity Simultaneous Speech-To-Speech Translation” by @t0m1ab, @lmazare, @EXGRV, @ptrkprz, @honualx and @neilzegh
🎙️➡️🌍 This work shows how a full-duplex SpeechLLM architecture can be adapted for simultaneous S2ST and S2TT.
Can LLM Agents Learn how to behave According to your preferences?
PersonaAgent is a framework that combines personalized memory and tool use to help LLM agents adapt their actions to individual users without fine-tuning the underlying model.
Our #PickOfTheWeek, selected by @mgaido91: "Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models" by Xuanru Zhou, Yiwen Shao, Jiahong Li, and Dong Yu 🎧
Can instruction-free training power the next generation of audio-language models?
Cool to see we are not the only ones working on building SpeechLLMs with instruction-following capabilities without requiring LLM adaptation and instruction-following data that is expensive to collect!
https://arxiv.org/abs/2608.18132
@fbk_mt
Our #PickOfTheWeek, selected by @BeatriceSavoldi:
"The AI Observatory: A Public Measure of Real-World AI Use" by @ShayneRedford, @AnkaReuel, @zoeykii et al.
A crucial framework tracking how AI models are deployed, adopted, and evaluated beyond lab benchmarks.
Pick of the week @fbk_mt
📚"The AI Observatory: A Public Measure of Real-World AI Use"
A study and taxonomy to explore real conversations across multiple datasets.
https://www.dataprovenance.org/ai_observatory.pdf