MCIF
MCIF (Multimodal Crosslingual Instruction Following) is a multilingual human-annotated benchmark based on scientific talks that is designed to evaluate instruction-following in crosslingual, multimodal settings over both short-...
Read Moreby Beatrice Savoldi | Jan 13, 2025 | Corpora | 0
mGeNTE (Multilingual Gender-Neutral Translation Evaluation) is a natural, multilingual corpus designed to benchmark gender-neutral language and automatic translation.mGente is built upon European Parliament speech data extracted...
Read Moreby Beomseok Lee | Aug 21, 2024 | Corpora | 0
Spoken Language Understanding (SLU) involves interpreting spoken input using Natural Language Processing (NLP). Voice assistants like Alexa and Siri are real-world examples of SLU applications. The core tasks in SLU include...
Read Moreby Mauro Cettolo | Apr 30, 2024 | Corpora | 0
Ready-to-use version for MT research purposes of the multilingual transcriptions of TED talks
Read Moreby Dennis Fucci | Oct 20, 2023 | Corpora | 0
Text corpora for Spanish, French, and Italian containing gendered words referring to the first-person speaker
Read Moreby Beatrice Savoldi | Oct 19, 2023 | Corpora | 0
The INclusive Evaluation Suite (INES) is a test set designed to assess MT systems ability to produce gender-inclusive translations for the German→English language pair. By design, each German source sentence in INES includes an...
Read Moreby Beatrice Savoldi | Oct 9, 2023 | Corpora | 0
GeNTE (Gender-Neutral Translation Evaluation) is a natural, bilingual corpus designed to benchmark the ability of machine translation systems to generate gender-neutral translations. Built from European Parliament speeches,...
Read Moreby Marco Gaido | Jul 7, 2023 | Corpora | 0
EC Short Clips is a test set dedicated to evaluate automatic subtitling systems.
Read Moreby Marco Gaido | Jul 7, 2023 | Corpora | 0
EuroParl Interviews is a test set dedicated to evaluate automatic subtitling systems.
Read Moreby Matteo Negri | Jun 1, 2023 | Corpora | 0
Multilingual benchmark built from European Parliament speeches and annotated with Named Entities and Terminology
Read Moreby Mauro Cettolo | May 30, 2023 | Corpora | 0
Annotation of dubbing segments based on the Heroes corpus
Read More
Our #PickOfTheWeek, selected by @BeatriceSavoldi:
"The AI Observatory: A Public Measure of Real-World AI Use" by @ShayneRedford, @AnkaReuel, @zoeykii et al.
A crucial framework tracking how AI models are deployed, adopted, and evaluated beyond lab benchmarks.
Pick of the week @fbk_mt
📚"The AI Observatory: A Public Measure of Real-World AI Use"
A study and taxonomy to explore real conversations across multiple datasets.
https://www.dataprovenance.org/ai_observatory.pdf
Can algorithmic gender prediction ever be valid?
Check out this week's top pick by @lina_conti: "Algorithmic Gender Prediction Is Illegitimate, But Gender Imputation Can Yield Valid Measurements" by @evandongyx & @ang3linawang.
Pick of the week by @evandongyx & @ang3linawang:
https://arxiv.org/pdf/2608.13444
Predicting gender from images or names can reveal discrimination. But the practice itself harms trans people. This paper works through when that tradeoff might be justified and how to do it responsibly.
Our pick of the week by
@dhairya_su47605
: "Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression" by @hanqi_xiao, @yilin_sung, @EliasEskin and @mohitban47
#Quantization #Interpretibility
#PickoftheWeek @fbk_mt
Super cool paper on leavaraging Interpretability for Compression!
https://arxiv.org/pdf/2504.07389
Our pick of the week:
"Large Language Diffusion Model" by Shen Nie, Fengqi Zhu, @ZebinYou, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, @LiChongxuan
It is very cool to see how the researcher combine diffusion model and transformer blocks to train