Luisa Bentivogli

Luisa Bentivogli heads the Machine Translation Group at Fondazione Bruno Kessler (FBK). From 2008 to 2013 she was assigned to the Centre for the Evaluation of Language and Communication Technologies (CELCT), first as Research Manager with the role of coordinating the activities of the centre (2008-2012), then as Director of the centre (2012-2013).

Her research interests include the evaluation and analysis of Machine Translation (MT) and Speech Translation (ST) systems, creation and annotation of multilingual resources, translation technologies for translators, crowdsourcing for natural language processing, and computational lexicography in a multilingual environment. Her current focus is on creating multilingual resources for speech translation (ST) and on assessing and mitigating gender bias in MT and ST, for which she recently won an Amazon Research Award.

Over the years, she has been involved in several EU-funded projects, including EuroSub (2021-2022), MateDUB++ (2021), MateDUB (2020), CEF Data Marketplace (2019-2021), EnetCollect (2017-2021), ModernMT (2015-2017), CRACKER (2015-2017), MosesCore (2012-2015), Excitement (2012-2014), EU-Bridge (2012-2014), MateCat (2011-2014), FlareNet (2008-2011), MEANING (2002-2005), in projects funded by the Autonomous Province of Trento, including LiveMemories (2008-2011), OntoText (2004-2007), WebFaq (2001-2004), and in job orders from private companies and public institutions (ongoing SmarTerp, TINIA).

She co-authored more than 90 scientific publications and serves as reviewer for journals, conferences and workshops. She regularly supervises internship, master, and PhD students. She has been involved in the organization of different tasks in several evaluation campaigns (e.g. IWSLT, RTE, SemEval, EVALITA) and other international events aimed at both the scientific community and translators (MT Marathon 2014, School of Advanced Technologies for Translators – SATT – 2016, 2017, 2018, 2019).

1. Piergentili, Andrea; Fucci, Dennis; Savoldi, Beatrice; Bentivogli, Luisa; Negri, Matteo,
   Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges,
   Proceedings of the First Workshop on Gender-Inclusive Translation Technologies (GITT 2023),
   European Association for Machine Translation,
   2023
   , pp. 71-
   83
2. Agarwal, Milind; Agrawal, Sweta; Anastasopoulos, Antonios; Bentivogli, Luisa; Bojar, Ondřej; Borg, Claudia; Carpuat, Marine; Cattoni, Roldano; Cettolo, Mauro; Chen, Mingda; Chen, William; Choukri, Khalid; Chronopoulou, Alexandra; Currey, Anna; Declerck, Thierry; Dong, Qianqian; Duh, Kevin; Estève, Yannick; Federico, Marcello; Gahbiche, Souhir; Haddow, Barry; Hsu, Benjamin; Mon Htut, Phu; Inaguma, Hirofumi; Javorský, Dávid; Judge, John; Kano, Yasumasa; Ko, Tom; Kumar, Rishu; Li, Pengwei; Ma, Xutai; Mathur, Prashant; Matusov, Evgeny; Mcnamee, Paul; Mccrae, John P.; Murray, Kenton; Nadejde, Maria; Nakamura, Satoshi; Negri, Matteo; Nguyen, Ha; Niehues, Jan; Niu, Xing; Ojha, Atul Kr.; Ortega, John E.; Pal, Proyag; Pino, Juan; van der Plas, Lonneke; Polák, Peter; Rippeth, Elijah; Salesky, Elizabeth; Shi, Jiatong; Sperber, Matthias; Stüker, Sebastian; Sudoh, Katsuhito; Tang, Yun; Thompson, Brian; Tran, Kevin; Turchi, Marco; Waibel, Alex; Wang, Mingxuan; Watanabe, Shinji; Zevallos, Rodolfo,
   Findings of the IWSLT 2023 Evaluation Campaign.,
   Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023),
   Association for Computational Linguistics,
   2023
   , pp. 1-
   61
3. Vanmassenhove, Eva; Savoldi, Beatrice; Bentivogli, Luisa; Daems, Joke; Hackenbuckner, Janica,
   Proceedings of the First Workshop on Gender-Inclusive Translation Technologies,
   Proceedings of the First Workshop on Gender-Inclusive Translation Technologies,
   European Association for Machine Translation,
   2023
4. Gaido, Marco; Fucci, Dennis; Negri, Matteo; Bentivogli, Luisa,
   How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation,
   in «»,
   Proceedings of the 9th Italian Conference on Computational Linguistics,
   ,
   vol. ,
   n. ,
   2023
   , pp. -
5. Fucci, Dennis; Gaido, Marco; Papi, Sara; Cettolo, Mauro; Negri, Matteo; Bentivogli, Luisa,
   Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection,
   in «»,
   Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,
   ,
   vol. ,
   n. ,
   2023
   , pp. 11505-
   11517
6. Savoldi, Beatrice; Gaido, Marco; Negri, Matteo; Bentivogli, Luisa,
   Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INES,
   Proceedings of the Eighth Conference on Machine Translation,
   2023
   , pp. 252-
   262
7. Fucci, Dennis; Gaido, Marco; Negri, Matteo; Cettolo, Mauro; Bentivogli, Luisa,
   No Pitch Left Behind: Addressing Gender Unbalance In Automatic Speech Recognition Through Pitch Manipulation,
   in «»,
   Proceedings of 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU),
   ,
   vol. ,
   n. ,
   2023
   , pp. -
   8
8. Piergentili, Andrea; Savoldi, Beatrice; Fucci, Dennis; Negri, Matteo; Bentivogli, Luisa,
   Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the {G}e{NTE} Corpus,
   Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,
   Association for Computational Linguistics,
   2023
   , pp. 14124-
   14140
9. Piazzolla, Silvia Alma; Savoldi, Beatrice; Bentivogli, Luisa,
   Good, but not always Fair: An Evaluation of Gender Bias for three Commercial Machine Translation Systems,
   in «HERMES»,
   n. 63,
   2023
10. Savoldi, Beatrice; Gaido, Marco; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco,
    Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation,
    in «»,
    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
    Association for Computational Linguistics,
    vol. ,
    n. ,
    2022
    , pp. 1807-
    1824
11. Savoldi, Beatrice; Gaido, Marco; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco,
    On the Dynamics of Gender Learning in Speech Translation,
    in «»,
    Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP),
    Association for Computational Linguistics,
    vol. ,
    n. ,
    2022
    , pp. 94-
    111
12. Karakanta, Alina; Bentivogli, Luisa; Cettolo, Mauro; Negri, Matteo; Turchi, Marco,
    Post-editing in Automatic Subtitling: A Subtitlers’ perspective,
    Proceedings of the 23rd Annual Conference of the European Association for Machine Translation,
    European Association for Machine Translation,
    2022
    , pp. 259-
    268
13. Bentivogli, Luisa; Cettolo, Mauro; Gaido, Marco; Karakanta, Alina; Negri, Matteo; Turchi, Marco,
    Extending the MuST-C Corpus for a Comparative Evaluation of Speech Translation Technology,
    Proceedings of the 23rd Annual Conference of the European Association for Machine Translation,
    European Association for Machine Translation,
    2022
    , pp. 359-
    360
14. Karakanta, Alina; Bentivogli, Luisa; Cettolo, Mauro; Negri, Matteo; Turchi, Marco,
    Towards a methodology for evaluating automatic subtitling,
    Proceedings of the 23rd Annual Conference of the European Association for Machine Translation,
    European Association for Machine Translation,
    2022
    , pp. 333-
    334
15. Anastasopoulos, Antonios; Barrault, Loc; Bentivogli, Luisa; Zanon Boito, Marcely; Bojar, Ondřej; Cattoni, Roldano; Currey, Anna; Dinu, Georgiana; Duh, Kevin; Elbayad, Maha; Emmanuel, Clara; Estève, Yannick; Federico, Marcello; Federmann, Christian; Gahbiche, Souhir; Gong, Hongyu; Grundkiewicz, Roman; Haddow, Barry; Hsu, Benjamin; Javorský, Dávid; Kloudová, Vĕra; Lakew, Surafel Melaku; Ma, Xutai; Mathur, Prashant; Mcnamee, Paul; Murray, Kenton; Nǎdejde, Maria; Nakamura, Satoshi; Negri, Matteo; Niehues, Jan; Niu, Xing; John, Ortega; Pino, Juan; Salesky, Elizabeth; Shi, Jiatong; Sperber, Matthias; Stüker, Sebastian; Sudoh, Katsuhito; Turchi, Marco; Virkar, Yogesh; Waibel, Alexander; Wang, Changhan; Watanabe, Shinji,
    Findings of the IWSLT 2022 Evaluation Campaign.,
    Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022),
    Association for Computational Linguistics,
    2022
    , pp. 98-
    157
16. Gaido, Marco; Savoldi, Beatrice; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco,
    How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation,
    in «»,
    Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021,
    Association for Computational Linguistics,
    vol. ,
    n. ,
    2021
    , pp. 3576-
    3589
17. Savoldi, Beatrice; Gaido, Marco; Bentivogli, Luisa; Negri, Marco; Turchi, Marco,
    Gender bias in machine translation,
    in «TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS»,
    ,
    ,
    vol. 9,
    n. ,
    2021
    , pp. 845-
    874
18. Cattoni, Roldano; Di Gangi, Mattia Antonino; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco,
    MuST-C: A multilingual corpus for end-to-end speech translation,
    in «COMPUTER SPEECH AND LANGUAGE»,
    vol. 66,
    2021
19. Bentivogli, Luisa; Cettolo, Mauro; Gaido, Marco; Karakanta, Alina; Martinelli, Alberto; Negri, Matteo; Turchi, Marco,
    Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?,
    Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers),
    Association for Computational Linguistics,
    2021
    , pp. 2873-
    2887
20. Gaido, Marco; Rodríguez, Susana; Negri, Matteo; Bentivogli, Luisa; Turchi, Marco,
    Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translation,
    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,
    Association for Computational Linguistics,
    2021
    , pp. 1707-
    1716
21. Savoldi, Beatrice; Bentivogli, Luisa,
    Gender bias and Machine Translation: On first looking into parallel corpora,
    Using Corpora in Contrastive and Translation Studies,
    2021
22. Gaido, Marco; Savoldi, Beatrice; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco,
    Breeding Gender-aware Direct Speech Translation Systems,
    Proceedings of the 28th International Conference on Computational Linguistics,
    2020
    , pp. 3951-
    3964
23. Kamran, Amir; Dzeguze, Dace; van der Meer, Jaap; Panic, Milica; Cattelan, Alessandro; Patrioli, Daniele; Bentivogli, Luisa; Turchi, Marco,
    CEF Data Marketplace: Powering a Long-term Supply of Language Data,
    Proceedings of the 22nd Annual Conference of the European Association for Machine Translation,
    European Association for Machine Translation,
    2020
    , pp. 459-
    460
24. Bentivogli, Luisa; Savoldi, Beatrice; Negri, Matteo; Di Gangi, Mattia Antonino; Cattoni, Roldano; Turchi, Marco,
    Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus,
    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,
    Association for Computational Linguistics,
    2020
    , pp. 6923-
    6933
25. Di Gangi, Mattia A.; Cattoni, Roldano; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco,
    MuST-C: a Multilingual Speech Translation Corpus,
    Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers),
    Association for Computational Linguistics,
    2019
    , pp. 2012-
    2017
26. Tebbifakhr, A.; Bentivogli, L.; Negri, M.; Turchi, M.,
    Machine Translation for Machines: the Sentiment Classification Use Case,
    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,
    2019
    , pp. 1368-
    1374
27. Scansani, Randy; Bernardini, Silvia; Ferraresi, Adriano; Bentivogli, Luisa,
    Do translator trainees trust machine translation? An experiment on post-editing and revision,
    Proceedings of Machine Translation Summit XVII Volume 2: Translator, Project and User Tracks,
    vol. 2,
    2019
    , pp. 73-
    79
28. Scansani, Randy; Bentivogli, Luisa; Bernardini, Silvia; Ferraresi, Adriano,
    MAGMATic: A Multi-domain Academic Gold Standard with Manual Annotation of Terminology for Machine Translation Evaluation,
    Proceedings of Machine Translation Summit XVII Volume 1: Research Track,
    vol. 1,
    2019
    , pp. 78-
    86
29. Bentivogli, Luisa; Bisazza, Arianna; Cettolo, Mauro; Federico, Marcello,
    Neural versus phrase-based MT quality: An in-depth analysis on English–German and English–French,
    in «COMPUTER SPEECH AND LANGUAGE»,
    vol. 49,
    2018
    , pp. 52-
    70
30. Bentivogli, Luisa; Cettolo, Mauro; Federico, Marcello; Federmann, Christian,
    Machine Translation Human Evaluation: an investigation of evaluation based on Post-Editing and its relation with Direct Assessment,
    Proceedings of the 15th International Workshop on Spoken Language Translation (IWSLT 2018),
    2018
    , pp. 62-
    69
31. Nicolas, Lionel; Lyding, Verena; Bentivogli, Luisa; Sangati, Federico; Monti, Johanna; Russo, Irene; Gretter, Roberto; Falavigna, Giuseppe Daniele,
    EnetCollect in Italy,
    Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018),
    CEUR Workshop Proceedings,
    vol. 2253,
    2018
32. Cettolo, Mauro; Federico, Marcello; Bentivogli, Luisa; Niehues, Jan; Stüker, Sebastian; Sudoh, Katsuitho; Yoshino, Koichiro; Federmann, Christian,
    Overview of the IWSLT 2017 Evaluation Campaign,
    Proceedings of the 14th International Workshop on Spoken Language Translation,
    2017
    , pp. 2-
    14
33. Bentivogli, Luisa; Dagan, Ido Kalman; Magnini, Bernardo,
    The Recognizing Textual Entailment Challenges: Datasets and Methodologies,
    Handbook of Linguistic Annotation,
    Springer,
    2017
    , pp. 1119-
    1147
34. Scansani, Randy; Federico, Marcello; Bentivogli, Luisa,
    Assessing the Use of Terminology in Phrase-Based Statistical Machine Translation for Academic Course Catalogues Translation,
    Proceedings of the Forth Italian Conference on Computational Linguistics (CLiC-it 2017),
    2017
35. Bentivogli, Luisa; Bernardi, Raffaella; Marelli, Marco; Menini, Stefano; Baroni, Marco; Zamparelli, Roberto,
    SICK through the SemEval glasses: Lessons learned from the evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment,
    in «LANGUAGE RESOURCES AND EVALUATION»,
    vol. 50,
    n. 1,
    2016
    , pp. 95-
    124
36. Bentivogli, Luisa; Bertoldi, Nicola; Cettolo, Mauro; Federico, Marcello; Negri, Matteo; Turchi, Marco,
    On the Evaluation of Adaptive Machine Translation for Human Post-Editing,
    in «IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING»,
    vol. 24,
    n. 2,
    2016
    , pp. 388-
    399
37. Bentivogli, Luisa; Cettolo, Mauro; Farajian, Mohammad; Federico, Marcello,
    WAGS: A Beautiful English-Italian Benchmark Supporting Word Alignment Evaluation on Rare Words,
    Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016),
    European Language Resources Association (ELRA),
    2016
    , pp. 3535-
    3542
38. Bentivogli, Luisa; Bisazza, Arianna; Cettolo, Mauro; Federico, Marcello,
    Neural versus Phrase-Based Machine Translation Quality: a Case Study,
    Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP),
    Association for Computational Linguistics (ACL),
    2016
39. Germann, U.; Barbu, Eduard; Bentivogli, Luisa; Bertoldi, Nicola; Bogoychev, N.; Buck, Christian; Caroselli, D.; Carvalho, L.; Cattelan, A.; Cattoni, Roldano; Cettolo, Mauro; Federico, Marcello; Haddow, B.; Madl, D.; Mastrostefano, L.; Mathur, Prashant; Ruopp, A.; Samiotou, A.; Sudharshan, V.; Trombetti, M.; van der Meer, J.,
    Modern MT: A New Open-Source Machine Translation Platform for the Translation Industry,
    in «BALTIC JOURNAL OF MODERN COMPUTING»,
    vol. 4,
    n. 2,
    2016
    , pp. 397-
    397
40. Barbu, Eduard; Escartín, Carla Parra; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco; Federico, Marcello; Mastrostefano, Luca; Orasan, Constantin,
    1st Shared Task on Automatic Translation Memory Cleaning: Preparation and Lessons Learned,
    Proceedings of the LREC 2016 2nd Workshop on Natural Language Processing for Translation Memories (NLP4TM 2016),
    2016
    , pp. 1-
    5
41. Bentivogli, Luisa; Federico, Marcello; Stüker, Sebastian; Cettolo, Mauro; Niehues, Jan,
    The IWSLT Evaluation Campaign: Challenges, Achievements, Future Directions,
    Proceedings of the LREC 2016 Workshop on Translation Evaluation: From Fragmented Tools and Data Sets to an Integrated Ecosystem,
    2016
    , pp. 14-
    19
42. Sprugnoli, Rachele; Moretti, Giovanni; Bentivogli, Luisa; Giuliani, Diego,
    Creating a ground truth multilingual dataset of news and talk show transcriptions through crowdsourcing,
    in «LANGUAGE RESOURCES AND EVALUATION»,
    2016
43. Barbu, Eduard; Parra Escartin, Carla; Bentivogli, Luisa; Negri, Matteo; Turchi, Marco; Orasan, Constantin; Federico, Marcello,
    The first Automatic Translation Memory Cleaning Shared Task,
    in «MACHINE TRANSLATION»,
    vol. 30,
    n. 3,
    2016
    , pp. 145-
    166
44. Cettolo, Mauro; Niehues, Jan; Stüker, Sebastian; Bentivogli, Luisa; Cattoni, Roldano; Federico, Marcello,
    The IWSLT 2016 Evaluation Campaign,
    Proceedings of the International Workshop on Spoken Language Translation,
    2016
45. Kotlerman, Lili; Dagan, Ido; Magnini, Bernardo; Bentivogli, Luisa,
    Textual entailment graphs,
    in «NATURAL LANGUAGE ENGINEERING»,
    vol. 21,
    n. 5,
    2015
    , pp. 699-
    724
46. Cettolo, Mauro; Niehues, Jan; Stüker, Sebastian; Bentivogli, Luisa; Cattoni, Roldano; Federico, Marcello,
    The IWSLT 2015 Evaluation Campaign,
    Proceedings of IWSLT,
    2015
47. Federico, Marcello; Negri, Matteo; Bentivogli, Luisa; Turchi, Marco,
    Assessing the Impact of Translation Errors on Machine Translation Quality with Mixed-effects Models.,
    Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP),
    2014
    , pp. 1643-
    1653
48. Marelli, M.; Bentivogli, Luisa; Baroni, M.; Bernardi, R.; Menini, Stefano; Zamparelli, R.,
    SemEval-2014 Task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment,
    Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014),
    2014
49. Marelli, M.; Menini, Stefano; Baroni, M.; Bentivogli, Luisa; Bernardi, R.; Zamparelli, R.,
    A SICK cure for the evaluation of compositional distributional semantic models,
    Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14),
    2014
50. Cettolo, Mauro; Niehues, J.; Stüker, S.; Bentivogli, Luisa; Federico, Marcello,
    Report on the 11th IWSLT Evaluation Campaign, IWSLT 2014,
    Proceedings of the International Workshop on Spoken Language Translation,
    Marcello Federico, Sebastian Stüker, François Yvon,
    2014
    , pp. 2-
    17
51. Girardi, Christian; Bentivogli, Luisa; Farajian, Mohammad Amin; Federico, Marcello,
    MT-EQuAl: a toolkit for human assessment of machine translation output,
    Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics,
    2014
    , pp. 120-
    123
52. Bentivogli, Luisa; Magnini, Bernardo,
    An Italian dataset of Textual Entailment Graphs for text exploration of customer interactions,
    Proceedings of the First Italian Conference on Computational Linguistics CLiC-it 2014,
    2014
    , pp. 63-
    66
53. Bentivogli, Luisa; Marchetti, A.; Pianta, Emanuele,
    The News People Search Task at EVALITA 2011: Evaluating Cross-Document Coreference Resolution of Named Person Entities in Italian News,
    Evaluation of Natural Language and Speech Tools for Italian - Proceedings of Evalita 2011,
    Springer-Verlag,
    2013
54. Negri, Matteo; Marchetti, A.; Mehdad, Yashar; Bentivogli, Luisa; Giampiccolo, Danilo,
    Semeval-2013 Task 8: Cross-lingual Textual Entailment for Content Synchronization,
    Proceedings of the Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013).,
    2013
    , pp. 25-
    33
55. Dzikovska, M.; Nielsen, R.; Brew, C.; Leacock, C.; Giampiccolo, Danilo; Bentivogli, Luisa; Clark, P.; Dagan, I.; Dang, H.,
    SemEval-2013 Task 7: The Joint Student Response Analysis and 8th Recognizing Textual Entailment Challenge,
    Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013),
    2013
56. Sprugnoli, Rachele; Moretti, Giovanni; Fuoli, M.; Giuliani, Diego; Bentivogli, Luisa; Pianta, Emanuele; Gretter, Roberto; Brugnara, Fabio,
    Comparing two methods for crowdsourcing speech transcription,
    Proceedings of ICASSP 2013,
    The Institute of Electrical and Electronics Engineers,Incorporated,
    2013
    , pp. 8116-
    8120
57. Cettolo, Mauro; J., Niehues; S., Stüker; Bentivogli, Luisa; Federico, Marcello,
    Report on the 10th IWSLT Evaluation Campaign,,
    Proceedings of the 10th Workshop on Spoken Language Translation,
    2013
58. Sprugnoli, Rachele; Moretti, Giovanni; Fuoli, M.; Bentivogli, Luisa; Pianta, Emanuele; Giuliani, Diego; Gretter, Roberto; Brugnara, Fabio,
    Uso del crowdsourcing per trascrizioni di alta qualità del linguaggio parlato: metodologie a confronto.,
    Multimodalità e Multilingualità. La sfida più avanzata della comunicazione orale,
    Bulzoni,
    2013
59. Negri, Matteo; A., Marchetti; Mehdad, Yashar; Bentivogli, Luisa; Giampiccolo, Danilo,
    Semeval-2012 Task 8: Cross-lingual Textual Entailment for Content Synchronization.,
    Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM),,
    Association for Computational Linguistics (ACL),
    2012
    , pp. 399-
    407
60. Negri, Matteo; Mehdad, Yashar; Giampiccolo, Danilo; A., Marchetti; Bentivogli, Luisa,
    Chinese Whispers: Cooperative Paraphrase Acquisition,
    Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12),
    European Language Resources Association (ELRA),
    2012
    , pp. 2659-
    2665
61. Federico, Marcello; S., Stüker; Bentivogli, Luisa; M., Paul; Cettolo, Mauro; T., Herrmann; J., Niehues; G., Moretti,
    The IWSLT 2011 Evaluation Campaign on Automatic Talk Translation,
    Proceedings of the International Conference on Language Resources and Evaluation (LREC),
    2012
    , pp. 3543-
    3550
62. M., Paul; E., Sumita; Bentivogli, Luisa; Federico, Marcello,
    Crowd-based MT Evaluation for non-English Target Languages,
    2012
    , pp. 229-
    236
63. Bentivogli, Luisa; A., Marchetti; Pianta, Emanuele,
    The News People Search Task at EVALITA 2011: Evaluating Cross-document Coreference Resolution of Named Person Entities in Italian News,
    2012
64. Federico, Marcello; Cettolo, Mauro; Bentivogli, Luisa; Michael, Paul; Sebastian, Stüker,
    Overview of the IWSLT 2012 Evaluation Campaign,
    Proceedings of the International Workshop on Spoken Language Translation (IWSLT),
    2012
    , pp. 12-
    33
65. Negri, Matteo; Bentivogli, Luisa; Mehdad, Yashar; Giampiccolo, Danilo; Marchetti, Alessandro,
    Divide and Conquer: Crowdsourcing the Creation of Cross-Lingual Textual Entailment Corpora.,
    Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing,
    2011
    , pp. 670-
    679
66. Bentivogli, Luisa; Federico, Marcello; Moretti, Giovanni; M., Paul,
    Getting Expert Quality from the Crowd for Machine Translation Evaluation,
    2011
67. Bentivogli, Luisa; P., Clark; I., Dagan; H., Dang; D., Giampiccolo,
    The Seventh PASCAL Recognizing Textual Entailment Challenge,
    Proceedings of the Text Analysis Conference (TAC 2011),
    2011
68. Federico, Marcello; Bentivogli, Luisa; M., Paul; S., Stüker,
    Overview of the IWSLT 2011 Evaluation Campaign,
    2011
69. Bentivogli, Luisa; Cabrio, Elena; Dagan, Ido Kalman; Giampiccolo, Danilo; M., Lo Leggio; Magnini, Bernardo,
    Building Textual Entailment Specialized Data Sets: a Methodology for Isolating Linguistic Phenomena Relevant to Inference,
    Seventh conference on International Language Resources and Evaluation (LREC 2010),
    European Language Resources Association (ELRA),
    2010
    , pp. 3542-
    3549
70. Bentivogli, Luisa; Giuliano, Claudio; Pamela, Forner; Alessandro, Marchetti; Pianta, Emanuele; Tymoshenko, Kateryna,
    Extending English ACE 2005 Corpus Annotation with Ground-truth Links to Wikipedia,
    Proceedings of the 2nd Workshop on The People’s Web Meets NLP: Collaboratively Constructed Semantic Resources,
    2010
71. A., Abad; Bentivogli, Luisa; Dagan, Ido Kalman; Giampiccolo, Danilo; S., Mirkin; Pianta, Emanuele; A., Stern,
    A Resource for Investigating the Impact of Anaphora and Coreference on Inference,
    Proceedings of the Language Resources and Evaluation Conference (LREC-2010),
    2010
    , pp. 128-
    135
72. Bentivogli, Luisa; P., Clark; I., Dagan; H., Dang; D., Giampiccolo,
    The Sixth PASCAL Recognizing Textual Entailment Challenge,
    2010
73. Bentivogli, Luisa; Dagan, Ido Kalman; Hoa, Dang; Giampiccolo, Danilo; Medea Lo, Leggio; Magnini, Bernardo,
    Considering Discourse References in Textual Entailment Annotation,
    5th International Conference on Generative Approaches to the Lexicon (GL 2009),
    2009
74. Bentivogli, Luisa; Dagan, Ido Kalman; Hoa, Dang; Giampiccolo, Danilo; Magnini, Bernardo,
    The Fifth PASCAL Recognizing Textual Entailment Challenge,
    TAC 2009 Workshop,
    no publisher,
    2009
75. Bentivogli, Luisa; Girardi, Christian; Pianta, Emanuele,
    Creating a Gold Standard for Person Cross-Document Coreference Resolution in Italian News,
    LREC 2008 Workshop on Resources and Evaluation for Identity Matching, Entity Resolution and Entity Management,
    European Language Resources Association,
    2008
76. Bentivogli, Luisa; Fioravanti, Anna; Fontanive, Patrizia; Pianta, Emanuele; Speranza, Manuela,
    Linee guida per la creazione della Scheda Ontologica e per l´annotazione della coreferenza globale delle entità presenti in I-CAB,
    2007
77. Bentivogli, Luisa; Pianta, Emanuele,
    Linee guida per la creazione del Gold Standard per la risoluzione della coreferenza di entità di tipo PERSONA in Adige-500K,
    2007
78. Pianta, Emanuele; Bentivogli, Luisa; Girardi, Christian; Magnini, Bernardo,
    Representing and Accessing Multilevel Linguistic Annotation using the MEANING Format,
    EACL-06 Workshop on Multi-dimensional Markup in Natural Language Processing (NLPXML-2006),
    ACL,
    2006
    , pp. 77-
    80
79. Bentivogli, Luisa; Magnini, Bernardo,
    ONTOTEXT Portal (Version 31 December 2006): Functional Specifications,
    2006
80. Bentivogli, Luisa; Magnini, Bernardo,
    Accesso all´informazione basato su conoscenza: interfaccia del portale ONTOTEXT (versione 31/12/2006),
    2006
81. Bentivogli, Luisa; Pianta, Emanuele,
    Exploiting parallel texts in the creation of multilingual semantically annotated resources: the MultiSemCor Corpus,
    in «NATURAL LANGUAGE ENGINEERING»,
    Cambridge University Press,
    vol. 11,
    n. 3,
    2005
    , pp. 247-
    261
82. Bentivogli, Luisa; A., Bocco; Pianta, Emanuele,
    ArchiWordNet: Integrating WordNet with Domain-Specific Knowledge,
    Second International WordNet Conference [GWC 2004],
    Masaryk University,
    2004
    , pp. 39-
    46
83. Bentivogli, Luisa; Pianta, Emanuele,
    Extending WordNet with Syntagmatic Information,
    Second International WordNet Conference [GWC 2004],
    Masaryk University,
    2004
    , pp. 47-
    53
84. M., Ranieri; Pianta, Emanuele; Bentivogli, Luisa,
    Browsing Multilingual Information with the MultiSemCor Web Interface,
    LREC 2004 Workshop on `The Amazing Utility of Parallel and Comparable Corpora`,
    European Language Resources Association,
    2004
    , pp. 38-
    41
85. Pianta, Emanuele; Bentivogli, Luisa,
    Annotating Discontinuous Structures in XML: the Multiword Case,
    LREC 2004 Satellite Workshop on XML-based richly annotated corpora,
    2004
    , pp. 30-
    37
86. Pianta, Emanuele; Bentivogli, Luisa,
    Knowledge Intensive Word Alignment with KNOWA,
    20th International Conference on Computational Linguistics [COLING 2004],
    ACL,
    2004
    , pp. 1086-
    1092
87. Bentivogli, Luisa; P., Forner; Magnini, Bernardo; Pianta, Emanuele,
    Revising the WORDNET DOMAINS Hierarchy: semantics, coverage and balancing,
    MLR2004: PostCOLING Workshop on Multilingual Linguistic Resources,
    2004
    , pp. 101-
    108
88. Bentivogli, Luisa; P., Forner; Pianta, Emanuele,
    Evaluating Cross-Language Annotation Transfer in the MultiSemCor Corpus,
    20th International Conference on Computational Linguistics [COLING 2004],
    ACL,
    2004
    , pp. 364-
    370
89. Pianta, Emanuele; Bentivogli, Luisa,
    Annotating Discontinuous Structures in XML: the Multiword Case,
    2003
90. Pianta, Emanuele; Bentivogli, Luisa,
    Translation as Annotation,
    AI*IA 2003 Workshop `Topics and Perspectives of Natural Language Processing in Italy`,
    2003
    , pp. 40-
    48
91. Bentivogli, Luisa; Girardi, Christian; Pianta, Emanuele,
    The MEANING Italian Corpus,
    Corpus Linguistics 2003 Conference,
    2003
    , pp. 103-
    112
92. Bentivogli, Luisa; Pianta, Emanuele,
    Beyond Lexical Units: Enriching WordNets with Phrasets,
    10th Conference of the European Chapter of the Association for Computational Linguistics (EACL`03),
    ACL,
    2003
    , pp. 67-
    70
93. Pianta, Emanuele; Bentivogli, Luisa; Girardi, Christian,
    MultiWordNet: developing an aligned multilingual database,
    First International Conference on Global WordNet,
    2002
    , pp. 293-
    302
94. Bentivogli, Luisa; Pianta, Emanuele,
    Opportunistic Semantic Tagging,
    Third International Conference on Language Resources and Evaluation (LREC 2002),
    2002
    , pp. 1401-
    1406
95. Bentivogli, Luisa; Pianta, Emanuele,
    Detecting hidden multiwords in bilingual dictionaries,
    Tenth EURALEX International Congress [EURALEX-2002],
    2002
    , pp. 785-
    793
96. Bentivogli, Luisa; Girardi, Christian; Giuliano, Claudio; Pianta, Emanuele,
    The MEANING Italian Corpus,
    2002
97. Bentivogli, Luisa; Girardi, Christian,
    The MEANING Italian Corpus,
    2002
98. Bentivogli, Luisa; Pianta, Emanuele,
    Beyond Lexical Units: Enriching WordNets with Phrasets,
    2002
99. Bentivogli, Luisa; Pianesi, Fabio; Pianta, Emanuele,
    PhiloNet: creating semantic concordances for the analysis of philosophical texts,
    2001 Joint International Conference of the Association for Computers and the Humanities and the Association for Literary and Linguistic Computing (ACH/ALLC 2001),
    2001
    , pp. 139-
    141
100. Bentivogli, Luisa; Forner, Pamela; Pianta, Emanuele,
     Manual word alignment guidelines for the MultiSemCor project,
     2001
101. Bentivogli, Luisa; Pianesi, Fabio; Pianta, Emanuele,
     From Word-based to Concept-based Text Analysis: the PhiloNet Project,
     Workshop Informatica umanistica: filosofia e risorse digitali,
     2000
102. Bentivogli, Luisa; Pianta, Emanuele,
     Looking for Lexical Gaps,
     Ninth EURALEX International Congress [EURALEX 2000],
     2000
     , pp. 663-
     669
103. Bentivogli, Luisa; Pianta, Emanuele; Pianesi, Fabio,
     Coping with Lexical Gaps when Building Aligned Multilingual Wordnets,
     Second International Conference on Language Resources and Evaluation (LREC 2000),
     2000
     , pp. 993-
     997
104. Bentivogli, Luisa,
     Relazioni lessicali e semantiche nella costruzione di un lessico computazionale multilingue: Problematiche Tecniche e Filosofiche,
     1999