The INclusive Evaluation Suite (INES) is a test set designed to assess MT systems ability to produce gender-inclusive translations for the German→English language pair. By design, each German source sentence in INES includes an expression that can be rendered by means of either an inclusive or non-inclusive expressions in the English target language.
Overall, INES comprises 162 manually curated German sentences, which are annotated with their corresponding (IN/N-IN) English expression.
For full details about the dataset, see the reference paper below.
How to obtain INES
The INES test suite is released under a Creative Commons Attribution 4.0 International license (CC BY 4.0).
INES contains post-edited text data generated with the OpenAI proprietary GPT model.
Please refer to OpenAI “Terms of Use” for details.
Reference papers
- If you use INES in your work, please cite the following paper:
Beatrice Savoldi, Marco Gaido, Matteo Negri, Luisa Bentivogli.
“Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INES“.
In Proceedings of the Eight Conference on Machine Translation (WMT 2023), 6th–10th December 2023, Singapore.
