The PF-STAR project intended to contribute to establish future activities in the field of multisensorial and multilingual communication (interface technologies) on firmer bases by providing technological baselines, comparative evaluations, and assessment of prospects of core technologies, which future research and development efforts can build from. To this end, the project addressed three crucial areas: technologies for speech-to-speech translation, the detection and expressions of emotional states, and core speech technologies for children. For each of them, promising technologies/approaches were selected, further developed and aligned towards common baselines. The results were assessed and evaluated with respect to both their performances and future prospects. To maximise the impact, the duration of the project was limited to 24 months, and the workplan was designed to delivered results in two stages: at mid-project term (month 14), and at the end of the project. This permitted to make relevant results available as soon as possible, and in particular on time for them to be used during the preparatory phase of the first call of FP6. The Lehrstuhl für Informatik 6 was involved in the comparative evaluation and further development of speech translation technologies. The statistical approach was compared to an interlingua based approach. After the evaluation phase, the two approaches were further developed and aligned towards common baselines. PF-STAR was supported by the European Union.
Our #PickOfTheWeek 📚: “High-Fidelity Simultaneous Speech-To-Speech Translation” by @t0m1ab, @lmazare, @EXGRV, @ptrkprz, @honualx and @neilzegh
🎙️➡️🌍 This work shows how a full-duplex SpeechLLM architecture can be adapted for simultaneous S2ST and S2TT.
Can LLM Agents Learn how to behave According to your preferences?
PersonaAgent is a framework that combines personalized memory and tool use to help LLM agents adapt their actions to individual users without fine-tuning the underlying model.
Our #PickOfTheWeek, selected by @mgaido91: "Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models" by Xuanru Zhou, Yiwen Shao, Jiahong Li, and Dong Yu 🎧
Can instruction-free training power the next generation of audio-language models?
Cool to see we are not the only ones working on building SpeechLLMs with instruction-following capabilities without requiring LLM adaptation and instruction-following data that is expensive to collect!
https://arxiv.org/abs/2608.18132
@fbk_mt
Our #PickOfTheWeek, selected by @BeatriceSavoldi:
"The AI Observatory: A Public Measure of Real-World AI Use" by @ShayneRedford, @AnkaReuel, @zoeykii et al.
A crucial framework tracking how AI models are deployed, adopted, and evaluated beyond lab benchmarks.
Pick of the week @fbk_mt
📚"The AI Observatory: A Public Measure of Real-World AI Use"
A study and taxonomy to explore real conversations across multiple datasets.
https://www.dataprovenance.org/ai_observatory.pdf