Large Language Model Evaluation and Sign Language Processing.
By Eleftherios Avramidis

Eleftherios Avramidis will give a talk on Large Language Model Evaluation and Sign Language Processing.

Abstract

Large Language Model Evaluation: Large Language Models (LLMs) are a popular technology with enormous capabilities, but despite it being quickly adopted in several fields, there is limited comprehension of their performance on a wide range of use cases. The talk will present my work to discover and analyze quality dimensions for evaluating use cases, and employing challenge-based diagnostic evaluation to reveal the weak points of the systems.

Sign Language Processing: The second part of the talk will focus on my research into the technologies required for machine translation of sign languages, and specifically translation from text of a spoken language to sign language and vice versa. This work includes published activities in data collection, machine translation to intermediate representation and explicit modelling of sign language elements, leveraging multimodal neural architectures and transfer learning methods.

Biography

Eleftherios Avramidis is a senior researcher at the German Research Centre for Artificial Intelligence. Since 2007, he has conducted research in the field of Natural Language Processing, and in 2018 he completed his PhD in Quality Estimation of Machine Translation. More recently, he has focused on multimodal approaches for sign language technologies and evaluation protocols for generative AI. He has taught and supervised theses at TU Berlin and the Berlin University of the Arts, and published his work in numerous venues of Computational Linguistics. He has also been organising shared tasks at the Conference on Machine Translation (WMT) for several years.