Seminars

Investigating self-supervised speech models ability to classify animal vocalizations: The case of gibbon's vocal identity
Jules Cauzinille

Abstract: With the advent of pre-trained self-supervised learning (SSL) models, speech processing research is showing increasing interest towards disentanglement and explainability. Amongst other methods, probing speech classifiers has emerged as a promising approach to gain new insights into SSL models out-of-domain performances. We explore knowledge transfer capabilities of pre-trained speech models with vocalizations from the closest living relatives of humans: non-human primates. We focus on identifying the vocal signatures of northern grey gibbons (Hylobates funereus) with probing and layer-wise analysis of state-of-the-art SSL speech models compared to pre-trained bird species classifiers and audio tagging models. By testing the reliance of said models on background noise and timewise information, as well as performance variations across layers, we propose a new understanding of the mechanisms underlying speech models efficacy as bioacoustic tools.

When: Mar 14, 2024 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Impact of the Nonverbal Behavior of Virtual Audience on Users’ Perception of Social Attitudes
Marion Ristorcelli

Abstract: In a virtual reality public speaking training system, it is essential to control the audience’s nonverbal behavior in order to simulate different attitudes. The virtual audience’s social attitude is generally represented by a two-dimensional valence-arousal model describing the opinion and engagement of virtual characters. In this article, we argue that the valence-arousal representation is not sufficient to describe the user’s perception of a virtual character’s social attitude. We propose a three-dimensional model by dividing the valence axis into two dimensions representing the epistemic and affective stance of the virtual character, reflecting the character’s agreement and emotional reaction. To assess the perception of the virtual characters’ nonverbal behavior on these two new dimensions, we conducted a perceptive study in virtual reality with 44 participants who evaluated 50 animations combining multimodal nonverbal behavioral signals such as head movements, facial expressions, gaze direction and body posture. The results of our experiment show that, in fact, the valence axis should be divided into two axes to take into account the perception of the virtual character’s epistemic and affective stance. Furthermore, the results show that one behavioral signal is predominant for the evaluation of each dimension: head movements for the epistemic dimension and facial expressions for the affective dimension. These results provide useful guidelines for designing the nonverbal behavior of a virtual audience for social attitudes simulation.

When: Mar 07, 2024 at 13:00 | Where: Zoom and Luminy | Language: French (presentation), English (slides) | Slides

Language acquisition: do children and language models follow similar learning stages?
Linnea Evanson

Abstract: During language acquisition, children follow a typical sequence of learning stages. We test whether, during its training, GPT-2 exhibits stages of language acquisition comparable to those observed in children aged between 18 months and 6 years, using 96 zero-shot linguistic probes. We then compare these evaluations with the behavior of 54 children during language production. Our analyses reveal three main findings. First, similarly to children, the language models tend to learn linguistic skills in a systematic order. Second, this learning scheme is parallel: the language tasks that are learned last improve from the very first training steps. Third, some – but not all – learning stages are shared between children and these language models. Overall, these results shed new light on the principles of language acquisition, and highlight important divergences in how humans and modern algorithms learn to process natural language.

When: Feb 22, 2024 at 11:00 | Where: Zoom | Language: English

Information transfers in conversation, automatic detection using language models
Eliot Maës

Abstract: FR. Deux personnes en conversation apprennent peu à peu à se connaitre, du fait de la mise en place du common ground. Nous étudions le développement de cette connaissance partagée au travers du prisme des transfers d’information dans la conversation, et présentons différentes méthodes pour essayer de localiser ces moments grâce à des modèles de langue. EN. Speakers in conversation gradually learn to know each other through the development of a common ground. We investigate the how this shared knowledge appears through the lens of information transfers in the conversation and present various methods to try to locate these moments using language models.

When: Feb 15, 2024 at 13:00 | Where: Zoom and Luminy | Language: French (presentation), English (slides)

Conflict management training through simulation with conversational agent - Progress and perspectives
Alice Delbosc

Abstract: This presentation serves as a follow-up to the one I presented during my internship in June 2023. My ongoing work revolves around enhancing the generation of non-verbal behavior, with the overarching objective of developing a simulation tool for conflict management utilizing an animated conversational agent. In addition to outlining the context of conflict management and highlighting the advancements since June 2023, this presentation aims to spark discussion on future prospects to capitalize on the team's expertise.

When: Jan 25, 2024 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Complexity factor on the CALOR-QA corpus
Elie Antoine

Abstract: Dans cette présentation je présente le corpus CALOR-QA, fruit d'une colaboration entre le LIS et ORANGE. Celui-ci est un corpus de questions-réponse annoté finement en cadre sémantique avec des informations précises sur quelle relation porte la question et quel élément de celle-ci est la réponse. Dans cette étude nous utilisons ce corpus pour mettre en avant des "facteurs de complexités" dans la tache de question-réponse. Nous les analysons sur plusieurs modèles pour mieux comprendre qu'est ce qui peut influer les performances de ces modèles et si tous ceux-ci réagissent de la même façon.

When: Jan 11, 2024 at 13:00 | Where: Zoom and Luminy | Language: French (presentation), English (slides) | Slides

Étude du transfert d'information dans les conversations naturelles
Hossam Boudraa

Abstract: Les théories de l'interaction suggèrent le développement d'un alignement structurel entre les locuteurs, cela nécessite qu'ils fassent certaines hypothèses sur la capacité de s'entendre incluant le choix du sujet, de la langue, de la variété linguistique. Ces hypothèses constituent ce qu'on peut appeler commodément "connaissance partagée" ou "common ground". Dans ce projet,nous proposons d'utiliser d'une part des métriques issues de la théorie de l'information afin d' identifier les pics d'informations échangées entre les participants et de mieux comprendre les dynamiques des échanges d'informations, d'autre part faire des résumés automatiques sur des conversations libres afin de localiser et identifier informations essentielles et pertinentes. Ces approches permettront ainsi d'obtenir une mesure objective de l'instanciation de la connaissance partagée . Notre étude se concentre sur un corpus de conversations libres, auxquelles a été ajouté une segmentation prosodique et une annotation experte des épisodes thématiques.

When: Dec 07, 2023 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Le Pouvoir du Machine Learning, du NLP, et de la Réalité Virtuelle pour la Prise de Parole en Public
Elodie Etienne

Abstract: Cette recherche se concentre sur la création d'un environnement de réalité virtuelle visant à améliorer les compétences en prise de parole en public. L'objectif initial est la conception d'une audience virtuelle réaliste à l'aide de techniques de réalité virtuelle et de photoréalisme. Ensuite, l'attention s'est portée sur l'analyse de la voix et du discours, en particulier la détection des émotions dans la voix des orateurs. Une base de données émotionnelle de haute qualité a été en cours de développement, ainsi que des recherches sur des algorithmes d'apprentissage automatique et profond pour cette analyse. Enfin, des avatars animés par l'intelligence artificielle sont en cours de création pour offrir une expérience d'apprentissage immersive aux orateurs. Cette recherche vise à améliorer la formation à la prise de parole en public en explorant les synergies entre la réalité virtuelle, l'analyse émotionnelle vocale et l'intelligence artificielle.

When: Nov 02, 2023 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

In real-time risk analysis on open sources
Jeremy Auguste

Abstract: In this talk, I'll present the Nunki.co startup and how we use open sources (social medias, online press, ...) to analyze and detect risks, by relying on various NLP tasks such as event classification, hate speech and sentiment analysis, machine translation, semantic parsing, or automatic geolocation. During the talk, I'll focus on three tasks which we actively work on: event classification (crisis analysis), targeted sentiment analysis and quantity extraction. For each of these tasks, I'll present limitations of existing state-of-the-art approaches and how we try to deal with these limitations.

When: Oct 26, 2023 at 13:00 | Where: Zoom and Luminy | Language: French (presentation), English (slides)

Presentation of the ChiCA corpus.
Abdellah Fourtassi

Abstract: This week, I will report progress on developing and annotating the ChiCA corpus, with children aged 7, 9, and 11 years old. Child-caregiver dyads were recorded at home in face-to-face conversations (using portable eye-tracking) and via video calls.

When: Oct 19, 2023 at 13:00 | Where: Zoom and Luminy | Language: English

ANR COMPO & ANR HEBBIAN
Alexis Nasr

Abstract: This week, we will present and discuss several new ANR projects where members of TALEP are involved.

When: Oct 12, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

LIS@DEFT'23 Can LLMs repond to MCQs? (a) yes; (b) no; (c) I don't know.
Benoit Favre

Abstract: In this talk, I will present the participation of the TALEP team at the DEFT 2023 shared task. The competition consists in automatically answering multiple choice questions from bio-medical exams. We developped an approach relying on finetuned large language models with the LoRA low-cost finetuning method. The proposed system, based on the LLaMa models, outperformed systems from all other participants in all metrics and tasks.

When: Sep 28, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Learning and Assessment of public speaking in virtual reality. An overview of my thesis topic.
Marion Ristorcelli

Abstract: Le projet REVITALISE, financé par l'ANR, a notamment pour objectif de produire un outil en réalité virtuelle afin de s'entrainer à prendre la parole en public. Dans le cadre de ma thèse, je cherche à identifier les caractéristiques de l'audience virtuelle qui rendent difficile la prise de parole en public en faisant varier le genre, la proportion de genre et l'attitude sociale de l'audience. La difficulté de la tâche peut également s'apercevoir à travers la performance des participants. C'est pourquoi, nous prévoyons de collecter un corpus d'interaction humain-audience virtuelle afin d'identifier les indices comportementaux verbaux et non verbaux qui corrèlent avec une bonne prise de parole en public. Cela nous permettra par la suite d'adapter la simulation en réalité virtuelle aux participants en fonction de leur profil (genre et attitude sociale de l'audience).

When: Jun 29, 2023 at 13:00 | Where: Zoom and Luminy | Language: English (slides), French (presentation) | Slides

Génération automatique des comportements faciaux : des données à l'évaluation
Alice Delbosc

Abstract: One of the challenges in the field of ACAs is the automatic generation of behaviors. While a lot of research focuses on the generation of body movements, few studies consider facial behaviors (facial expressions, head movements and gaze direction). Yet these non-verbal behaviors play a crucial role in interactions. Several studies have shown that inappropriate facial expressions create a strange feeling towards the agent. Head movements, on the other hand, increase speech intelligibility and improve the way the virtual agent is perceived in general. In this presentation, we propose a new model for the automatic generation of facial expressions, head movements and gaze, built from generative adversarial networks on an audio-visual corpus. In particular, we will present the scientific issues involved in developing such models: data pre-processing, model post-processing and objective and subjective evaluation metrics.

When: Jun 22, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Classifying feedback communicative functions.
Carol Figueroa

Abstract: In human-human conversations, short feedback utterances such as 'uh-huh', 'yeah', and 'wow' serve different communicative functions. For example, 'yeah' can indicate a response to a question, express agreement to an opinion, convey surprise, or simply signal that the interlocutor should continue speaking, depending on the prosodic realization, as well as the conversational context. In this talk, I will present the performance of different models and features for classifying the communicative function of short feedback utterances in dialog.

When: May 25, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Towards a better understanding of vision-language transformer models.
Emmanuelle Salin

Abstract: I will first introduce vision-language transformers and the different possible choices in pre-training and architecture. In particular, I will focus on their pre-training datasets and explain related issues. Finally, I will discuss the difficulties encountered when evaluating those models, and how different evaluation methods can help better understand those models.

When: May 11, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Exploring Social Sciences Archives with Explainable Document Linkage through Question Generation
Elie Antoine

Abstract: In this talk, we introduce the question answering paradigm as a way to explore digitized archive collections for Social Science studies. Question generation can be used as a way to create explainable links between documents. Question generation for document linking is validated qualitatively and quantitatively on a new corpus of digitized archive collection of a French Social Science journal. We also quantitatively evaluate the links produced by our method to those produced by more classical similarity methods.

When: Apr 20, 2023 at 13:00 | Where: Zoom and Luminy | Language: English

Automatic detection of children's communicative signals in video calls.
Dhia Elhak Goumri

Abstract: With the increasing use of video chats by children, the need for tools that facilitate the scientific study of their communicative behavior becomes more pressing. We investigate the automatic detection – from video calls – of two major signals in children's social coordination, smiles and gaze. While there has been significant advancement in the field of computer vision to model such signals, very little work has been done to put these techniques to the test in the noisy, variable context of video calls, and even fewer studies (if any) have investigated children's video calls specifically. We provide a first exploration into this question, testing and comparing two modeling approaches. a) a feature-based approach that relies on state-of-the-art software like OpenFace for feature extraction, and b) an end-to-end approach where models are directly optimized to classify the behavior of interest from raw data. We found that using features generated by OpenFace provides a better solution in the case of smiles, whereas using simple end-to-end architectures proved to be much more helpful in the case of looking behavior. A broader goal of this preliminary work is to provide the basis for a public, comprehensive toolkit for the automatic processing of children's communicative signals from video chat, facilitating research in children's online multimodal interaction.

When: Apr 13, 2023 at 13:00 | Where: Zoom and Luminy | Language: English

A formal linguistic approach to hate speech detection.
Susana Campillo

Abstract: Detecting verbal violence in online discussions and social media are well-known challenges. In fact, in the last few years there has been a lot of research on automatic detection of verbal violence by using different computational tools to automatically classifying and detecting online abuse. However, false positives and false negatives are still found. The goal of my PhD project is to provide a formalization of the relevant linguistic information for automatic detection of verbal violence in Spanish tweets. We have compiled and annotated a corpus of 3,000 tweets. Every tweet is annotated as violent or non-violent, and as explicit, implicit or both. The corpus has been manually analyzed in terms of linguistic resources employed for explicitly and implicitly communicating verbal violence. The goal of this talk is to present our corpus and some of the results obtained so far in order to discuss potential interactions between linguistic and computational approaches to the problem of verbal violence detection.

When: Apr 06, 2023 at 13:00 | Where: Zoom and Luminy | Language: English

Improving children's math performance with a virtual role model against Stereotype Threat effects.
Marjorie Armando

Abstract: The main goal of this thesis is to improve the learning in mathematics for 10-13 years old students. We propose a pedagogical virtual agent to improve the performance of students in mathematics by optimizing their cognitive capacity through the reduction of the phenomenon known as “stereotype threat”. Several studies have shown that stereotype threat leads to underperformance among students when they are the target of negative stereotypes about their skills, this is the case for girls in mathematics. I will present some results on how children perceive our virtual agents.

When: Mar 30, 2023 at 13:00 | Where: Zoom and Luminy | Language: English

Analyse orientée corpus d'universaux de Greenberg sur Universal Dependencies
Hee-Soo Choi

Abstract: L’étude des universaux repose fortement sur une tradition empirique et typologique mais également sur des connaissances tirées d’ouvrages de référence. À travers nos expériences, nous fournissons des résultats fondés uniquement sur de grandes quantités de données avec un échantillon de 141 corpus, soit 74 langues d’Universal Dependencies 2.7. Avec l’outil GREW (Guillaume, 2021), nous avons ainsi déterminé trois ordres de mots (l’ordre sujet - verbe - objet, l’ordre adposition - nom et l’ordre adjectif - nom) et vérifié quatre universaux. En faisant le choix de traiter chaque corpus individuellement, nous avons pu évaluer l’homogénéité entre corpus d’une même langue et analyser les raisons des possibles divergences. Enfin, notre étude sur 74 langues permet également de soulever des incohérences interlinguistiques liées au schéma d’annotations.

When: Mar 23, 2023 at 13:00 | Where: Zoom and Luminy | Language: French

Understanding Children's Multimodal Conversational Development: Challenges and Opportunities.
Abdellah Fourtassi

Abstract: Children’s ability to master skills for coordinated conversations is crucial for their healthy social and cognitive development. However, existing work has only partly addressed this phenomenon, and until recently, scientists have lacked the appropriate research tools to account for the entire complexity of this phenomenon. The main goal of this paper is to pinpoint the reasons that have made the study of conversation development a formidable challenge, drawing on insights from conversation analysis, pragmatics, psycholinguistics, and child development. In particular, our theories should account for (a) how children learn to coordinate across multiple levels (e.g., managing turn-taking, negotiating shared understanding, and engaging in coherent exchange with the interlocutor), (b) how they learn to use multimodal signaling in face-to-face interactions, and (c) how they adapt to the communicative conventions of their linguistic community while using their skills flexibly across various conversational contexts. I outline one promising research approach that has the potential to better handle this elusive phenomenon; an approach that capitalizes on new technological opportunities in remote data collection methods and high-scalability Machine Learning tools to help provide a quantitative understanding of how children grow to become competent communicators.

When: Mar 02, 2023 at 13:00 | Where: Zoom and Luminy | Language: English

Premières évaluations et nouveaux use-cases, quelles limitations et quelles opportunités pour les Large Language Models en contexte opérationnel ?
Géraldine Damnati

Abstract: Cette intervention sera précédée d'une introduction rapide de Benoit Favre sur les modèles et méthodes de TAL sur lesquelles repose ChatGPT. À la suite du seminaire, nous proposons une discussion sur chatGPT et son impact sur nos étudiant.e.s pour réaliser projets et évaluations.

When: Feb 09, 2023 at 13:00 | Where: Luminy (Hexagone)+zoom | Language: French

Variable-rate hierarchical representation learning
Santiago Cuervo

Abstract: The success of deep learning comes from its ability to capture the hierarchical structure of data by learning high-level representations defined in terms of low-level ones. We explore self-supervised learning of hierarchical representations of speech by applying self-supervised learning at multiple levels. Inspired by the fact that speech is often described at a high-level as a sequence of discrete units unevenly distributed in time (eg. phones, syllables, words), we propose a model in which the output of an acoustic representation learner is non-uniformly downsampled to directly minimize the loss of a high-level model that enforces a prior of discreteness. Accounting for the structure of the speech signal improves upon single-level models and enhances the disentanglement of the learned representations, as measured by downstream speech recognition tasks, while resulting in a meaningful segmentation of the signal that closely resembles phone boundaries.

When: Feb 02, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

LIS and Orange at the Dialog Systems Technology Challenge (DSTC11)
Léo Jacqmin

Abstract: A majority of recent research on task-oriented dialogue systems has focused on written dialogue corpora such as MultiWOZ. With dialogue agents becoming more and more prominent in our daily lives, there has been a renewed interest in spoken dialogue systems. However, current systems trained on written corpora face robustness issues associated with spoken dialogue, such as ASR errors and disfluencies. One track in this year's Dialog Systems Technology Challenge (DSTC11) proposed to address this problem in the context of the dialogue state tracking task, releasing a spoken version of MultiWOZ. In this presentation, I will give an overview of our participation in this shared task, covering what worked, what did not, and what insights can be drawn from our results.

When: Jan 12, 2023 at 13:00 | Where: Zoom and Luminy | Language: English or French | Slides

Compositionality and logic in language
Maria Boritchev

Abstract: Compositionality is a vastly discussed subject across natural language semantics, logic, but also natural language processing and nowadays, neural networks. In this talk, I will talk about compositionality from a (computational) linguistics perspective, and then I will present an ongoing work on compositionality in neural networks. In this work, we investigate the capacity of neural networks (NNs) to learn compositional structures by focusing on a well-defined simple logical corpus, and on proof-centred compositionality. We conduct our investigation in a minimal setting by creating a simple logical corpus, where all compositionality-related phenomena come from the structure of proofs as all the sentences of the corpus are propositional logic implications. By training NNs on this corpus we test two aspects of compositionality, through variations of proof lengths and permutations of the constants.

When: Jan 05, 2023 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Artificial neural networks as models of human language learning
Alex Warstadt

Abstract: The success of artificial neural networks in NLP in recent years calls for a re-evaluation of our understanding of the possibilities for learning grammar from data alone. I argue the case for using artificial neural networks to test hypotheses about human language acquisition. Compared to human subjects, artificial learners based on neural networks are far easier to study in terms of ethics, expense, and expanded possibilities for experimental design. I will present progress towards this goal from multiple directions, including resources for evaluating the grammatical knowledge of neural models and new model learners trained in more human-like environments. I conclude with experiments that showcase new possibilities for tackling previously difficult-to-test hypotheses about the effects of the learner's environment on grammatical generalization.

When: Dec 01, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Exploring accessible modalities for accessible conversational interactions
Laurianne Sitbon

Abstract: The World Wide Web of 2022 is a platform for sharing, socialising and synthesizing vast amounts of information. However, for the approximately 3% of the population with intellectual disability, access remains limited. Specific accessible designs have attempted to support digesting new or complex information, as the key barrier to engagement with online information. Yet people with intellectual disability often do not fit a neatly labelled diagnostic category, often having a combination of underlying cognitive, communicative, motor and sensory conditions. This requires reframing how we approach technology design to focus on people's strengths rather than attempting to address ill-defined deficits. In this talk, Laurianne will present what she and her team learnt through 5 years of fieldwork, co-designing interactive information access systems with adults with intellectual disability. She will demonstrate with examples how iterative approaches that centre on people's competencies, and recognise support networks as part of key competencies, can help reframe our research positioning ensure future designs are both inclusive and respectful of individuals of all abilities.
Project website

When: Nov 24, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Why do large language models align with human brains: insights, opportunities, and challenges
Mariya Toneva

Abstract: Pretrained language models that have been trained to predict the next word over billions of text documents have been shown to also significantly predict brain recordings of people comprehending language. Understanding the reasons behind the observed similarities between language in machines and language in the brain can lead to more insight into both systems. In this talk, we will discuss a series of recent works that make progress towards this question along different dimensions. The unifying principle among these works that allows us to make scientific claims about why one black box (language model) aligns with another black box (the human brain) is our ability to make specific perturbations in the language model and observe their effect on the alignment with the brain. Building on this approach, these works reveal that the observed alignment is due to more than next-word prediction and word-level semantics and is partially related to joint processing of select linguistic information in both systems. Furthermore, we find that the brain alignment can be improved by training a language model to summarize narratives. Taken together, these works make progress towards determining the sufficient and necessary conditions under which language in machines aligns with language in the brain.

When: Nov 17, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Modeling speech acquisition using self-supervised machine learning, a focus on the acoustic-to-articulatory mapping
Thomas Hueber

Abstract: This presentation covers recent contributions in the study of speech representations coming from models trained with self-supervision in relation to articulatory characteristics of the speech signal.

When: Nov 10, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Emerging linguistic universals in communicating neural network agents
Rahma Chaabouni

Abstract: The ability to acquire and produce a language is a key component of intelligence. If communication is widespread among animals, human language is unique in its productivity and complexity. In this talk, I focus on works that build up on the emergent communication field to investigate the well-standing question of the source of natural language. In particular, these works use communicating deep neural networks that can develop a language to solve a collaborative task. Comparing the emergent language properties with human cross-linguistic regularities can provide answers to the crucial questions about the origin and evolution of natural language. Indeed, if neural networks develop a cross-linguistic regularity spontaneously, then the latter would not depend on specific biological constraints. Looking at neural networks as another expressive species can shed light on the source of cross-linguistic regularities - a fundamental research interest in cognitive science and linguistics. I will focus on four cross-linguistic regularities related to word length, word order, semantic categorization, and compositionality. Across the different studies, we find that some of these regularities arise spontaneously while others are missing in neural networks' languages. We connect the former case to the presence of shared communicative constraints such as the discrete nature of the communication channel. On the latter, we relate the absence of human-like regularities to the lack of constraints either on the learners' side (e.g., the least-effort constraints) or language functionality (e.g., the transmission of information).

When: Nov 03, 2022 at 13:00 | Where: Zoom and Luminy | Language: English

Structure in language acquisition models
Lukas Galke

Abstract: Structure appears in various aspects in the context of language learning systems: First, language exhibits an internal compositional structure, whose emergence and influence on artificial learning systems is yet poorly understood. Then, there is external structure in the form of contextual side information beyond the raw content, which is a valuable resource to be tapped. Finally, we have an induced structure that originates from the models with which we process language. In this talk, I present results to unveil the role of structure in language learning systems. We will start by examining induced structure in text classification systems, for which models can be categorised into bag-of-words, sequence, and graph-based models. We will see that a bag-of-words multilayer perceptron is surprisingly strong and outperforms recent graph-based approaches, while pre-trained Transformer-based sequence models form the state of the art. Thereafter, we will look into external structure at the example of lifelong node classification in evolving graph data with text-attributed nodes. We will learn that reusing implicit knowledge stored in the model parameters allows for smaller history sizes in a sliding-window-based training procedure. In this setting, we will further introduce techniques for the automatic detection of previously unseen classes. Lastly, we will explore the effect of compositional structure on language learnability. In contrast with recent results from emergent communication, we will see that the compositional structure of natural language is indeed beneficial for memorisation and generalisation in language learning systems, even if not modelled explicitly.

When: Oct 20, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Quantum Machine Learning, Cybersecurity, Gender Equality and Gender Bias
Partha Pakray

Abstract:

When: Oct 13, 2022 at 13:00 | Where: Zoom and Luminy | Language: English

Self-supervised representation learning of primate vocalisations
Jules Cauzinille

Abstract: This will be a review of my last project at LISN for which I worked on an expressive annotation framework and on the development of arousal recognition models for monomodal broadcast political speech. I will then give an overview of my PhD project on the development and probing of self-supervised acoustic representation learning (and synthesis) models trained on primate vocalizations.

When: Oct 06, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

End-to-end model for named entity recognition from speech without paired training data
Salima Mdhaffar

Abstract: Recent works showed that end-to-end neural approaches tend to become very popular for spoken language understanding (SLU). Through the term end-to-end, one considers the use of a single model optimized to extract semantic information directly from the speech signal. A major issue for such models is the lack of paired audio and textual data with semantic annotation. In this work, we propose an approach to build an end-to-end neural model to extract semantic information in a scenario in which zero paired audio data is available. Our approach is based on the use of an external model trained to generate a sequence of vectorial representations from text. These representations mimic the hidden representations that could be generated inside an end-to-end automatic speech recognition (ASR) model by processing a speech signal. An SLU neural module is then trained using these representations as input and the annotated text as output. Last, the SLU module replaces the top layers of the ASR model to achieve the construction of the end-to-end model. Our experiments on named entity recognition, carried out on the QUAERO corpus, show that this approach is very promising, getting better results than a comparable cascade approach or than the use of synthetic voices.

When: Sep 29, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Testing the Developmental Plausibility of BERT by Capturing the Role of Verb-Event Structure in Early Word Sense Disambiguation
Francesco Cabiddu

Abstract: BERT is a state-of-the-art language model that forms contextual representations while being able to abstract linguistic structures at multiple levels. These aspects are consistent with usage-based learners as posited by exemplar accounts of language learning, making the use of BERT a unique opportunity to test the plausibility of exemplar theories through an architecture that possesses a degree of neurobiological plausibility, processes naturalistic input and can perform a multitude of tasks. Although different studies have shown that BERT can capture aspects of adult language knowledge, it is still unclear whether BERT could also represent a plausible model of language development. The aim of this project will be to start examining this question by leveraging recently collected data which show that 4-year-olds can use verb-event structures (i.e., semantic restrictions that verbs impose on their arguments) alongside low-level word associations to resolve lexical ambiguities. I will outline a plan to test BERT’s ability to capture such data, using a recent framework for the evaluation of sense embeddings in language models used in combination with sense-annotated corpora.

When: Sep 22, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Brain basis of turn-taking in natural conversation.
Dhia Elhak Goumri

Abstract: People in a conversation normally take turns in speaking, that is, one person listens while the other person speaks. This is a fundamental aspect in dialogue in order to have a successful interaction. We address the brain basis of turn-taking. More concretely, we study the brain activity at different frequency bands, looking for the preparation effect of turn-taking. Contrary to other studies, we focus on a corpus of free conversations. We observe for the first time a reduction of the alpha band before the turn-taking in a natural conversation. We also precise the temporal location of this reduction and its amplitude. We show that this alpha reduction has a robust effect in regions of the brain responsible for language production and planning.

When: Sep 08, 2022 at 13:00 | Where: Zoom and Luminy | Language: English

To the limits of distributional semantics and beyond
Denis Paperno

Abstract: Joint seminar with SELEXINI kick-off workshop

When: Jul 07, 2022 at 14:30 | Where: Zoom and Luminy | Language: English | Slides

Abstraction ou hallucination ? État des lieux et évaluation du risque pour les modèles de génération de résumés automatiques de type séquence-à-séquence
Eunice Akani

Abstract: La génération de texte a récemment connu un très fort intérêt au vu des avancées notables dans le domaine des modèles de langage neuronaux. Malgré ces avancées, cette tâche reste difficile quand il s’agit d’un résumé automatique de texte par abstraction. Certains systèmes de résumés génèrent des textes qui ne sont pas forcément fidèles au document source. C’est sur cette thématique que porte notre étude. Nous présentons une typologie d’erreurs pour les résumés automatique et ainsi qu’une caractérisation du phénomène de l’abstraction pour les résumés de référence afin de mieux comprendre l’ampleur de ces différents phénomènes sur les entités nommées. Nous proposons également une mesure d’évaluation du risque d’erreur lorsqu’un système tente de faire des abstractions sur les entités nommées d’un document.

When: Jun 16, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Étiquetage ou génération de séquences pour la compréhension automatique du langage en contexte d'interaction?
Rim Abrougui

Abstract: La tâche de compréhension automatique du langage en contexte d’interaction (NLU pour Natural Language Understanding) est souvent réduite à la détection d’intentions et de concepts sur des corpus mono-domaines annotés avec une seule intention par énoncé. Afin de dépasser ce paradigme, nous cherchons à aborder des référentiels plus complexes en visant des représentations sémantiques structurées au-delà du simple modèle intention/concept. Nous nous intéressons au corpus MultiWOZ, couramment utilisé pour le suivi de l’état du dialogue. Nous questionnons la projection de ces annotations sémantiques complexes pour le NLU, en comparant plusieurs approches d’étiquetage de séquence, puis en proposant un nouveau formalisme inspiré des méthodes de génération de graphe pour la modélisation sémantique AMR. Nous discutons enfin le potentiel des approches génératives.

When: Jun 02, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Tâches auxiliaires pour l’analyse vers graphes de dépendances
Marie Candito

Abstract: The biaffine parser of Dozat and Manning (2017) was successfully extended to semantic dependency parsing (SDP) (Dozat and Manning, 2018). Its performance on graphs is surprisingly high given that, without the constraint of producing a tree, all arcs for a given sentence are predicted independently from each other (modulo a shared representation of tokens). To circumvent such an independence of decision, while retaining the O(n2) complexity and highly parallelizable architecture, we propose to use simple auxiliary tasks that introduce some form of interdependence between arcs. Experiments on the three English acyclic datasets of SemEval 2015 task 18 (Oepen et al., 2015), and on French deep syntactic cyclic graphs (Ribeyre et al., 2014) show modest but systematic performance gains on a near state-of-the-art baseline using transformer-based contextualized representations. This provides a simple and robust method to boost SDP performance.

When: May 19, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Représentation multimodale de conversations pour la détection de messages abusif
Richard Dufour

Abstract: Cette présentation étudie les différentes représentations pour la détection de messages abusifs dans un contexte d'interaction. Les expériences comparent des méthodes fondées sur le texte et des méthodes fondées sur le graphe d'interactions.

When: May 12, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Séminaire du pôle SD: A quick tour: Neural Network Interpretability
Hanwei Zhang

Abstract: Présentation au séminaire "Interprétabilité /Explicabilité des modèles d'apprentissage" du pôle science des données du LIS : https://www.lis-lab.fr/pole-sciences-des-donnees/

When: May 09, 2022 at 10:30 | Where: Zoom and St-Charles | Slides

Séminaire du pôle SD: The Many Flavours of CAM
Felipe Torres Figueroa

Abstract: Présentation au séminaire "Interprétabilité /Explicabilité des modèles d'apprentissage" du pôle science des données du LIS : https://www.lis-lab.fr/pole-sciences-des-donnees/

When: May 09, 2022 at 11:00 | Where: Zoom and St-Charles | Slides

Séminaire du pôle SD: Interpretable RNNs
Hamed Benazha

Abstract: Présentation au séminaire "Interprétabilité /Explicabilité des modèles d'apprentissage" du pôle science des données du LIS : https://www.lis-lab.fr/pole-sciences-des-donnees/

When: May 09, 2022 at 11:30 | Where: Zoom and St-Charles | Slides

Présentation et brainstorming autour du robot Furhat
Magalie Ochs

Abstract:

When: May 05, 2022 at 13:00 | Where: Zoom and Luminy | Language: French

Expressions multi-mots et acquisition du langage
Leonardo Pinto-Arata

Abstract: Cette présentation présente les résultats d'une expérience sur l'apprentissage implicite de séquences de mots. Le travail a analysé la longueur, la fréquence et l'espacement entre les répétitions, et leur influence sur l'apprentissage.

When: Apr 28, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Automatic analysis of errors in automatic speech recognition systems from end-users reception
Thibault Bañeras Roux

Abstract:

When: Apr 21, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Assessing the ability of neural language models to abstract syntactic representation: an analysis based on French long-distance agreement
Bingzhi Li

Abstract: Many recent works have demonstrated that unsupervised sentence representations of neural networks encode syntactic information by observing that neural language models are able to predict the agreement between a verb and its subject. We take a critical look at this line of research by showing that it is possible to achieve high accuracy on this agreement task with simple surface heuristics, indicating a possible flaw in our assessment of neural networks’ syntactic ability. Our fine-grained analyses of results on the long-range French object-verb agreement show that contrary to LSTMs, Transformers are able to capture a non-trivial amount of grammatical structure.

When: Apr 14, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Interprétabilité A Priori et Explicabilité A Posteriori dans le Traitement Automatique des Langues
Tom Bourgeade

Abstract: Avec l'avènement des architectures Transformer en TAL il y a quelques années, nous avons observé des progrès sans précédents dans diverses tâches de classification ou de génération de textes. Cependant, l'explosion du nombre de paramètres et de la complexité de ces modèles "boîte noire" de l'état de l'art, rendent de plus en plus évident le besoin désormais urgent de transparence dans les approches d'apprentissage automatique. La capacité d'expliquer, d'interpréter et de comprendre les décisions algorithmiques deviendra primordiale à mesure que les modèles informatiques deviennent de plus en plus présents dans notre vie quotidienne. Dans ce travail, nous explorons plus spécifiquement deux aspects majeurs de l'AI explicable, dans le contexte des tâches et des modèles de TAL : dans la première partie, nous abordons le sujet de l'interprétabilité intrinsèque, qui englobe toutes les méthodes qui sont naturellement faciles à expliquer. En particulier, nous nous concentrons sur les représentations de plongement de mots, qui sont une composante essentielle de pratiquement toutes les architectures de TAL. Dans la deuxième partie, nous explorons les méthodes d'explicabilité post-hoc, qui peuvent cibler des modèles déjà entraînés, et tenter d'extraire diverses formes d'explications de leurs décisions.

When: Apr 07, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

De CALOR-QUEST à CALOR-DIAL
Frédéric Béchet

Abstract: Cette présentaton survole différents travaux effectués en collaboration avec Orange Labs sur les questions de la compréhension de documents et la génération automatique de questions dans des configurations divereses ayant un impact important sur la performance des modèles.

When: Mar 31, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Apprendre à renoncer: apprentissage de retour arrière dans un système d'analyse glouton
Alexis Nasr

Abstract: Les modèles de TAL gloutons sont généralement victimes de leur gourmandise et peuvent se retrouver dans des impasses ou dans des situations incohérentes. Dans cette présentation, nous présenterons une manière originale de résoudre ce problème en autorisant le modèle à effectuer des retours arrières. Pour apprendre à déterminer les moments où un retour arrière est pertinent, nous utilisons l'apprentissage par renforcement, qui permet au modèle, lors de l'apprentissage de tenter de manière aléatoire des retours arrière afin d'évaluer leur chance de succès.

When: Feb 10, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

A Multimodal Corpus for the Study of Child Conversation
Abdellah Fourtassi

Abstract: The study of how children develop their conversational skills is an important scientific frontier at the crossroad of social, cognitive, and linguistic development with important applications in health, education, and child-oriented AI. While recent advances in machine learning techniques allow us to develop formal theories of conversational development in real-life contexts, progress has been slowed down by the lack of corpora that both approximate naturalistic interaction and provide clear access to children’s non-verbal behavior in face-to-face conversations. This work is an effort to fill this gap. We introduce ChiCo (for Child Conversation), a corpus we built using an online video chat system. Using a weakly structured task (a word-guessing game), we recorded 20 conversations involving either children in middle childhood (i.e., 6 to 12 years old) interacting with their caregivers (condition of interest) or the same caregivers interacting with other adults (a control condition), resulting in 40 individual recordings. Our annotation of these videos has shown that the frequency of children’s use of gaze, gesture, and facial expressions mirrors that of adults. Future modeling research can capitalize on this rich behavioral data to study how both verbal and non-verbal cues contribute to the development of conversational coordination

When: Feb 03, 2022 at 13:00 | Where: Zoom | Language: French | Slides

Speech @ BigScience - Analyse syntaxique de la parole
Benoit Favre, Franck Dary

Abstract: Progrès du groupe de travail sur la parole dans le projet BigScience

When: Jan 27, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Projet ANR SELEXINI : Semantic Lexicon Induction for Interpretability and Diversity in Text Processing
Carlos Ramisch

Abstract: Nouveau projet ANR pour le développement et l'évaluation de méthodes d'induction de lexiques sémantiques hybrides à partir d'embeddings contextuels, wiktionary et de grandes bases de textes non annotées : https://selexini.lis-lab.fr

When: Jan 20, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Projet ANR REVITALISE : viRtual bEhaVioral skIlls TrAining for pubLIc SpEaking
Magalie Ochs

Abstract: Nouveau projet ANR pour le développement d'une plate-forme de formation virtuelle des compétences comportementales pour la prise de parole en public

When: Jan 20, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Suivi de l'état du dialogue : passé, présent et futur
Léo Jacqmin

Abstract: Dans le contexte d'un système de dialogue, le suivi de l'état du dialogue (dialogue state tracking) consiste à extraire à chaque tour de parole une représentation des besoins de l'utilisateur. Il est une composante clé des systèmes de dialogue destinés à accomplir une tâche (task-oriented dialogue systems) : le module de gestion du dialogue (dialogue policy) utilise cette représentation afin de sélectionner l'action suivante à accomplir (par ex. informer, demander une précision, ...). Lors de cette présentation, je donnerai une vue d'ensemble sur le suivi de l'état du dialogue, des approches historiques aux méthodes actuelles, puis j'exposerai les problèmes ouverts et comment je compte les aborder dans le cadre de ma thèse.

When: Jan 13, 2022 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Summarizing scientific papers given user-desired queries in zero-shot context
Amir Soleimani

Abstract: We study the zero-shot setting for the aspect-based scientific document summarization task. Summarizing scientific documents with respect to an aspect can remarkably improve document assistance systems and readers experience. However, existing large-scale datasets contain a limited variety of aspects, causing summarization models to over-fit to a small set of aspects. We establish baseline results in zero-shot performance (over unseen aspects and the presence of domain shift), paraphrasing, leave-one-out, and limited supervised samples experimental setups. We propose a self-supervised pre-training approach to enhance the zero-shot performance. Experimental results on the FacetSum and PubMed aspect-based datasets show promising performance when the model is pre-trained using unlabeled in-domain data.

When: Jan 06, 2022 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Social media data in public health research, two cases of study
Raquel Urena

Abstract: The world wide success of large scale social information systems with diverse purposes, such as e-commerce platforms, facilities sharing communities and social networks, make them a very promising paradigm for large scale information sharing and management. With this regard more than 50% of the world wide population constitutes active users of social media platforms. From the point of view of health social media is a good source of information about users’ opinions, aptitudes and behaviors towards health issues. However the anonymity, distributed and open nature of these frameworks, that, on the one hand, foster the communication capabilities of their users, may contribute, on the other hand, to the propagation of low quality information and to an information overload. In this talk we are going to focus in two current projects: (i) Artificial intelligence for Drug users, whose goal is to use AI methodologies to develop a recommender system that promotes the sharing of knowledge and personalized information between DU communities on the largest French speaking PWUD online community; and (ii) the analysis of the HCQ controverse in social networks in the first COVID-19 wave in France.

When: Dec 09, 2021 at 13:00 | Where: Zoom and Luminy | Language: TBA | Slides

Hate speech target identification and characterization
Anaïs Ollagnier

Abstract: In an international context of increasing hate, racism and xenophobia in Europe and the U.S., social media have become a privileged tool for hate dissemination, propaganda and victimization. The need for computational methods that automatically detect such hateful content online has lately attracted a lot of interest in the Natural Language Processing community. Whilst hate speech detection has been mainly considered so far as a binary classification problem, recent studies have highlighted the importance to reach a fine-grained online hate speech characterization to provide appropriate solutions to curb online abusive behaviors. In this context, this talk presents my efforts on identifying and characterizing hate speech targets on Twitter. I propose to address this task adopting a clustering approach to enable and capture targeting characteristics in hateful contents (i.e., types of hate, such as race or religion). In addition, I will present the methodology used to investigate hate speech properties related to specific targets unveiled using the proposed detection approach. Briefly, I will also cover my previous contributions related to text mining that I performed on various purposes using different techniques including, data modeling and visualization, classification and recommendation.

When: Dec 02, 2021 at 13:00 | Where: Zoom and Luminy | Language: English

CoCoDev project
Abdellah Fourtassi

When: Nov 18, 2021 at 13:00 | Where: Zoom and Luminy | Language: English

Giving Out or Happy Out? Processing Multiword Expressions in Irish
Abigail Walsh

Abstract: Like looking for a needle in a haystack, it can be challenging for computers to process, translate and handle idiomatic expressions. Multiword Expressions (MWEs) like these include a variety of linguistic constructions such as idioms, light verbs, compound nouns, and more. MWEs are known to pose problems for many NLP tasks, and these problems can be exacerbated for low-resource languages such as Irish, due to a scarcity of both data and relevant research. This presentation explores the topic of improving the automatic processing of Irish MWEs, by developing lexical resources for Irish MWEs, and tackling the task of automatic identification.

When: Nov 04, 2021 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Zero-shot and Few-shot documents classification in biomedical domain.
Simon Lupart

Abstract: MeSH (Medical Subject Headings) is a large thesaurus created by the National Library of Medicine used for fine-grained indexing and search of publications in the biomedical domain. In the context of the pandemic, numerous MeSH descriptors from this thesaurus have emerged and so the number of related articles. To face the number of new descriptors and articles, the problem needs to be considered as a zero/few-shot classification problem. In this work we start from the hypothesis that rich semantic information available in MeSH has potential to improve BioBERT representations and make them more suitable for zero/few-shot tasks. We propose different architectures to address this problem. We analyse the results through real few-shot /zero-shot tasks. We also perform so-called "probing tasks" where we want to investigate to what extent the learnt representations improve hierarchical relations present in MeSH.

When: Oct 21, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Probing joint vision-and-language representations
Badreddine Farah

When: Oct 14, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Analyse morpho-syntaxique massivement multilingue à l’aide de ressources typologiques, d’annotations universelles et de plongements de mots multilingues
Manon Scholivet

Abstract: L’annotation de données est un problème majeur dans toutes les tâches d’apprentissage automatique. Dans le domaine du Traitement Automatique des Langues (TAL), ce problème est multiplié par le nombre de langues existantes. De nombreuses langues se retrouvent sans annotations, et sont alors mises à l’écart des systèmes de TAL. Une solution possible pour intégrer ces langues dans les systèmes est de tenter d’exploiter les langues disposant de nombreuses annotations, d’apprendre des informations sur ces langues bien dotées, et de transférer ce savoir vers les langues peu dotées. Pour cela, il est possible de se reposer sur des initiatives comme les Universal Dependencies, qui proposent un schéma d’annotation universel entre les langues. L’utilisation de plongements de mots multilingues et de traits typologiques issus de ressources comme le World Atlas of Language Structures (WALS) sont des solutions permettant un partage de connaissances entre les langues. Ces pistes sont étudiées dans le cadre de cette thèse, à travers la prédiction de l’analyse syntaxique, de la morphologie et des parties du discours sur 41 langues au total. Nous montrons que l’impact du WALS peut être positif dans un cadre multilingue, mais que son utilité n’est pas systématique dans une configuration d’apprentissage zero-shot. D’autres représentations des langues peuvent être apprises sur les données, et donnent de meilleurs résultats que le WALS, mais ont l’inconvénient de ne pas fonctionner dans un cadre de zero-shot. Nous mettons également en évidence l’importance de la présence d’une langue proche lors de l’apprentissage des modèles, ainsi que les problèmes liés à l’utilisation d’un modèle de caractère pour les langues isolées.

When: Oct 05, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Models and Resources for Attention-based Unsupervised Word Segmentation
Marcely Zanon Boito

Abstract: Documenting languages helps to prevent the extinction of endangered dialects - many of which are otherwise expected to disappear by the end of the century. When documenting oral languages, for which no written form is available, Unsupervised Word Segmentation from speech is a useful, yet challenging, task. It consists in producing time-stamps for slicing utterances into smaller segments corresponding to words. In this seminar, I will present our speech processing pipeline, which produces word segmentation in a documentation setting. This setting corresponds to leveraging minimal amounts of data: the unsupervised word segmentation task is tackled using only 4 hours of speech data. To cope with the lack of data, we use an attention-based approach that takes advantage of aligned translations in order to ground the discovered word segments.

When: Sep 30, 2021 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Learning and Processing Language from Wearables: Opportunities and Challenges (dry run of ACL keynote)
Alejandrina Cristia

Abstract: Recent years have seen tremendous improvement in the ease with which we can collect naturalistic language samples via devices worn over long periods of time. These allow unprecedented access to ego-centered experiences in language perceived and produced, including by young children. For example, in a newly-formed consortium, we pulled together over 40k hours of audio, collected from 1, 001 children growing up in industrialized or hunter-horticulturalist populations, located in one of 12 countries. Such data are interesting for many purposes, including as 1. fodder for unsupervised language learning models aimed at mimicking what the child does; 2. indices of early language development that can be used to assess the impact of behavioral and pharmacological interventions; and 3. samples of the natural use of language(s) in low-resource and multilingual settings. The technology allowing to carve out interesting information from these large datasets, however, is lagging behind – but this may not be such a bad thing after all, since the ethical, technical, and legal handling of such data also need some work to increase the chances that the net impact of research based on this technique is positive. In this talk, I draw from cutting-edge research building on long-form recordings from wearables and a framework for doing the most good we can (effective altruism) to highlight surprising findings in early language acquisition, and delineate key priorities for future work.

When: Jul 22, 2021 at 13:00 | Where: Zoom and Luminy | Language: English

Why are GPUs faster than CPUs for the matrix calculations of deep learning libraries?
Laércio Pilla

Abstract: This talk presents a quick answer and a longer explanation for the question in its title. The longer explanation goes into details related to the architectural differences between CPUs and GPUs, the three laws that guide parallel performance, and some final points related to matrix calculations.

When: Jul 15, 2021 at 13:00 | Where: Zoom and Luminy | Language: French or English | Slides

A Fuzzy Sociolinguistic Model for Gender Prediction in Spanish Social Network Texts
Damián Morales

Abstract: In a context marked by the exponential growth of social platforms, Computational Sociolinguistics aims to reveal and define trends and linguistic patterns that are correlated with social variables such as age (Nguyen et al. 2013), gender (Burger et al. 2011), or origin (Eisenstein et al. 2010). In this direction, our research focused on the analysis of a dataset made up of 76,000 messages and more than 21 million words in Spanish from the social network Netlog in order to design a fuzzy model for automatic gender prediction based on sociolinguistics conclusions. This will allow us, on the one hand, to validate previous sociolinguistic approaches through computational techniques and, on the other hand, to refine the existing computational models for gender prediction. Thus, we propose a classification model structured in six linguistic levels (orthographic, lexical, morphological, syntactic, digital, and pragmatic-discursive) and made up of 633 features.

When: Jul 08, 2021 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

Génération automatique de questions et capacité de généralisation des modèles de compréhension automatique de documents
Elie Antoine

When: Jul 01, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Génération automatique de résumés critiques d’articles à but de veille médicale
Loïc Neyrat

Abstract: Au plus fort de la crise sanitaire, plus de deux milles articles sont arrivés chaque semaine sur les bureaux des professionnels de santé participant au projet Bibliovid, un projet de veille scientifique créée pour la pandémie de Covid-19. Pour chacun des articles, ces médecins et chercheurs doivent produire un résumé critique, reprenant les éléments du papier mais également commentant les méthodes et les résultats présentés. Le traitement automatique du langage naturel peut être une solution pour automatiser cette tâche. Ainsi, l’objectif de mon stage fût d’évaluer des approches de résumé par extraction de phrases utilisant des fonctions ensemblistes particulières, appelées sous-modulaires, dans la réalisation de tels résumés.

When: Jul 01, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

An empirical study of domain adaptation for named entity recognition on historical documents
Baptiste Blouin

When: Jun 24, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Multiword Expression Features for Automatic Hate Speech Detection
Nicolas Zampieri

Abstract: The task of automatically detecting hate speech in social media is gaining more and more attention. Given the enormous volume of content posted daily, human monitoring of hate speech is unfeasible. In this work, we propose new word-level features for automatic hate speech detection (HSD): multiword expressions (MWEs). MWEs are lexical units greater than a word that have idiomatic and compositional meanings. We propose to integrate MWE features in a deep neural network-based HSD framework. Our baseline HSD system relies on Universal Sentence Encoder (USE). To incorporate MWE features, we create a three-branch deep neural network: one branch for USE, one for MWE categories, and one for MWE embeddings. We conduct experiments on two hate speech tweet corpora with different MWE categories and with two types of MWE embeddings, word2vec and BERT. Our experiments demonstrate that the proposed HSD system with MWE features significantly outperforms the baseline system in terms of macro-F1.

When: Jun 17, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Apprentissage par renforcement d’un analyseur syntaxique en transitions avec retour-arrière
Maxime Petit

Abstract: Dans cet article, nous cherchons à apprendre à un analyseur à remettre en cause ses choix. Nous nous plaçons dans le cadre du transition based dependancy parsing et utilisons le jeu de transition arc-eager auquel nous ajoutons une action. Cette action consiste à autoriser le modèle à annuler sa dernière action afin de pouvoir modifier sa réponse, on l’appellera le retour-arrière. Notre modèle est un perceptron multicouche entraîné par une méthode d’apprentissage par renforcement, le deep Q-learning. À des fins expérimentales, on se place dans le même cadre que la lecture humaine. C’est à dire que notre modèle a seulement accès à l’information du mot courant et des mots précédemment lus. Nos résultats montrent que, dans ce cadre, le modèle performe mieux avec le nouveau jeu de transition. De plus, le modèle a appris à utiliser le retour-arrière et est capable de corriger certaines erreurs.

When: Jun 09, 2021 at 14:00 | Where: Zoom and Luminy | Language: French | Slides

Cross-lingual Embeddings Evaluation
Thibault Roux

Abstract: Word embeddings are vector representations of words learned from massive corpora. Used as a mathematical way of representing words in machine learning models, they can be used for Natural Language Processing (NLP) tasks such as text mining, machine translation, question answering, topic classification and automatic summarization. Word embeddings are the mainstream representations for words in NLP models which require annotated data, not available in low-resource languages. Cross-Lingual Embeddings (CLE) can address this issue by enabling cross-lingual transfer learning. In order to transfer, it is important that a word is close from its translation in the embeddings space ; and that embeddings have a good quality. We formally evaluate the intrinsic quality of monolingual embeddings before and after projection in the cross-lingual embedding. We also evaluate how translation pairs are close thanks to the Bilingual Lexicon Induction task. Finally, we observe if there is a correlation between these intrinsic scores and a POS (part-of-speech) tagging task. The embeddings used were designed and employed for massively multilingual Universal Dependencies parsing and POS tagging as partof Scholivet’s thesis.

When: Jun 09, 2021 at 14:00 | Where: Zoom and Luminy | Language: French | Slides

SLICE 2.0: Weakly supervised interpretable word embedding learning and evaluation
Adrien Pupier

Abstract: SLICE is an in house model, developed by the TALEP team at LIS laboratory, which aims to create lightweight interpretable word embeddings. However, the SLICE paper left many questions unanswered. In this paper, we greatly optimize the process of creating the embeddings by replacing the multiple binary models with a single multi-class model. Moreover, we extend the approach to use finer grained senses and observe the effect of different languages on this method. Then, we experiment with different sense granularities and how they interact to improve our results on a word sense disambiguation task. We found out that finer grain sense could help the more coarse sense. With this method, we outperform the result obtained in the SLICE paper in French with coarse granularity. Finally, we found out how many monosemic words (seeds) per sense are needed to obtain a satisfactory result and the variability of a random sample of seeds. This allows us to evaluate the effort needed to broaden this method to more senses and to say that the original number of seeds used in the SLICE paper is larger than what would be required for this task.

When: Jun 09, 2021 at 14:00 | Where: Zoom and Luminy | Language: French | Slides

Models of laughter dynamics in early child-parent interaction
Gabriel Meunier

Abstract: Ce travail vise à construire un système qui détecte le rire de l’enfant dans des enregistrements audio. L’objectif général à long terme est de faciliter l’étudie scientifique de rire comme un précurseur du développent langagier des enfants dans le milieu naturel. On utilise des outils de détection d’événement sonore qui ont pour but d’identifier le type et les bornes temporelles d’un ou plusieurs sons spécifiques. Dans notre cas, on s’intéresse à la détection de rire dans un contexte d’interaction entre une mère et son enfant. Il existe actuellement seulement des modèles visant la détection du rire chez les adultes mais ceux-ci ne se généralise pas très bien avec les enfants. Nous avons donc pour but de trouver un modèle permettant de réaliser cette tâche. Dans cette étude, nous commencerons par tester différentes méthodes de classification traditionnelles comme les SVM, pour finir sur des modèles profonds de petite à moyenne taille.

When: Jun 09, 2021 at 14:00 | Where: Zoom and Luminy | Language: French | Slides

Nouvelles avancées dans la définition linguistique des POS: vers un tagset sans adverbes
José Deulofeu

When: May 27, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Analyzing complexity factors for Spoken Language Understanding on benchmark and deployed service corpora
Rim Abrougui

Abstract: Travail réalisé pour interspeech 2021 présentant une comparaison entre le corpus Djingo d'Orange et les autres benchmarks SLU.

When: May 20, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Génération automatique de questions pour l'apprentissage de modèles de Machine Reading Comprehension
Jeremy Auguste

When: May 06, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Multi-SimLex: A Large-Scale Evaluation of Multilingual and Crosslingual Lexical Semantic Similarity
Thierry Poibeau

Abstract: We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering data sets for 12 typologically diverse languages, including major languages (e.g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e.g., Welsh, Kiswahili). Each language data set is annotated for the lexical relation of semantic similarity and contains 1,888 semantically aligned concept pairs, providing a representative coverage of word classes (nouns, verbs, adjectives, adverbs), frequency ranks, similarity intervals, lexical fields, and concreteness levels. Additionally, owing to the alignment of concepts across languages, we provide a suite of 66 crosslingual semantic similarity data sets. Because of its extensive size and language coverage, Multi-SimLex provides entirely novel opportunities for experimental evaluation and analysis. On its monolingual and crosslingual benchmarks, we evaluate and analyze a wide array of recent state-of-the-art monolingual and crosslingual representation models, including static and contextualized word embeddings (such as fastText, monolingual and multilingual BERT, XLM), externally informed lexical representations, as well as fully unsupervised and (weakly) supervised crosslingual word embeddings. We also present a step-by-step data set creation protocol for creating consistent, Multi-Simlex–style resources for additional languages. We make these contributions—the public release of Multi-SimLex data sets, their creation protocol, strong baseline results, and in-depth analyses which can be helpful in guiding future developments in multilingual lexical semantics and representation learning—available via a Web site that will encourage community effort in further expansion of Multi-Simlex to many more languages. Such a large-scale semantic resource could inspire significant further advances in NLP across languages. Joint work with Ivan Vulić, Simon Baker, Edoardo Maria Ponti, Ulla Petti, Ira Leviant, Kelly Wing, Olga Majewska, Eden Bar, Matt Malone, Thierry Poibeau, Roi Reichart, Anna Korhonen.

When: Apr 29, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

NLP models as vaccines for language problems - Significant lessons from experimental sciences
Carlos Ramisch

Abstract: La présentation survole un ensemble de techniques pour calculer la significativité des différences obtenues par deux systèmes sur un échantillon de test. Le but est de partager des expériences sur la méthodologie expérimentale pour réfléchir ensemble à comment revoir nos pratiques voire adopter systématiquement ces techniques lors de nos analyses de résultats expérimentaux.

When: Apr 22, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Globalizing BERT-based Transformer Architectures for Long Document Summarization
Quentin Grail

Abstract: Fine-tuning a large language model on downstream tasks has become a commonly adopted process in the Natural Language Processing (NLP). However, such a process, when associated with the current transformer-based architectures, shows several limitations when the target task requires to reason with long documents. In this work, we introduce a novel hierarchical propagation layer that spreads information between multiple transformer windows. We adopt a hierarchical approach where the input is divided in multiple blocks independently processed by the scaled dot-attentions and combined between the successive layers. We validate the effectiveness of our approach on three extractive summarization corpora of long scientific papers and news articles. We compare our approach to standard and pre-trained language-model-based summarizers and report state-of-the-art results for long document summarization and comparable results for smaller document summarization.

When: Apr 15, 2021 at 13:00 | Where: Zoom and Luminy | Language: French

TALEP at CMCL 2021 Shared Task: Non Linear Combination of Low and High-Level Features for Predicting Eye-Tracking Data
Franck Dary

Abstract: In this paper we describe our contribution to the CMCL 2021 Shared Task, which consists in predicting 5 different eye tracking variables from English tokenized text. Our approach is based on a neural network that combines both raw textual features we extracted from the text and parser-based features that include linguistic predictions (e.g. part of speech) and complexity metrics (e.g., entropy of parsing). We found that both the features we considered as well as the architecture of the neural model that combined these features played a role in the overall performance. Our system achieved relatively high accuracy on the test data of the challenge and was ranked 2nd out of 13 competing teams and a total of 30 submissions.

When: Apr 08, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Evaluating the Acquisition of Semantic Knowledge from Cross-situational Learning in Artificial Neural Networks
Mitja Nikolaus

Abstract: When learning their native language, children acquire the meanings of words and sentences from highly ambiguous input without much explicit supervision. One possible learning mechanism is cross-situational learning, which has been successfully tested in laboratory experiments with children. Here we use Artificial Neural Networks to test if this mechanism scales up to more natural language and visual scenes using a large dataset of crowd-sourced images with corresponding descriptions. We evaluate learning using a series of tasks inspired by methods commonly used in laboratory studies of language acquisition. We show that the model acquires rich semantic knowledge both at the word- and sentence-level, mirroring the patterns and trajectory of learning in early childhood. Our work highlights the usefulness of low-level co-occurrence statistics across modalities in facilitating the early acquisition of higher-level semantic knowledge.

When: Apr 01, 2021 at 13:00 | Where: Zoom and Luminy | Language: English | Slides

FrSemCor: Annotating a French corpus with supersenses
Lucie Barque

Abstract: French, as many languages, lacks semantically annotated corpus data. Our aim is to provide the linguistic and NLP research communities with a gold standard sense-annotated corpus of French, using WordNet Unique Beginners as semantic tags, thus allowing for interoperability. In this paper, we report on the first phase of the project, which focused on the annotation of common nouns. The resulting dataset consists of more than 12,000 French noun tokens which were annotated in double blind and adjudicated according to a carefully redefined set of supersenses. The resource is released online under a Creative Commons Licence. Joint work with P. Haas, R. Huyghe, D. Tribout, M. Candito, B. Crabbé and V. Segonne.

When: Mar 25, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Ressources de calcul distribué au LIS
Franck Dary

Abstract: Présentation pratique des ressources de calcul sur Jean Zay et au mésocentre AMU.

When: Mar 18, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Dimension émotionnelle des textes et compréhension
Delphine Battistelli

Abstract: Le terme de "compréhension de textes" se fait à nouveau jour en TAL depuis quelques années après avoir été longtemps délaissé - ou du moins contourné - au profit d'autres termes tel que celui d'extraction d'information en particulier. Il évoque la question des dimensions sémantiques nécessaires à l'interprétation d'un texte et, parmi les plus communément investies, on trouvera celles de la temporalité, de l'espace et de la cause. Dans certains modèles psycholinguistiques de la compréhension, ces dimensions sont également abordées, avec un intérêt grandissant porté en outre sur une autre dimension : la dimension émotionnelle. Son activation plus ou moins importante dans un texte en favoriserait la compréhension globale. Je présenterai ici la perspective adoptée sur la dimension émotionnelle des textes dans le cadre de travaux menés au sein du projet ANR TexToKids (2019-2023) consacrés à la compréhension de textes par des enfants jeunes lecteurs ; et plus précisément au développement d'outils de prédiction de recommandations d’âge. Y a t-il des modes d'expression et des types d'émotions plus facilement accessibles aux enfants selon certaines classes d'âge ? Comment ces informations s'articulent-elles avec celles provenant d'autres dimensions sémantiques telles que la cause et la temporalité notamment ? Le type de texte (journalistique, fictionnel, encyclopédique) joue-t-il un rôle ? Ce type de questionnements occupe une place centrale dans le schéma d'annotation des textes proposé et la méthodologie proposée.

When: Mar 11, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Breaking news in NLP: bigger is better! but is it really?
Léo Bouscarrat

Abstract: Presentation/discussion of the paper 'On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?' by Bender et al. (2021) and related topics.

When: Feb 25, 2021 at 13:15 | Where: Zoom and Luminy | Language: French | Slides

Vision and Language Pre-trained Models
Emmanuelle Salin

Abstract: A state of the art in vision and language pre-trained models.

When: Feb 18, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

Travail de master
Rim Abrougui

When: Feb 04, 2021 at 13:00 | Where: Zoom and Luminy | Language: French

Distributed Learning for speech recognition in the context of privacy protection
Eunice Akani

Abstract: In view of the frequent use of automatic speech recognition systems in many devices and the fact that the system is trained on a large amount of users data which may contain private information; this internship aims to bridge the gap between the improvement of ASR system and the protection of sensitive users information. Thus, many acoustics models were trained on users data and some information such as weight matrix were extracted from the acoustic models. Using a clustering method, the data of the same speaker were grouped together. Moreover, the clustering revealed a gender grouping. Thus, analysis were done on speaker identification and gender grouping. The results obtained show that at the first layers of the models, it is possible to have meta information from the speech such as gender which is not the case for the higher layers. With regards to speaker identification, the best result was obtained from the first layer. Furthermore, the results depend on the number of epochs on which the model is trained. This work gave first results and opens other lines of research.

When: Jan 28, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

The Inadequacy of the Mode in Neural Machine Translation
Wilker Aziz

Abstract: Neural sequence generation systems oftentimes generate sequences by searching for the most likely sequence under the learnt probability distribution. This assumes that the most likely sequence, i.e. the mode, under such a model must also be the best sequence it has to offer (often in a given context, e.g. conditioned on a source sentence in translation). Recent findings in neural machine translation (NMT) show that the true most likely sequence oftentimes is empty under many state-of-the-art NMT models. This follows a large list of other pathologies and biases observed in NMT and other sequence generation models: a length bias, larger beams degrading performance, exposure bias, and many more. Many of these works blame the probabilistic formulation of NMT or maximum likelihood estimation. We provide a different view on this: it is mode-seeking search, e.g. beam search, that introduces many of these pathologies and biases, and such a decision rule is not suitable for the type of distributions learnt by NMT systems. We show that NMT models spread probability mass over many translations, and that the most likely translation oftentimes is a rare event. We further show that translation distributions do capture important aspects of translation well in expectation. Therefore, we advocate for decision rules that take into account the entire probability distribution and not just its mode. We provide one example of such a decision rule, and show that this is a fruitful research direction.

When: Jan 21, 2021 at 13:00 | Where: Zoom and Luminy | Language: English

Knowledge graph embeddings
Sébastien Montella

When: Jan 14, 2021 at 13:00 | Where: Zoom and Luminy | Language: French | Slides

WebNLG Challenge 2020
Sébastien Montella

When: Oct 15, 2020 at 13:00 | Where: Zoom and Luminy | Language: French | Slides