July 19, 1994 - July 19, 2027

  • Date:11ThursdayJuly 2024

    Designing Language Models to Think Like Humans

    More information
    Time
    11:00 - 12:00
    Location
    Gerhard M.J. Schmidt Lecture Hall
    Lecturer
    Dr. Chen Shani
    Post-doctoral researcher NLP group Stanford University
    Organizer
    Department of Brain Sciences
    Contact
    DetailsShow full text description of Host: Prof. Misha Tsodyks misha@weizmann.ac.il For access...»
    Host: Prof. Misha Tsodyks misha@weizmann.ac.il

    For accessibility issues:naomi.moses@weizmann.ac.il
    AbstractShow full text abstract about While language models (LMs) show impressive text manipulatio...»
    While language models (LMs) show impressive text manipulation capabilities, they also lack commonsense and reasoning abilities and are known to be brittle. In this talk, I will suggest a different LMs design paradigm, inspired by how humans understand it. I will present two papers, both shedding light on human-inspired NLP architectures aimed at delving deeper into the meaning beyond words.
     The first paper [1] accounts for the lack of commonsense and reasoning abilities by proposing a paradigm shift in language understanding, drawing inspiration from embodied cognitive linguistics (ECL). In this position paper we propose a new architecture that treats language as inherently executable, grounded in embodied interaction, and driven by metaphoric reasoning.
     The second paper [2] shows that LMs are brittle and far from human performance in their concept-understanding and abstraction capabilities. We argue this is due to their token-based objectives, and implement a concept-aware post-processing manipulation, showing it matches human intuition better. We then pave the way for more concept-aware training paradigms.  

    [1] Language (Re)modelling: Towards Embodied Language Understanding
    Ronen Tamari, Chen Shani, Tom Hope, Miriam R L Petruck, Omri Abend, and Dafna Shahaf. 2020.
    In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 6268–6281, Online. Association for Computational Linguistics. 

    [2] Towards Concept-Aware Large Language Models
    Shani, Chen, Jilles Vreeken, and Dafna Shahaf.
    In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 13158-13170. 2023.
     
    Bio: Chen Shani is a post-doctoral researcher at Stanford's NLP group, collaborating with Prof. Dan Jurafsky. Previously, she pursued her Ph.D. at the Hebrew University under the guidance of Prof. Dafna Shahaf and worked at Amazon Research. Her focus lies at the intersection of humans and NLP, where she implements insights from human cognition to improve NLP systems.
    Lecture