Past events

While language models (LMs) show impressive text manipulation capabilities, they also lack commonsense and reasoning abilities and are known to be brittle. In this talk, I will suggest a different LMs design paradigm, inspired by how humans understand it. I will present two papers, both shedding light on human-inspired NLP architectures aimed at delving deeper into the meaning beyond words.  The first paper [1] accounts for the lack of commonsense and reasoning abilities by proposing a paradigm shift in language understanding, drawing inspiration from embodied cognitive linguistics (ECL). In this position paper we propose a new architecture that treats language as inherently executable, grounded in embodied interaction, and driven by metaphoric reasoning.  The second paper [2] shows that LMs are brittle and far from human performance in their concept-understanding and abstraction capabilities. We argue this is due to their token-based objectives, and implement a concept-aware post-processing manipulation, showing it matches human intuition better. We then pave the way for more concept-aware training paradigms.   [1] Language (Re)modelling: Towards Embodied Language Understanding Ronen Tamari, Chen Shani, Tom Hope, Miriam R L Petruck, Omri Abend, and Dafna Shahaf. 2020. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 6268–6281, Online. Association for Computational Linguistics.  [2] Towards Concept-Aware Large Language Models Shani, Chen, Jilles Vreeken, and Dafna Shahaf. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 13158-13170. 2023.   Bio: Chen Shani is a post-doctoral researcher at Stanford's NLP group, collaborating with Prof. Dan Jurafsky. Previously, she pursued her Ph.D. at the Hebrew University under the guidance of Prof. Dafna Shahaf and worked at Amazon Research. Her focus lies at the intersection of humans and NLP, where she implements insights from human cognition to improve NLP systems.

1.  Can we reconstruct images that a person saw, directly from his/her fMRI brain recordings?  2.  Can we reconstruct the training data that a deep-network trained on, directly from the parameters of the network?   The answer to both of these intriguing questions is “Yes!”  In this talk I will show how these can be done. I will then show how exploring the two domains in tandem can potentially lead to significant breakthroughs in both fields. More specifically: (i)  I will show how combining the power of Brains & Machines can potentially be used to bridge the gap between those two domains. (ii) Combining the power of Multiple Brains (scanned on different fMRI scanners with NO shared stimuli) can lead to new breakthroughs and discoveries in Brain-Science. We refer to this as “the Wisdom of a Crowd of Brains”. In particular, we show that a Universal Encoder can be trained on multiple brains with no shared data,  and that information can be functionally mapped between different brains.

The capacity of animals to respond to stimuli in their surroundings is crucial for their survival. In mammals, complex evaluations of the environment require large numbers and different subtypes of neurons. The nematode C. elegans utilize its compact nervous system to process environmental cues and tune behavior. Integration of opposing spatial information and adaptation to distinct types of addictive substances are only a few challenges that require efficient and effective use of the worm’s compact nervous system. We describe how distinct environmental cues can converge onto common neural networks and molecular mechanisms but generate diverse neuronal and behavioral responses. Using a multidisciplinary approach, we completed several parallel aims, including the development of two novel research methods

While language models (LMs) show impressive text manipulation capabilities, they also lack commonsense and reasoning abilities and are known to be brittle. In this talk, I will suggest a different LMs design paradigm, inspired by how humans understand it. I will present two papers, both shedding light on human-inspired NLP architectures aimed at delving deeper into the meaning beyond words.  The first paper [1] accounts for the lack of commonsense and reasoning abilities by proposing a paradigm shift in language understanding, drawing inspiration from embodied cognitive linguistics (ECL). In this position paper we propose a new architecture that treats language as inherently executable, grounded in embodied interaction, and driven by metaphoric reasoning.  The second paper [2] shows that LMs are brittle and far from human performance in their concept-understanding and abstraction capabilities. We argue this is due to their token-based objectives, and implement a concept-aware post-processing manipulation, showing it matches human intuition better. We then pave the way for more concept-aware training paradigms.   [1] Language (Re)modelling: Towards Embodied Language Understanding Ronen Tamari, Chen Shani, Tom Hope, Miriam R L Petruck, Omri Abend, and Dafna Shahaf. 2020. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 6268–6281, Online. Association for Computational Linguistics.  [2] Towards Concept-Aware Large Language Models Shani, Chen, Jilles Vreeken, and Dafna Shahaf. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 13158-13170. 2023.   Bio: Chen Shani is a post-doctoral researcher at Stanford's NLP group, collaborating with Prof. Dan Jurafsky. Previously, she pursued her Ph.D. at the Hebrew University under the guidance of Prof. Dafna Shahaf and worked at Amazon Research. Her focus lies at the intersection of humans and NLP, where she implements insights from human cognition to improve NLP systems.

1.  Can we reconstruct images that a person saw, directly from his/her fMRI brain recordings?  2.  Can we reconstruct the training data that a deep-network trained on, directly from the parameters of the network?   The answer to both of these intriguing questions is “Yes!”  In this talk I will show how these can be done. I will then show how exploring the two domains in tandem can potentially lead to significant breakthroughs in both fields. More specifically: (i)  I will show how combining the power of Brains & Machines can potentially be used to bridge the gap between those two domains. (ii) Combining the power of Multiple Brains (scanned on different fMRI scanners with NO shared stimuli) can lead to new breakthroughs and discoveries in Brain-Science. We refer to this as “the Wisdom of a Crowd of Brains”. In particular, we show that a Universal Encoder can be trained on multiple brains with no shared data,  and that information can be functionally mapped between different brains.

The capacity of animals to respond to stimuli in their surroundings is crucial for their survival. In mammals, complex evaluations of the environment require large numbers and different subtypes of neurons. The nematode C. elegans utilize its compact nervous system to process environmental cues and tune behavior. Integration of opposing spatial information and adaptation to distinct types of addictive substances are only a few challenges that require efficient and effective use of the worm’s compact nervous system. We describe how distinct environmental cues can converge onto common neural networks and molecular mechanisms but generate diverse neuronal and behavioral responses. Using a multidisciplinary approach, we completed several parallel aims, including the development of two novel research methods