לוח שנה משולב

In this talk we introduce the problem of random walks on the Cayley graph of a finite group, some techniques for its study, and some of the basic results, including numerical experiments. This is a mixture of basic group theory, representation theory, probability theory, and graph theory.

I’ll discuss various ways to use small cancellation methods to produce groups with desired properties. In particular, I’ll demonstrate how to construct groups whose semigroup Zariski topology is strictly coarser than their group Zariski topology (answering a question by Elliott, Jonusas, Mesyan, Mitchell, Morayne, and Peresse)

The dominant paradigm in generative modeling consists of two steps: i) pre-training on a large-scale but unsafe dataset, ii) aligning the pre-trained model with human values via fine-tuning. This practice is considered safe, as no current method can recover the unsafe, pre-fine-tuning model weights. In this paper, we demonstrate that this assumption is often false. Concretely, we present Spectral DeTuning, a method that can recover the weights of the pre-fine-tuning model using a few low-rank (LoRA) fine-tuned models. In contrast to previous attacks that attempt to recover pre-fine-tuning capabilities, our method aims to recover the exact pre-fine-tuning weights. Our approach exploits this new vulnerability against large-scale models such as a personalized Stable Diffusion and an aligned Mistral.

Bio:

Eliahu Horwitz is a PhD candidate in Computer Science at the Hebrew University of Jerusalem, working under the supervision of Prof. Yedid Hoshen. His research area is computer vision, with a focus on representation learning and generative models. Currently, his work revolves around reversing the training trajectories of neural networks.

A recipient of the KLA Scholarship for Outstanding Graduate Students and a CIDR (Center for Interdisciplinary Data Science Research) fellow, Eliahu’s academic achievements are complemented by his practical experience. Before transitioning to research, he honed his skills as a self-taught software developer, working with diverse technologies across the tech stack at both startups and large-scale companies. His latest research can be found on his website: pages.cs.huji.ac.il/eliahu-horwitz.

In this talk we will give an introduction to property testing questions in group theory. Property testing problems were mentioned in Alex’s talk, and come up naturally in various branches of theoretical computer science, as well as mathematics and physics. For example, the question of cocycle expansion, “are almost cocycles close to actual cocycles”, is a typical property testing problem. Another example is the following: Given two permutations that commute with high probability on randomly sampled entries, are they close to actual commuting permutations? For groups, here are key notions: Given a group G, it is said to be permutation stable if approximate actions of G on finite sets by permutations are close to actual finite actions of G. G is said to be Hilbert Schmidt stable if the same can be said about approximate finite dimensional representations of G. We will introduce these properties, give a lot of examples and mention connections with the study of characters and invariant random subgroups, as well as the questions of soficity and Connes embeddability of groups.

In this last talk of the seminar, we will pack together all the topics discussed over the semester. In particular, we will suggest a path toward finding a non sofic group. 

While cognitive neuroscientists have uncovered principles of perceptual decision-making by analyzing choices and neuronal firing across thousands of trials, we do not yet know the behavioral or neuronal dynamics underlying one SINGLE choice. For instance, why might a subject judge a given stimulus in category A 70% of the time but in category B 30%? Until we can work out precisely what determines single-decisions – this choice, right now – the mechanisms of real-world decision-making will remain unknown. In tactile psychophysical tasks with rats and humans, we are trying to sort out factors that explain the variability in judgments (across trials) to the identical stimulus input. We identify four factors: (i) trial-to-trial fluctuations in sensory coding, (ii) temporal context, namely, the history of preceding stimuli and choices, (iii) attention, and (iv) bias (predictions originating in beliefs about the environment’s probabilistic structure). The strategy is to bring these factors under experimental control, rather than leaving them to vary according to uninterrogated states within the subject. Psychophysics from rats and humans show that large chunks of variability are accounted for by these factors; evidence from cortical neuronal populations in rats provides some mechanistic grounding.

While language models (LMs) show impressive text manipulation capabilities, they also lack commonsense and reasoning abilities and are known to be brittle. In this talk, I will suggest a different LMs design paradigm, inspired by how humans understand it. I will present two papers, both shedding light on human-inspired NLP architectures aimed at delving deeper into the meaning beyond words.
 The first paper [1] accounts for the lack of commonsense and reasoning abilities by proposing a paradigm shift in language understanding, drawing inspiration from embodied cognitive linguistics (ECL). In this position paper we propose a new architecture that treats language as inherently executable, grounded in embodied interaction, and driven by metaphoric reasoning.
 The second paper [2] shows that LMs are brittle and far from human performance in their concept-understanding and abstraction capabilities. We argue this is due to their token-based objectives, and implement a concept-aware post-processing manipulation, showing it matches human intuition better. We then pave the way for more concept-aware training paradigms.  

[1] Language (Re)modelling: Towards Embodied Language Understanding
Ronen Tamari, Chen Shani, Tom Hope, Miriam R L Petruck, Omri Abend, and Dafna Shahaf. 2020.
In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 6268–6281, Online. Association for Computational Linguistics. 

[2] Towards Concept-Aware Large Language Models
Shani, Chen, Jilles Vreeken, and Dafna Shahaf.
In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 13158-13170. 2023.
 
Bio: Chen Shani is a post-doctoral researcher at Stanford's NLP group, collaborating with Prof. Dan Jurafsky. Previously, she pursued her Ph.D. at the Hebrew University under the guidance of Prof. Dafna Shahaf and worked at Amazon Research. Her focus lies at the intersection of humans and NLP, where she implements insights from human cognition to improve NLP systems.