Below are some selected projects where I made core contributions. For an exhaustive list, see my [Google Scholar].

• [Reducing the Scope of Language Models]
  David Yunis, Siyu Huo, Chulaka Gunasekara, Danish Contractor
  We conduct a broad empirical study of different methods for scoping the generations of language models to specific domains. We find that a combination of supervised fine-tuning and a method called circuit breakers are often best. Preprint 2025. [link]

  
• [Approaching Deep Learning through the Spectral Dynamics of Weights]
  David Yunis, Kumar Kshitij Patel, Samuel Wheeler, Pedro Savarese, Gal Vardi, Karen Livescu, Michael Maire, Matthew Walter
  We show that there is a consistent low-rank bias across many models and tasks during deep learning optimization, and that this bias connects a host of different poorly understood phenomena, including grokking, lottery tickets and linear mode connectivity. Preprint 2024. [link]

  
• [Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning]
  David Yunis, Justin Jung, Falcon Dai, Matthew Walter
  We create a new action space for reinforcement learning using subword tokenization methods (like BPE) borrowed form natural language processing. This leads to huge speedups and efficiency gains in sparse-reward tasks (100x). Published at NeurIPS 2024. [link]

  
• [Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations]
  Xiao Zhang^*, David Yunis^*, Michael Maire
  We design a method for probing activations of pretrained vision models using spectral clustering, where the graph is defined by attention matrices. We show that query-key graphs encode spatial information, while value graphs encode semantic information, and extract competitive segmentations without labels. Published at CVPR 2024. [link]

  
• [On Convexity and Linear Mode Connectivity in Neural Networks]
  David Yunis, Kumar Kshitij Patel, Pedro Savarese, Gal Vardi, Jonathan Frankle, Matthew Walter, Karen Livescu, Michael Maire
  We show that the linear mode connectivity phenomenon is not restricted to isolated linear paths, but rather there is an entire high-dimensional convex subspace of optima. We also show that optimization is not convex anywhere along the trajectory, and that the optima in these subspaces are quite distinct functions. Presented at OPT 2022. [link]