Yoav Gelberg

About Me

I'm an incoming CS PhD student at Stanford. I'm interested in architectures and learning algorithms for language models, especially for long-horizon tasks, and in the structure of neural network parameter spaces. I'm currently an intern on Yaron Lipman's team at Meta, working on multi-token prediction. Before Stanford, I studied at Oxford, where I worked with Haggai Maron, Michael Bronstein, and Yarin Gal. I also spent time at Sakana AI, co-organized the First Weight Space Learning Workshop at ICLR, studied at the Technion as a Rothschild scholar, and taught math and programming at the ARDC.

Selected Publications

* denotes equal contribution; full list on Google Scholar

KV-CAT forward pass and loss

Training Transformers for KV Cache Compressibility

Yoav Gelberg*, Yam Eitan*, Michael Bronstein, Yarin Gal, Haggai Maron

HiLD @ ICML 2026

Animated figure from DroPE: extending LLM context by dropping positional embeddings.

Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings

Yoav Gelberg, Koshi Eguchi, Takuya Akiva, Edoardo Cetin

ICLR 2026

GradMetaNet overview figure: an equivariant architecture for learning on gradients.

GradMetaNet: An Equivariant Architecture for Learning on Gradients

Yoav Gelberg*, Yam Eitan*, Aviv Navon, Aviv Shamsian, Theo (Moe) Putterman, Michael Bronstein, Haggai Maron

NeurIPS 2025

Weight Space Learning Workshop @ ICLR 2025

LoL paper figure: GL-equivariant processing of low-rank weight spaces.

Leaning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models

Theo (Moe) Putterman*, Derek Lim*, Yoav Gelberg, Stephanie Jegelka, Haggai Maron

LoG 2025, Oral Presentation 🎤

Weight Space Learning Workshop @ ICLR 2025

LOSNet paper figure on detecting hallucinations and data contamination.

Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions

Fabrizio Frasca*, Guy Bar-Shalom*, Derek Lim, Yoav Gelberg, Yftah Ziser, Ran El-Yaniv, Gal Chechik, Haggai Maron

AAAI 2026

R2-FM Workshop @ ICML 2025

Topological Blindspots paper figure on expressivity in topological deep learning.

Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

Yam Eitan*, Yoav Gelberg*, Guy Bar-Shalom, Fabrizio Frasca, Michael Bronstein, Haggai Maron

ICLR 2025, Oral Presentation 🎤

Animated figure on permutation-invariant posteriors under model symmetries.

Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks

Yoav Gelberg, Tycho van der Ouderaa, Mark van der Wilk, Yarin Gal

GRaM Workshop @ ICML 2024, Best Paper Award 🏆