Yoav Gelberg

Yoav Gelberg

About Me

I'm a student at the University of Oxford, working with Haggai Maron, Michael Bronstein, and Yarin Gal. I am broadly interested in language model architectures, long horizon tasks, and the structure of neural network parameter spaces. I am currently an intern at Meta, previously spent time at Sakana AI, and co-organized the Weight Space Learning Workshop at ICLR 2025. Before Oxford, I studied at the Technion as a Rothschild scholar and taught math and programming at the ARDC.

Selected Publications

* denotes equal contribution; full list on Google Scholar

KV-CAT forward pass and loss

Training Transformers for KV Cache Compressibility

Yoav Gelberg*, Yam Eitan*, Michael Bronstein, Yarin Gal, Haggai Maron

HiLD @ ICML 2026

Animated figure from DroPE: extending LLM context by dropping positional embeddings.

Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings

Yoav Gelberg, Koshi Eguchi, Takuya Akiva, Edoardo Cetin

ICLR 2026

Code, Blog

GradMetaNet overview figure: an equivariant architecture for learning on gradients.

GradMetaNet: An Equivariant Architecture for Learning on Gradients

Yoav Gelberg*, Yam Eitan*, Aviv Navon, Aviv Shamsian, Theo (Moe) Putterman, Michael Bronstein, Haggai Maron

NeurIPS 2025

Weight Space Learning Workshop @ ICLR 2025

LoL paper figure: GL-equivariant processing of low-rank weight spaces.

Leaning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models

Theo (Moe) Putterman*, Derek Lim*, Yoav Gelberg, Stephanie Jegelka, Haggai Maron

LoG 2025, Oral Presentation 🎤

Weight Space Learning Workshop @ ICLR 2025

Misalignment paper figure on vision-language representation gaps.

Misalignment Between Vision-Language Representations in Vision-Language Models

Yonatan Gideoni, Yoav Gelberg, Tim G. J. Rudner, Yarin Gal

UniReps + CogInterp Workshops @ NeurIPS 2025

LOSNet paper figure on detecting hallucinations and data contamination.

Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions

Fabrizio Frasca*, Guy Bar-Shalom*, Derek Lim, Yoav Gelberg, Yftah Ziser, Ran El-Yaniv, Gal Chechik, Haggai Maron

AAAI 2026

R2-FM Workshop @ ICML 2025

Topological Blindspots paper figure on expressivity in topological deep learning.

Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

Yam Eitan*, Yoav Gelberg*, Guy Bar-Shalom, Fabrizio Frasca, Michael Bronstein, Haggai Maron

ICLR 2025, Oral Presentation 🎤

Code

Animated figure on permutation-invariant posteriors under model symmetries.

Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks

Yoav Gelberg, Tycho van der Ouderaa, Mark van der Wilk, Yarin Gal

PMLR, GRaM Workshop @ ICML 2024, Best Paper Award 🏆