SMS scnews item created by Catherine Meister at Fri 7 Nov 2025 1017
Type: Seminar
Distribution: World
Expiry: 28 Nov 2025
Calendar1: 13 Nov 2025 1300-1400
CalLoc1: SMRI Seminar Room (A12 Room 301)
CalTitle1: SMRI Seminar, Geometry of Prediction in Large Language Models
Auth: cmeister@staff-10-48-18-173.vpnuser.sydney.edu.au (cmei0631) in SMS-SAML

SMRI Seminar: Murfet

Geometry of Prediction in Large Language Models

Daniel Murfet, University of Melbourne

SMRI Seminar, 13th November 2025

1 pm – 2 pm, SMRI Seminar Room (A12 Macleay Room 301)

Abstract: Our current best models of human language are neural networks with billions of parameters, trained by stochastic gradient descent. This training process somehow manages to take patterns from the training data and embed them into the parameters of these networks, but why and how this works remains quite mysterious. Part of the mathematical picture involves the (algebraic) geometry of the loss function that drives this gradient descent, according to deep work in Bayesian statistics by Sumio Watanabe. I’ll explain some of these connections between neural networks and geometry, and how at Timaeus (an AI-safety non-profit) we are applying these ideas to interpretability, an exciting new field that aims to discover the underlying “algorithms” driving neural network computation.

For seminar updates, see our What's On Calendar