Machine Learning Seminar

We describe a new class of probabilistic models, squared families, where densities are defined by squaring a linear transformation of a statistic and normalising with respect to a base measure. Key quantities, such as the normalising constant and certain statistical divergences, admit a helpful parameter-integral decomposition giving a closed form normalising constant in many cases of interest. Parametrising the statistic using neural networks results in highly expressive yet tractable models, with universal approximation properties. This approach naturally extends to other probabilistic settings, such as modelling point processes. We illustrate the effectiveness of squared neural probabilistic models on a variety of tasks, demonstrating their ability to represent complex distributions while maintaining analytical and computational advantages. Joint work with Russell Tsuchida, Jiawei Liu, and Cheng Soon Ong.

Speaker Bio: Dino Sejdinovic is a Professor of Statistical Machine Learning at the University of Adelaide (since 2022), where he is affiliated with the Australian Institute for Machine Learning (AIML) and the Responsible AI Research Centre (RAIR). He also holds visiting appointments with the Nanyang Technological University, Singapore and the Institute of Statistical Mathematics, Tokyo. He was previously an Associate Professor at the Department of Statistics, University of Oxford and a Turing Faculty Fellow of the Alan Turing Institute. He held postdoctoral positions at the University College London and the University of Bristol and received a PhD in Electrical and Electronic Engineering from the University of Bristol (2009). His research spans a wide variety of topics at the interface between machine learning and statistical methodology, including large-scale nonparametric and kernel methods, robust and trustworthy machine learning, causal inference, and uncertainty quantification.

In today's data-driven landscape, harnessing vast and varied datasets provides unparalleled opportunities for knowledge extraction and informed decision-making. Yet, within this abundance, a pivotal concern emerges: the quality and origin of the data, a challenge pervasive across diverse fields such as health sciences, epidemiology, economics, and beyond. The presence of noisy data or measurement error can obscure patterns, introduce biases, and undermine the reliability of analyses. Such an issue has attracted extensive attention in both statistical and machine learning communities. In this talk, I will briefly discuss the complexities arising from dealing with noisy data and their potential to impede statistical inference or machine learning procedures. Through this exploration, we aim to shed light on the importance of addressing data quality issues and developing strategies to mitigate their adverse effects on decision-making processes.

Transformer-based architectures have emerged as cornerstones of deep learning, powering breakthroughs in natural language processing via innovations like parameter-efficient fine-tuning and scalable model designs. Yet despite their remarkable empirical performance, a unified theoretical framework remains elusive. In this talk, I will introduce a novel distribution-regression perspective for transformers, offering rigorous analysis and fresh insights into their underlying mechanisms.

Speaker Bio: Peilin Liu is a second-year PhD candidate at the School of Mathematics and Statistics and the Brain and Mind Centre, University of Sydney. His research spans the theoretical foundations of deep learning theory, large language models, and medical AI.

The evolution of AI has progressed rapidly from traditional machine learning models for prediction and classification to large language models (LLMs) for generative AI and now to autonomous, agentic AI systems. Despite these advancements, ensuring AI remains human-centered and trustworthy is a critical societal need. In this talk, Dr. Kush R. Varshney will explore the specifications required to ensure AI systems demonstrate reliability, human interaction, purpose alignment, and respect for human agency and dignity. He will discuss the current gaps in theoretical foundations for LLM-based agentic AI and propose how a systems theory approach could enhance prediction, control, and optimization in AI.

Speaker Bio: Dr. Kush R. Varshney is an IBM Fellow and leads Human-Centered Trustworthy AI Research at the IBM Thomas J. Watson Research Center, NY. His contributions to AI include developing well-known open-source tools such as AI Fairness 360, AI Explainability 360, Uncertainty Quantification 360, and AI FactSheets 360. He has been recognized with multiple IBM Corporate Technical Awards and is a Fellow of IEEE. His book Trustworthy Machine Learning (2022) explores the theoretical underpinnings of fair and explainable AI.