Sandra Fortini
Bocconi University, Italy
On the theoretical foundations of predictive Bayesian learning
There is currently a renewed interest in the Bayesian predictive approach to statistics. The talk offers a review on foundational concepts and focuses on predictive modeling, which by directly reasoning on prediction, bypasses inferential models or may characterize them. Predictive characterizations are detailed in exchangeable and partially exchangeable settings, for a large variety of data structures.
The underlying concept is that Bayesian predictive rules are probabilistic learning rules, formalizing through conditional probability how we learn on future events given the available information.
This concept has implications in any statistical problem and in inference, from classic contexts to less explored challenges, such as providing Bayesian uncertainty quantification to predictive algorithms in data science.
Keywords: Bayesian foundations; Predictive characterizations; Predictive sufficiency; Partial exchangeability
Joint work with Sonia Petrone.
Lorenzo Cappello
Pompeu Fabra University
Recursive methods for predictive Bayes
Predictive Bayes is driving the development of new recursive methods for use in predictive resampling. In the literature, we observe both the adaptation of existing algorithms and the creation of entirely new ones tailored to this purpose. In this tutorial, we will explore both perspectives. First, we will review the derivation of several well-established algorithms and examine how they can inspire new proposal mechanisms. Next, we will consider the reverse situation: starting with a preferred algorithm and evaluating its suitability within this emerging framework
Bernardo Flores López
University of Texas, Austin, USA
Predictive Coresets
Modern data analysis often involves massive datasets with hundreds of thousands of observations, making traditional inference algorithms computationally prohibitive. Coresets are methods designed to select a smaller subset of observations while maintaining similar learning performance. Traditional approaches involve finding sparse weights that minimize the Kullback-Leibler (KL) divergence between the likelihoods of the original and weighted datasets; however, they are ill-posed for nonparametric models, where the likelihood is often intractable. We propose an alternative variational method which leverages randomized posteriors and finds weights to match the unknown posterior predictive distributions conditioned on the full and reduced datasets. Our approach provides a general algorithm based on predictive recursions suitable for nonparametric priors.. We evaluate our method’s performance on diverse problems, including random partitions and density estimation.
Keywords: Dimensionality reduction; coresets; random partitions; species sampling model
Carlos Erwin Rodríguez
IIMAS-UNAM, Mexico
Martingale Posterior Inference for Finite Mixture Models and Clustering
Martingale posterior inference is employed to quantify uncertainty for a finite mixture model with an unknown number of components. New ideas for Bayesian analysis, particularly focusing on martingale posterior distributions, are applied to focus on clustering. The fundamental concept involves constructing appropriate martingales for the unknown parameters. A key outcome of this approach is the ability to conduct posterior analysis of clusters while circumventing the label-switching problem. This methodology is further extended to finite populations, where the mixture is utilized to impute missing part of the population and perform inference for the parameter of interest. The proposed methodology is demonstrated through the analysis of four real datasets.
Keywords: Akaike weights; EM algorithm; Finite population; Model averaging; Predictive inference.
Joint work with Stephen G. Walker and Ramses H. Mena
Hristo Sariev
Sofia University “St. Kliment Ohridski”, Bulgaria
Characterization of exchangeable Hoeffding decomposable sequences
The Hoeffding decomposition is a classical method for studying U-statistics based on sequences of i.i.d. random variables. The concept was extended by G. Peccati [Ann. Probab. 32 (2004) 1796-1829] to exchangeable processes, but apart from a few examples, it was not known which exchangeable processes were explicitly Hoeffding decomposable. In this study, we provide a complete characterization of the class of exchangeable Hoeffding decomposable processes in terms of their predictive distributions. In particular, we show that they are mixtures of exchangeable measure-valued Polya urn sequences, from which we derive the general form of their prior distributions.
Keywords: Hoeffding decomposition; Exchangeability; Polya urns; Predictive distributions
Joint work with Mladen Savov; Stefan Gerdjikov
Shengjun (Percy) Zhai
The University of Chicago, USA
Minimax Bayesian Predictive Inference with the Horseshoe Prior
This work is focused on distributional prediction of a high-dimensional Gaussian vector with a sparse mean, the accuracy of which measured by the Kullback-Leibler loss. Several priors have been considered in the current literature, including discrete priors and Laplace priors deployed inside the spike-and-slab framework. This work complements the toolbox by considering the Horseshoe prior. We start with the oracle case where the sparsity level is known, and demonstrate that the Horseshoe prior provides a predictive risk that attains the minimax rate with a properly calibrated parameter. Without the knowledge of sparsity level, we consider the full Bayes method that imposes a hierarchical prior based on the Horseshoe, which reaches a minimax rate adaptively. These hierarchical priors are continuous and fully automatic (i.e. without the need to specify hyper-parameters), and are therefore easy to implement. Since the Horseshoe is a continuous mixture of Gaussian priors, the predictive density can be written as a continuous mixture of normal densities, making the predictive inference computationally inexpensive, a property desired by the practitioners.
Keywords: Horseshoe Prior; Predictive Inference; Sparse Normal Means; Kullback-Leibler Loss; Asymptotic Minimaxity
Joint work with Veronika Rockova