COMPARISON OF DIFFERENT DENSITY ESTIMATORS FOR INFINITE DIMENSIONAL EXPONENTIAL FAMILIES
Open Access
Author:
Rao, Aniruddha Rajendra
Graduate Program:
Statistics
Degree:
Master of Science
Document Type:
Master Thesis
Date of Defense:
July 02, 2019
Committee Members:
Dr. Bharath Sriperumbudur, Thesis Advisor/Co-Advisor Dr. Ephraim Hanks, Committee Member Benjamin Shaby, Committee Member
Keywords:
Kernel Methods Density Estimation Exponential Family Computation Infinite Dimension
Abstract:
In this thesis, we consider the problem of estimating an unknown density, p_o belonging
to an infinite dimensional exponential family P parametrized by functions in a
reproducing kernel Hilbert space (RKHS) H. P is quite rich in the sense that a broad
class of densities on R^d can be approximated arbitrarily well in Kullback-Leibler (KL)
divergence by elements in it. The main focus of the thesis is to propose and compare
the performance of various estimators of p_o. General methods like maximum likelihood
estimation (MLE) or pseudo MLE do not result in practically useful estimators due to
their inability to efficiently handle the log-partition function. In this work, we consider
three different estimators, (i) Kernel Density Estimator (KDE), which is a classical
non-parametric density estimator, (ii) Score Matching Estimator (SME), based on
minimizing the Fisher divergence, J(p_o||p) between p_o and p in P, which involves
solving a simple finite-dimensional linear system and (iii) Approximate Matching
estimator (AME), which is a variation of SME but computationally more efficient.
We show through numerical simulations that KDE performs better in the univariate
case, while the other two methods have superior performance in high dimensional
scenarios.
