Non-Parametric Finite Multivariate Mixture Models with Applications

Open Access
Graduate Program:
Doctor of Philosophy
Document Type:
Date of Defense:
September 26, 2014
Committee Members:
  • David Russell Hunter, Dissertation Advisor
  • David Russell Hunter, Committee Chair
  • Bruce G Lindsay, Committee Member
  • Le Bao, Committee Member
  • John C Liechty, Committee Member
  • Stephanie Trea Lanza, Committee Member
  • Runze Li, Committee Member
  • mixture model
  • conditional independence
  • nonparametric estimation
  • penalized smoothed likelihood
  • independent component analysis
This research set out to investigate and build upon the foundation for the nonparametric estimation of finite multivariate mixture models given the conditional independence assumption, set forth in a series of studies over the last decade. We proposed a novel formulation of the objective function in terms of penalized smoothed Kullback-Leibler divergence under a reduced parameter space. A special optimization landscape and scheme was discovered in working out the majorizationminimization method for the estimation problem which leads to a closed form of the nonlinearly smoothed majorization-minimization (NSMM) algorithm. We established a sharpened monotonicity property that precisely measures the distance between successive iterates of the algorithm and proved the existence of a solution to the main optimization problem for the first time in literature. The estimation theory for this basic model together with the special optimization scheme can be adapted to the investigation of an important extension of the model that incorporates component-wise independent component analysis (ICA). The NSMMICA algorithm has been developed and a discretized version of it, which interweaves NSMM and weighted FastICA has been implemented in the R package icamix as a model-based clustering tool. We demonstrated the use of the newly developed methods/algorithms by applications in image analysis and unsupervised learning.