Assessing Classification Uncertainty On Astronomical Objects With Measurement Error
Open Access
- Author:
- Shy, Sarah
- Graduate Program:
- Statistics
- Degree:
- Master of Science
- Document Type:
- Master Thesis
- Date of Defense:
- December 08, 2021
- Committee Members:
- Hyungsuk Tak, Thesis Advisor/Co-Advisor
Eric D Feigelson, Committee Member
Ephraim Hanks, Professor in Charge/Director of Graduate Studies
G. Jogesh Babu, Committee Member - Keywords:
- Bayesian posterior predictive distribution
catalog data
quasar
random forest
support vector machine - Abstract:
- Most general-purpose classification methods, such as support-vector machine (SVM) and random forest (RF), fail to account for an unusual characteristic of astronomical data: known measurement error uncertainties. In astronomical data, this information is often given in the data but discarded because popular machine learning classifiers cannot incorporate it. We propose a simulation-based approach that incorporates heteroscedastic measurement error into any existing classification method to better quantify uncertainty in classification. The proposed method first simulates perturbed realizations of the data from a Bayesian posterior predictive distribution of a Gaussian measurement error model. Then, a chosen classifier is fit to each simulation. The variation across the simulations naturally reflects the uncertainty propagated from the measurement errors in both labeled and unlabeled data sets. We demonstrate the use of this approach via two numerical studies. The first is a thorough simulation study applying the proposed procedure to SVM and RF, which are well-known hard and soft classifiers, respectively. The second study is a realistic classification problem of identifying high-z (2.9 ≤ z ≤ 5.1) quasar candidates from photometric data. The data were obtained from merged catalogs of the Sloan Digital Sky Survey, the Spitzer IRAC Equatorial Survey, and the Spitzer-HETDEX Exploratory Large-Area Survey. The proposed approach reveals that out of 11,847 high-z quasar candidates identified by a random forest without incorporating measurement error, 3,146 are potential misclassifications. Additionally, out of ∼1.85 million objects not identified as high-z quasars without measurement error, 936 can be considered candidates when measurement error is taken into account.