An Analysis and Framework for Multi-label Image Classification of Insect Taxonomy with Convolutional Neural Networks
Open Access
- Author:
- Pham, Viet
- Graduate Program:
- Computer Science and Engineering
- Degree:
- Master of Science
- Document Type:
- Master Thesis
- Date of Defense:
- March 17, 2022
- Committee Members:
- Chitaranjan Das, Program Head/Chair
Vijaykrishnan Narayanan, Thesis Advisor/Co-Advisor
John Morgan Sampson, Committee Member - Keywords:
- multi-label image classification
image classification
computer vision
machine learning
insect taxonomy
convolutional neural networks - Abstract:
- Multi-label image classification is the task of assigning multiple labels to an image. As of current, there are only a few bodies of work in this specific task and its application to classifying insects and their taxonomy, and therefore, this is the task that this body of work explored and contributed to. Furthermore, for many use cases, the multiple labels of an object are not unrelated, but they have a dependent relation with each other, such as the hierarchy of insect taxonomy. Since others that have worked on this task have not explored in-depth the possible relations among the labels of an object and how a model might make use of them, this body of work also sought to provide insight into this inquiry. The approach was to first build a framework with a pipeline that will facilitate this research process, of which involves data collection and preparation, model training, and model evaluation. Hence, this work was able to develop a framework that contains code to support all three steps, in which images of desired insects can be automatically downloaded, structured, and correctly labeled for model training and evaluation with the desired model and test image set. This framework was then used for evaluating multiple models, including a single-label image classifier as a baseline, with an f-beta metric, and with comparisons on classifications of image frames from video footage that were manually labeled by expertise. Conclusively, this work found that the multi-label image classifiers were able to achieve f-beta scores greater than 95%, and that for some of the footage, the model was able to obtain the correct results with a percentage of 24.4% in the order level and of 23.1% in the family level. Lastly, in addition to the order percentage accuracy score on the footage being higher than the family percentage score, the model was able to classify most insect families that were in the same order that it has trained on but in a different family that it has not seen correctly. With these results, there is a lot of potential to use the developed framework and generated insights to run more experiments with better models and larger datasets, obtaining stronger answers for the research questions proposed.