Efficient Deep Neural Networks Architectures for Video Analytics Systems
Restricted (Penn State Only)
- Author:
- Hakimi, Zeinab
- Graduate Program:
- Computer Science and Engineering (PHD)
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- July 07, 2023
- Committee Members:
- Chitaranjan Das, Program Head/Chair
Vijaykrishnan Narayanan, Chair & Dissertation Advisor
Jack Carroll, Outside Unit & Field Member
Rui Zhang, Major Field Member
John Sampson, Major Field Member - Keywords:
- Deep Neural Networks
Object Recognition
Convolutional Neural Networks
Edge Computing
Distributed Camera Intelligence System - Abstract:
- In recent years, there has been a remarkable surge in the volume of digital data across various formats and domains. For instance, modern camera systems leverage new technologies and the fusion of information from multiple views to capture high-quality images. As a result of this data explosion, there is a growing interest and demand for analyzing information using data-intensive machine learning algorithms, particularly deep neural networks (DNNs). However, despite the success of deep learning approaches in various domains, their performance on small edge devices with constrained computing power and memory are limited. The primary objective of this thesis is to design efficient intelligent vision systems that effectively overcome the limitations of deep neural networks (DNNs) when deployed on edge devices with limited resources. This work explores a variety of methods aimed at optimizing the utilization of information and context in the design of DNN architectures. By leveraging these techniques, the proposed systems aim to enhance the performance and efficiency of DNNs in resource-constrained environments. Specifically, the thesis proposes context-aware methods to differentiate between low and high quality sensors representations by incorporating the context into the CNN models and reduce the computation and communication costs of edge devices in a distributed camera system. The primary objective is to minimize the computation and communication costs associated with edge devices in a distributed camera system. In addition, the thesis proposes a fault-tolerant mechanism to address the challenges posed by abnormal and noisy data in the system, particularly due to unknown conditions. This mechanism serves as a solution to mitigate the adverse effects of such data, ensuring the reliability and robustness of the proposed system. Furthermore, a resolution-aware multi-view design is outlined to address data transmission and power challenges in embedded devices. Moreover, the thesis introduces a patch-based attention-likelihood technique, designed to enhance the recognition performance of small objects within high-resolution images. This technique effectively reduces the computational burden of handling high-resolution images on edge devices by processing sub-samples of the input patches. By selectively attending to relevant patches, the proposed approach significantly improves the overall efficiency of object recognition while maintaining a high level of accuracy. Finally, the thesis introduces an efficient task-adaptive visual transformer model specifically designed for fine-grained classification tasks on IoT devices. By optimizing the system's performance for IoT devices, it enables efficient and reliable fine-grained classification without compromising computational resources or compromising the accuracy of results. Overall, this thesis offers a comprehensive approach to overcoming the limitations associated with deploying deep neural networks (DNNs) on edge devices within visual intelligent systems.