Enabling Intelligent Vision Systems in a Configurable Multi-algorithm Pipeline

Open Access
Cotter, Matthew Joseph
Graduate Program:
Computer Science and Engineering
Doctor of Philosophy
Document Type:
Date of Defense:
March 04, 2015
Committee Members:
  • Vijaykrishnan Narayanan, Dissertation Advisor
  • Mary Jane Irwin, Committee Member
  • John Morgan Sampson, Committee Member
  • Mary Beth Rosson, Committee Member
  • John Phillip Sustersic Jr., Special Member
  • Dr Steven P Levitan, Special Member
  • Configurable Systems
  • Vision Algorithms
  • Intelligent Vision
  • Hardware Accelerators
The machine vision community has expended tremendous effort in the research and development of algorithms in an effort to develop a system that is capable of seeing the world as humans do. These algorithms often focus on the accomplishment of specific tasks analogous to human vision such as scene awareness, object detection, object recognition, and object tracking. Joining forces with cognitive neuroscientists has steered much of this research towards the development of algorithms that not only accomplish the required tasks, but endeavor to do so in a biologically inspired fashion. Still, development and evaluation of these so-called neuromorphic algorithms is often done in isolation, with little regard given to the rest of the system necessary to make this human-like system a reality. This dissertation provides a framework for the current and future development of complex and highly integrated multi-algorithm vision systems. This framework not only enables the composition of such systems, but enables seamless development and integration of improved algorithmic modules. In addition to this high-level system composition framework, the Cerebrum tool, targeted at development of hardware-accelerated architectures is detailed in this work. This tool enables the creation of such hardware-based accelerators by researchers and engineers without specific or detailed knowledge of the target hardware platform. In addition to the framework and tools, this dissertation also details the analysis, development and evaluation of hardware accelerators for HMAX object recognition and AIM saliency detection. Armed with this intelligent framework and algorithmic accelerators, demonstrations of vision systems that leverage multiple algorithms are constructed and evaluated. Hierarchical object classification, leveraging the benefits of Exemplar SVM and accelerated HMAX is shown to provide performance superior to either algorithm in isolation. Furthermore, a more complex system, targeting the domain of personal retail assistance is composed and demonstrated for the benefit of visually impaired persons. With an eye towards future systems, this dissertation also serves to evaluate and explore a number of technologies whose time is coming. New transistors, such as Tunnel FETs, and novel architectures, such as coupled oscillator arrays, are examined to identify benefits and concerns in their use for the development of future visual systems, both at the algorithmic and circuit/component level. This work also explores the potential for inclusion of additional data modalities, such as audio, for a more effective understanding of scene awareness. The flexibility of the framework described here enables the inclusion of these emerging devices, architectures, and modalities alongside traditional software and hardware-accelerated implementations within a unified system in order to develop, evaluate, and deploy all of the components required for any given visual system.