Detecting the Instability of Oncoming Vehicles Using Optical Flow and Map-Based Context

Open Access
Monaco, Christopher D
Graduate Program:
Mechanical Engineering
Master of Science
Document Type:
Master Thesis
Date of Defense:
November 07, 2016
Committee Members:
  • Sean N Brennan, Thesis Advisor/Co-Advisor
  • Kurt A Hacker, Committee Member
  • optical flow
  • visual odometry
  • instability
  • vehicles
  • map
  • stereo camera
  • vehicle detection
  • unscented kalman filter
  • navigation
  • advanced driver assistance systems
  • stereo image processing
  • kalman filters
  • road vehicles
  • optical flow vectors
  • perception
  • gps-denied
  • automotive safety
  • vehicle instability
  • vehicle state estimation
  • egomotion estimation
  • ego-vehicle
  • collision avoidance
  • stereo vision
This thesis seeks to determine the feasibility of detecting the instability of oncoming vehicles using optical flow from a stereo camera. A "visual odometry" method was selected to estimate the egomotion of an instrumented mapping vehicle from its onboard stereo camera and inertial measurement unit (IMU) data. This method analyzes optical flow to estimate the egovehicle's translational and rotational motion. Next, the algorithm classifies features as either static or dynamic to detect dynamic vehicles. Once detected, optical flow analysis then estimates the external vehicle's states using extracted features from the moving egovehicle. This process enables the novel concept of using optical flow to detect an oncoming vehicle's instability. The detection of vehicle instability is greatly aided by location context that can be used to warn drivers of imminent risk. To provide context, this thesis considers the use of a map to isolate instabilities that may result in entry of the egovehicle's lane. Specifically, a map provides additional knowledge of an external vehicle's road radius from position measurements and map data. This permits the estimation of a external vehicle's neutral steering yaw rate. If the external vehicle's measured yaw rate is in excess of its estimated neutral steering yaw rate, this then indicates instability and a likelihood that the vehicle will be unable to maintain consistent lane tracking. This algorithm was tested offline with data collected from The Pennsylvania State University's mapping vehicle and a precisely controlled test vehicle at The Pennsylvania State University Larson Transportation Institute Test Track. While this concept's validation was limited - and not the direct goal of this thesis - the results show promise for this concept's feasibility and future validation. This work is intended to be implemented as part of a more robust and complex real-time architecture. When deployed as an Advanced Driver Assistance System (ADAS) on a moving vehicle, it can ensure early detection of an oncoming vehicle's instability, prompting increased caution to mitigate collision risks. Furthermore, when deployed statically at an intersection, it can detect low friction-areas based on common areas of vehicle instability, prompting remedial action to fix the roadway. In both situations, this novel concept has the potential to reduce roadway fatalities and serious injuries.