©Brain4Cars
Imagine that you are driving a car down a road in an unfamiliar area. You quickly turn right after realizing that your destination is on the right, then BOOM! You collide with a cyclist.
The past decade has seen the rise of various driver assist technologies to allow people to drive more safely and easily. Brain4Car, developed by scientists at Cornell and Stanford universities, applies Deep Learning to learn and monitor drivers’ behaviors in order to alert them before something bad is about to happen just like the above scenario. Deep Learning or Deep Machine Learning refers to artificial neural networks that are composed of multiple layers to model complex relationships between datasets. The goal of the project is to anticipate driver maneuvers in a timely manner to prevent possible dangers such as collision.
According to Ashesh Jain, the project lead and a PhD at Stanford, their primary focus was to equip cars with as many sensors as possible to monitor drivers. In this way, the predictive system can learn and therefore know in advance what you are about to do while driving.
Ashesh’s team collected video footages of 10 drivers over a period of two months. A complete dataset included 2 million video frames taken in diverse landscapes such as a natural freeway and local streets. A total of 1,180 miles were driven by the drivers and the system was developed using the videos of the driver inside the car and the road in front, the vehicle’s dynamics, global position coordinates (GPS), and street maps.
The most current version of Brain4Car is able to provide driver assist to anticipate driver’s maneuvers 3.5 seconds before they occur with 90.5% precision in real-time. The accuracy of predicting driver’s behavior increased from 80% last year to 90.5% this year by adapting recurrent neural network based sensor fusion (combination of different sets of sensory data) and incorporating the driver’s 3D head-pose (combination of driver’s position and orientation).
The system is amazing yet still limited. Currently, it can only predict five maneuvers: left turn, right turn, left lane change, right lane change, straight driving. Advanced driver assistance system (ADAS) is still an ongoing project. In a tangential study, researcher from Human-Computer Interaction Institute at Carnegie Mellon University collected data from 15 drivers, including vehicle motion, traffic states, driver’s physiological responses, to develop a system that mediates interruptions while driving. The system alerts the driver to react accordingly to changing driving conditions.
