Preventing truck crashes: A dashcam-based approach for detecting ‘microsleep’ behavior

Truck drivers unintentionally taking ‘microsleeps’ of a few seconds can cause terrible traffic accidents, but efforts and technologies aiming to prevent the problem have focused mainly on monitoring driver eye activity while missing a host of other key indicators of the problem.

Researchers using dashcam footage of real-world collisions involving large trucks to analyze driver and vehicle behavior has found that anti-drowsiness alarms or similar technologies to prevent falling asleep at the wheel need to go beyond a focus on monitoring the drivers’ eyes, and consider other microsleep behaviors including a relaxation of back and neck muscles and abnormal activity of the vehicle itself.

A study reporting the researchers’ findings was published in the journal Accident Analysis & Prevention.

Traffic accidents involving large trucks in the United States have increased 47 percent according to the National Safety Council, and in 2021, these vehicles were responsible for nine percent of all vehicles involved in fatal crashes. Several studies worldwide have highlighted in particular the severity of collisions related to drowsy driving and their substantial risk of fatality.

A major risk here comes from what researchers term “microsleeps.” A microsleep is a brief episode of sleep that lasts for just a few seconds and can occur involuntarily during wakefulness. It can lead to impaired cognitive and motor performance, including slower reaction times, decreased attention, and even complete loss of consciousness.

Microsleep episodes can have a significant impact on driving performance and increase the likelihood of dozing off while driving. In 2022, National Police Agency of Japan announced that falling asleep at the wheel was to be included in the same category of “forward inattention” as distraction, which has been the most frequent cause of fatal traffic accidents in the country for over ten years.

A great deal of research into typical microsleep behaviors has investigated one or more aspects of the problem such as eye closure or changes in pupil diameter. No study has yet performed an analysis that takes a more holistic approach considering multiple aspects of the problem, combining the driver’s eye and mouth movements as well as their entire body and vehicle driving behavior. Most such studies have also only considered driver performance during computer simulations. No research has yet looked at actual collisions of large trucks.

“But in recent years, there has been an explosion in the use of dashcam video recorders,” said Hajime Kumagai, lead author of the paper and a specialist in sleep medicine with the Graduate School of Biomedical and Health Sciences at Hiroshima University.

“This means that a great many traffic accidents, and their possible relationship to episodes of microsleeping have been recorded. This inspired us to employ dashcam footage to investigate microsleep-related behaviors immediately prior to real-world truck collisions.”

In total, the researchers reviewed 3,120 seconds of video footage from interior and exterior dashcams across 52 cases from truck collisions in Japan that had actually happened. They analyzed the footage from one minute before the crash right up until the crash. In each incident, the person at the wheel had been a professional driver, and all crashes occurred on either urban roads or highways.

The footage was visually analyzed in a second-by-second manner to simultaneously evaluate any eye changes and microsleep-related activity including the driver’s actions to try to prevent falling asleep (anti-sleepiness behaviors), behavioral signs of microsleep, and abnormal vehicle behavior.

The researchers concluded that the key signs of microsleep include behaviors suggesting a creeping sleepiness, such as the absence of body movement, relaxation of antigravity muscles (the muscles such as back and neck muscles, as well as calves and quadriceps used to support ourselves against the force of gravity), eyes being half-closed, and closure of the eyes for one second or more.

“Up to now, technologies used to attempt to detect and counter falling asleep at the wheel have focused mainly on monitoring the driver’s eyes,” said Toshiaki Shiomi, a co-author of the study. “What we found means that it is crucial to monitor not just the eyes but also the driver’s whole body, as well as vehicle behavior in order to reliably detect microsleep-related activity.”

This should also help in avoiding false alarms by ensuring a clear distinction between microsleep-related and non-microsleep-related behaviors.

Developing advanced safety systems that can detect and alert drivers of potential microsleep episodes involving a whole-of-body approach combined with monitoring of the vehicle for typical microsleep-related abnormal vehicle maneuvers could be an effective intervention for reducing accidents in the trucking industry.

The team hopes now to support work developing such advanced safety systems. These might include technologies that monitor both the inside and outside of the truck using cameras or other sensors, as well as deployment of algorithms that can analyze such data in real-time to detect signs of sleepiness or fatigue. These systems could then alert drivers with an alarm when they are at risk of falling asleep at the wheel.