Strangely enough dangerous driving such as speeding and hard breaking is NOT the main cause of accidents. Of all car accidents 94 percent are caused by driver error and 57 percent by distracted driving.
Note: The number of deadly and serious accidents is not decreasing for many years so we need to do something more radical.
The idea is change driver behavior by monitoring and improve driving style in order to reduce injuries and damages.
The driver coach system will check for all safety aspects (360 degrees 3D aspects):
- Warn for Distractions (eating, drinking, phone usage 📞, loud music 🎵)
- Alert for Drowsiness (tired driver 💤 wakeup sound 🔈)
- Improve your Driving style and skills (situational awareness, mirror checking)
The objective is to make driving safer by developing an affordable personal driving coach. Therefore I open sourced this plan and its code and hope others including companies and organisations will step in and adopt it.
Approach:
- Get more people involved in the project (developers, safety experts, evangelists, test dummies)
- Verify that the idea is sound and can result in an affordable and fully functioning solution
- Develop a working demonstration to show its capabilities
- Develop software components and building blocks (software libraries, hardware design, system configurations)
- Open source the idea, plans, methods and software libraries
- Broadcast the idea, get organisations involved
- Deliver a device that can do the job and is afforable for everyone
There is a path from idea to a working solution that has to be discovered. This requirers significant effort and expertise. I hope people will step in to join. Organisations can join to and decide to contribute to he project or step out at some time and continue on their own. All for the greater good of mankind and ethernal glory.
You can find the plan in the project wiki a link to the prsentation pdf , ongoing activities in project. Any help is welcome.
Note: the video shows Rick August and his safe driving instruction video's. We cannot put a Rick next to every driver. But we try to provide useful feedback to every driver Rick August: Smart Drive Test video
Each year traffic accidents in the Netherlands with 6.8 million cars cost about 600 lives, 30.000 injuries and 17 billion euro. Each fatal accident costs 1,5 million euro and each serious accident about 250.000 euro.
Currently most car lease and insurance companies focus on reduction of aggressive driving habits (speeding or hard breaking). They provide their customers with Apps that use Smartphone sensors for tracking vehicle location and acceleration. But this approach does not cover all aspects (distraction,drowsiness and driving maneuvers) and is therefore less effective to reduce accidents. Drivers usually know when they have aggressive driving habits. and aggressive drivers are usually not willing to change their habits. But cautious drivers are the majority and they are willing to learn and improve their driving skills in order to make their life safer. From a safety perspective the driver is better helped with a system that monitors driver alertness, warns them in potential dangerous situations and provides useful feedback for improving driving skills. This will reduce their chance of getting into an accident with 57 percent.
How can we get the best for everyone? Companies that care about social responsibility to their customers should consider all previous arguments and design a system that provides real benefits for their customers. Such a system is however more complex and costly to develop and implement. An open source solution might be helpful to reduce cost if its flexible enough to be adapted to specific requirements.
So the main question is: How can we develop an affordable and flexible solution that provides benefits for all
| Consumer | Business case |
|---|---|
| * Promote safe driving habits | * Car lease (safety as a service, reduce car accidents) |
| * Improve safe driving behaviour | * Insurance (reduce car accidents) |
| * Help to prevent car accidents | * Road service organizations (safety for its members) |
| * Make Driver Monitoring systems available for low cost | * Driving school (better feedback, better results) |
| * Lower costs for society | |
| * While keeping privacy |
A good starting point to introduce this system would be driving schools. They could use these devices in their daily practice and lend them to their customers when they practice under parental supervision. By lending them to young drivers on training, consumers will getting used to these devices and this will lead to general acceptance.
I think the biggest challenge is to convince people to use these kind of systems. Acceptance depends on two factors: how effective it is in making driving safe and if we can trust the system behind it. The effectiveness has to be proven with field trials and the results must be verified and reported to the public.
We all have our thoughts on devices that observe us especially when we are making errors. Everything the system does should therefore be crystal clear:
- How does it work
- What kind of data is being processed
- How does it make its decisions (criteria)
- What will be done with conclusions
- Who owns this data and has control over it
- Can i put my trust in this system
Everyone has to make up their own mind and decide:
Do I want to assess and improve my driving skills in order to make driving safe for myself and the people around me.
I personally hope that we can make driving safer. My wife lost her father in a road accident and every life is valuable and worth saving.
To get user acceptance I propose to publish the general design of the system so people can understand how it works and what it does. A further assurance might be an independent auditor or trustworthy organisation, auditing the final product and a statement that the product does what is promises.
After each trip the system sends a summary to the driver's smartphone for quick review. When at home specific situations can be reviewed in depth with the Driving Coach App. The App will show snapshots or short video with detailed data and advise for improvement.
The collected data is kept on the device under full control of the driver and can be deleted when required. Data can be shared with selected parties for specific applications such as an insurance company or a safe driver leader board where drivers can compare themselves with other drivers.
The plan is to setup a demo to verify its feasibility, then to develop a prototype for field testing to investigate its effectiveness.
Developing a driver monitoring system is not a trivial task, and there are several ways to get there. The easiest is of course just to buy such a system from a reliable supplier or develop one using a toolkit. But this may be expensive or not suit your needs. And how do you get the expertise to choose the right solution? I decided to try to develop one myself to see how far I can come. Its an interesting project.
- Develop a basic prototype
- learn more about the subject of safe driving, sensors, data processing, machine learning
- build a proof of concept based on a simple system such as laptop / webcam / smartphone
- find out if its possible to develop the POC into a cost effective solution
- demonstrate its capabilities
- If this first step succeeds, try to interest parties to actually further develop and use such systems
- If not then I have lost nothing and learned a lot.
For now im evaluating some vision and machine learning toolkits such as OpenCV, Microsoft OpenVino and Google MediaPipe. Deployment on Smartphone or IoT devices such as Coral Dev Board, Jetson Nano
- Good driving habits
- Wearing seat belt
- Active driving posture
- Hands on the steering wheel
- Looking to the general driving direction
- Checking for traffic from all directions
- Left/right/rear mirror checking
- Signalling direction changes to other drivers
- Abstain from distractions
- Mobile phone usage (handheld, call, texting)
- Operating the console (radio / airco)
- Talking to passengers
- Eating or drinking
- Other (reaching for something, singing)
- Driver physics
- Body, arm and hand position
- Head pose (rotation, looking direction)
- Driver drowsiness
- Eye blinking/closed
- Head bobbing
- Mouth yawning
- System design
- Specifications (domain knowledge
- Determine actual driving state based on sensor data
- Combine driving state with driver activity and evaluate its behavior (scoring)
- System limitations (processing power, battery consumption, operating system)
- Physical conditions
- Lighting conditions
- Vibrations
- Mounting (where, how, car types)
- Calibration & Testing (test data)
- Ethical and legal issues
-
Driver Sensors
- Face location (is there a driver)
- Driver recognition (who is driving)
- Head Pose estimation (looking direction)
- Face landmarks (eyes, mouth)
- Hand location (on steering wheel)
- Image classification of dangerous situations
-
Vehicle sensors
- Acceleration (left/right turn, breaking)
- Vehicle speed
-
Tools
- Python, OpenCV (CV2), Dlib, fast.ai
- Python-video-annotator code / documentation
- UAH DriveSet Reader git
-
Datasets
-
Example solutions
- Futurebridge link
-
Example code:
- Learn Open CV Examples code
- Brain4Cars technical research website
- Ground AI Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach* link
- Gaurav Shadev Drowsiness Detection github
- Aontoine Lame GazeTracking code
- CNN Human face detector model
- DBSE-Monitor
Selecting the best face detection method: Source: Face Detection – OpenCV, Dlib and Deep Learning ( C++ / Python ) link
