With autonomous vehicles looming over the not so distant future, many tech wizards are working hard to improve the less than perfect detection technology we’re currently using.
Daniel Gehrig and Davide Scaramuzza from the Department of Informatics at the University of Zurich (UZH) have combined a bio-inspired event camera with artificial intelligence to develop a much more effective detection system.
7,508 people were killed by cars in America in 2022, the highest rate of death by vehicles in 41 years, according to the Governors Highway Safety Association.
Most current systems are frame-based, 30-50 frames per second, alerting the driver audibly or visually, and maybe activating emergency braking. An artificial neural network can be trained to recognize objects in their images, such as pedestrians, bikes, and other cars.
"But if something happens during the 20 or 30 milliseconds between two snapshots, the camera may see it too late. The solution would be increasing the frame rate, but that translates into more data that needs to be processed in real-time and more computational power," says Daniel Gehrig, first author of the paper, published in Nature.
Event cameras are a recent innovation and work slightly differently. Instead of a constant frame rate, they have smart pixels that record information whenever they detect movement.
“This way, they have no blind spot between frames, which allows them to detect obstacles more quickly. They are also called neuromorphic cameras because they mimic how human eyes perceive images", says Davide Scaramuzza, head of the Robotics and Perception Group.
Get the Digital Camera World Newsletter
The best camera deals, reviews, product advice, and unmissable photography news, direct to your inbox!
Similar to the frame cameras they have their own shortcomings. They can miss things that move slowly, and are not easily converted into the data used to train the AI algorithm.
Gehrig and Scaramuzza have created a hybrid product between the two systems.
It incorporates a standard camera that takes 20 images per second, and an AI system called a convolutional neural network to recognize cars or people.
The data from the event camera is then used with another AI system, an asynchronous graph neural network, which is known to be good at analyzing 3D data that changes over time.
Detections from the event camera are used to anticipate detections by the standard cameras, and also boosts its performance.
“The result is a visual detector that can detect objects just as quickly as a standard camera taking 5,000 images per second would do but requires the same bandwidth as a standard 50-frame-per-second camera”, said Gehrig.
The biggest advancement here is that the system can effectively detect cars and pedestrians that enter the field of view between subsequent frames of the standard camera. According to Scaramuzza and Gehrig, this will make driving safer for both drivers and pedestrians, especially at high speeds.
According to the scientists, this design could be made more powerful in the future by integrating LiDAR sensors, like the ones currently used in self-driving cars.
Keep yourself safe while driving and record any dangerous traffic with our guide to the best dashcam, and the best front and rear dashcams.
Also check out our guide to the best budget dashcams.
