New turning assistant for trucks with virtual reality goggles

Researchers at Fraunhofer IOSB have developed a new vehicle assistance system to help truck drivers maneuver. A virtual reality goggle no longer gives them a blind spot.

A simulated look from the cab of a truck. The camera shot is superimposed on the driver's perspective. Photograph: Fraunhofer IOSB

A simulated look from the cab of a truck. The camera image overlaps with the driver’s perspective. A car is visible in front of the vehicle as if the truck were transparent.

Photo: Fraunhofer IOSB

Every third accident between a truck and a cyclist happens when turning. Because the blind spot is large, and so far only a small part of the vehicles on Germany’s roads are equipped with assistance systems such as cameras. It is not just the turning maneuvers that can lead to dangerous situations. Even obstacles directly in front of the vehicle can not be seen by the driver from his elevated position. The Fraunhofer Institute for Optronics, System Technology and Image Evaluation IOSB has now developed a solution for such situations that goes beyond the familiar camera assistance systems: thanks to virtual reality glasses (VR glasses), the driver can drive through the truck see through. The scientists also tested the principle for passenger cars.

3-D imaging with stereo cameras

Even with additional mirrors, it is extremely difficult to maneuver a truck with a trailer and to cover all relevant areas to capture. In city traffic, it is also crucial to work with a system that the driver can use as intuitively as possible to avoid possible delays in the response time. Daniel Reichert has therefore developed a novel vehicle assistance system at Fraunhofer IOSB that makes the truck transparent in the driver’s perception.

This is made possible by the use of stereo cameras. Stereo cameras have two lenses that record at the same time, so that the automatic composition of the images creates a three-dimensional impression. A computer generates a virtual view from it. The pursued but the driver is not on a screen, but on a VR glasses. At the same time, his normal driver’s perspective overlaps with the three-dimensional camera images. Visually, the body is partially transparent. If the driver wants to check whether an obstacle is directly in front of the truck, he looks down and sees the empty road – or the pedestrian, which is detected by the cameras.

Algorithm calculated when maneuvering position of objects

As additional security, a special algorithm detects the exact position of objects in the room and matches them with the position of the vehicle. If the distance becomes too small, the software will first color the obstacle orange and then red as soon as proximity is considered dangerous. The coloration makes in the perception of the driver obviously a big difference, as a test series has shown: Daniel Secker from Fraunhofer IOSB has transferred the system to cars and 15 users asked to park backwards, first without and then with the virtual reality glasses , also with and without colored object (s). The result was that the glasses helped the drivers to better estimate the distance to the objects, but only through the optical coloring of the boxes. The VR glasses alone did not significantly improve the parking behavior.

The scientists are satisfied with their results, which in their view are not yet marketable. Because the used virtual reality glasses were very clunky and therefore not comfortable to wear. The goal was practical VR glasses that the driver could wear like ordinary glasses. The additional virtual display levels would then only appear during maneuvering maneuvers. The eyewear available today would have too narrow a field of vision.

Federal Government subsidy program for turn-off assistants

Turn-off assistants can significantly reduce accidents involving trucks. It is therefore likely that they will become compulsory in the EU from 2024, but only for new vehicles. The Federal Ministry of Transport is trying to accelerate this development for Germany through a subsidy program. Since the beginning of 2019, 5 million euros have been made available per year, with an estimated 5 years. For the current year, the subsidies are already bound by the high number of applications received, so the demand is high. In addition, there have already been 44 companies with logistics operations for the “Aktion Abbiegeassistent” of the Federal Ministry of Transport obliged to retrofit their lorries and have the turn-in assistants integrated directly in new purchases.