World premiere: ZF Annotate uses AI for the development of ADAS and AD systems

Modern, automated driver assistance systems require a large number of sensors in order to precisely analyze the vehicle’s environment and derive safe driving maneuvers. To further advance the development of these ADAS and AD solutions, the technology group has developed the cloud-based and AI-enabled validation service ZF Annotate. At the ZF Global Technology Day on June 27, 2024, in Jeversen/Hanover, the Group presents the fast, cost-effective and market-ready service for passenger cars and commercial vehicles to the media for the first time.

Precise and reliable data is essential for the development of advanced assistance systems in vehicles. Cameras, radar, lidar or ultrasonic sensors continuously supply information from which the vehicle creates a three-dimensional image of its environment. The systems must recognize a wide variety of objects in real time, including vehicles, people, lanes and traffic signs, for example.

This sensor data must be processed digitally correctly so that the vehicle always receives the “absolute truth” – known in the industry as “ground truth” – to calculate and implement a driving function based on it. Comparing the collected sensor information with a reliable and high-precision reference sensor set increases accuracy. This is where ZF Annotate comes in.

AI for more speed and increased safety

Based on the customer’s own vehicle data and additional ZF sensor data recordings – the reference measurement – the cloud-based service solution provides the “ground truth.” ZF Annotate acts as a redundant setup that is independent of the sensor set to be checked and is confronted with the same information while driving on the road. The recorded data is then uploaded to the cloud and analyzed. Thanks to artificial intelligence, all relevant objects are accurately marked, classified, attributed and assigned unique ID numbers and moving objects are tracked. This object information forms part of the complete description of the environment model — the ground truth. After this “annotation,” the software provides a highly precise comparative measurement. This makes ZF Annotate a state-of-the-art, AI-supported validation solution for testing and training modern ADAS/AD systems from Level 2+ to Level 5.

“ZF Annotate combines the advantages of a robust and independent reference sensor set with a scalable cloud service that uses intelligent 2D and 3D tracking algorithms,” says Dr. Holger Klein, Chairman of the Board of Management.

Previous comparable systems mainly relied on 2D annotation for the validation of reference data and thus map the environment in distance and horizontal angle. The 3D-capable ZF Annotate adds height information to the data. “This makes our solution a unique, intelligent ‘cloud factory’ in the industry, which optimizes a previously labor-intensive, costly service in several respects,” explains Klein.

Up to 10 times faster and 80 percent cheaper

The precise and reliable data from ZF Annotate’s reference measurements can therefore significantly accelerate the development and fine-tuning of complex ADAS and AD systems. Until now, the validation of such systems involved a great deal of work and was correspondingly time-consuming and costly, as the reference data was traditionally annotated manually by humans.

“Thanks to artificial intelligence, we can speed up the validation process by up to 10 times and thus reduce an annotation from 12 to two months. This saves our customers time and money,” explains Klein. “With ZF Annotate, we are able to generate a ‘ground truth’ in the shortest possible time,” says Klaus Hofmockel, Head of Research and Development in the Driver Assistance Systems and Electronics division. “With the ability to work 24 hours a day, seven days a week, our cloud-based service completes the validation of reference data in a remarkably short time compared to the market, without any loss of quality.”

Develop once, roll-out everywhere: Flexibility in every respect

Depending on customer requirements, the reference sensors are either used on the test vehicle itself or in “Pursuit” mode. This is a sensor set mounted on a separate reference data vehicle. Pursuit mode can be used without major adjustments to the vehicle being tested – for example, when evaluating parking scenarios in public spaces.

This flexibility in application makes ZF Annotate independent of specific sensor manufacturers. The otherwise cost-intensive modification of customers’ high-tech test vehicles is also no longer necessary. Customers can therefore also include the service in development projects that have already been started. In this way, ZF Annotate supports the validation process in the most effective way and thus the goals of every developer on the way to a flawless product.

Furthermore, the reference data recorded is not just limited to a front view. Depending on customer requirements, the reference sensor set can provide a comprehensive 360-degree view, providing a detailed and precise representation of the vehicle’s surroundings.

ZF is known for developing new technologies for various application areas. ZF Annotate can be used across all vehicle classes in both the passenger car and commercial vehicle sectors.

“Reliability and efficiency are significant when it comes to validating reference data in the development of increasingly popular advanced driver assistance systems,” says Klein. “ZF Annotate offers both in a scalable system. With our solution, ZF is delivering another building block on the road to the software-defined vehicle.”