AVCC Members: you have immediate priority access to this report in AVCC’s archives. Please log in to download, or email [email protected] for assistance with the report.

Not an AVCC member? A complimentary abstract of the report is below. If you would like additional information or access to the full report, please send an email to our membership team at [email protected] and we will reply shortly.

TR-007: Compute Scenarios for Benchmarking Machine Learning for Automated and Assisted Driving Systems​

Technical Report 007 (TR-007) is the third and final TR on recommendations for how to benchmark Machine Learning (ML) for automated and assisted driving systems. It completes TR-003 which addresses conditions and reporting, and TR-004 which focuses on models and datasets, by decomposing complex driving functions into less complex building blocks. These building blocks capture the essential structure in an automotive ML pipeline and are referred to as the AVCC compute scenarios, or simply as compute scenarios. 

Scope of the Document

There are four defined compute scenarios that complement each other and stress the system under test (SUT) in diverse ways. For example, by running a compute scenario, a consumer of the performance data will be able to tell if the SUT is suitable for a batch compute or sensor fusion use case. If desired, compute scenarios can be strung together to create a slightly more complex use case. This can be useful when latency performance data of the longest ML pipeline in the driving function needs to be known. 

This TR has three sections leading up to the AVCC compute scenario chapter: sensor suite, compute architecture, and function category. These sections lay the foundation for the AVCC compute scenarios and decomposition sections. The purpose of the first section is to determine and quantify the amount of data which is produced by the sensor suite in three phases: raw, pre-processing, and post-inferencing. The second section shows how the compute architecture in a vehicle can be structured to process the sensor data. The third section describes how the driving features can be implemented by processing the sensor data in the compute unit(s) in the vehicle. Generally, a more sophisticated sensor suite requires higher compute capability to process the data, which increases the cost of the vehicle. In turn, this enables higher levels of automation. 

Together, TR-003, TR-004, and TR-007 will give OEMs, Tier 1s, and suppliers the guidance necessary to develop a common set of benchmarks enabling fair, or also, “apples-to-apples” comparisons. In addition to making fair comparisons between different solutions, it also allows suppliers to invest in the right solution. This will help the automotive industry move faster and make procurement decisions with higher confidence, leading to more competitive and cost-effective solutions and in the end, better products for the consumers. 

An overview of the Table of Contents:

• Introduction
• Sensor Suite (Sensor Types and Sensor Data, including camera, lidar, radar, audio, voltage and current, ultrasonic, time alignment)
• Compute Architecture (Endpoint, Zonal/Domain Controller, Central Compute)
• Function Category (Collision Avoidance Static, Collision Avoidance Dynamic, Trajectory and Motion Planning)
• AVCC Compute Scenarios (Independent, Concurrency, Batch, Fusion)
• Decomposition / Identification of AVCC Compute Scenarios

For access to the full report, please email our membership team at [email protected].