SAN JOSE, CA and SAN FRANCISCO, CA – MAY 17, 2023 – AVCC®, a global autonomous vehicle (AV) consortium that specifies and benchmarks solutions for AV computing in the automotive industry and MLCommons®, an open, global engineering consortium dedicated to making machine learning (ML) better for everyone, are announcing the industry’s first Automotive Benchmark to take specifications and benchmarks and move them through to open-source software and certification. Initial participation in the benchmark includes Arm (co-chair), Bosch, cTuning Foundation, KPIT, NVIDIA, Red Hat, Qualcomm, Inc. (co-chair), Samsung Electronics, and other industry leaders.
The use of ML, especially for the perception system in autonomous vehicles, has increased dramatically over the last few years, unlocking new innovations such as automatic lane-keeping that make roads safer. As vehicles are becoming more intelligent, the industry needs a common set of ML benchmarks to enable fair and accurate comparisons between different technologies.
The goal of the partnership is to develop an industry-standard Automotive Benchmark Suite for use by OEMs and automotive suppliers using AI/ML Deep Neural Network (DNN) technology. The Suite will build upon the AVCC AI/ML Benchmark Technical Reports and the MLPerf™ benchmark suites developed by MLCommons. This common set of benchmarks will also help guide the industry’s collective engineering around future platforms, accelerating the development of new capabilities. Development of the automotive benchmark suite will occur in a multi-phase approach, with the goal of delivering open-sourced software solutions by the end of the year.
“We are excited to bring our expertise in machine learning to the automotive industry, working together with the AVCC,” said David Kanter, executive director of MLCommons. “We believe this opportunity will help spur innovation and encourage standards around increasingly intelligent and capable vehicles.”
“OEMs and automotive suppliers are currently challenged to understand a solution’s compute performance and system resource requirements,” said Armando Pereira, president of AVCC. “The work of our joint task force will finally give the industry an easy and certified source of information these players need to make significant decisions on their selection of suppliers and project investment.”
The technical work has begun, and broad participation from the industry is encouraged. The Automotive Benchmark initiative seeks input from all AVCC and MLCommons members, along with other industry players. If your organization is using (or looking to use) automotive AI technology, you are encouraged to participate in this joint initiative to support the development of the new Standard Automotive Benchmark Suite.
To join the Standard Automotive Benchmark Suite initiative, visit: https://mlcommons.org/en/groups/inference-automotive/.
For more information about AVCC, visit: www.avcc.org.
For more information about MLCommons, visit: https://mlcommons.org/en
About AVCC
AVCC is a global autonomous vehicle (AV) consortium that specifies and benchmarks solutions for AV computing, cybersecurity, functional safety, and building block interconnects. The AVCC is a not-for-profit membership organization building an ecosystem of OEMs, automotive suppliers, semiconductor and software suppliers in the automotive industry. The Consortium addresses the complexity of the AV environment and promotes member-driven dialogue within technical working groups to address non-differentiable common challenges. AVCC is committed to driving the evolution of autonomous and automated solutions up to L5 performance over the next decades. www.avcc.org
About MLCommons
MLCommons is an open engineering consortium with a mission to make machine learning better for everyone through benchmarks and data. The foundation for MLCommons began with the MLPerf benchmark in 2018, which rapidly scaled as a set of industry metrics to measure machine learning performance and promote transparency of machine learning techniques. In collaboration with its 50+ founding partners – global technology providers, academics and researchers, MLCommons is focused on collaborative engineering work that builds tools for the entire machine learning industry through benchmarks and metrics, public datasets and best practices.
