One of the most drastic differences between vehicles today and vehicles ten years ago is the sudden proliferation of ADAS. 

A general overview of different ADAS technology. Image used courtesy of Continental Engineering Services

Almost any new car today is rife with sensors and cameras to ensure driving is as easy and as safe as possible for the user.

To support these ADAS systems, a plethora of computing resources is necessary. One of the most popular computing resources in ADAS systems is the ISP.

Today, Arm announced a new ISP designed specifically for ADAS and automation applications, which we’ll discuss, as well as the role of ISPs in ADAS systems.

The Need for ISPs in ADAS

Driving is a visual task, and, as such, cameras have become arguably the single most crucial sensor for enabling ADAS.

In today's ADAS systems, one can find cameras serving all sorts of purposes, such as: 

• Surround-view systems
• Driver monitoring systems
• Object detection/collision avoidance systems

Behind this hardware, ADAS relies on a series of image processing and machine learning techniques such as computer vision and object detection to interpret what is happening in these images and how to respond. 

An overview of an ADAS, which often relies on visual input. Image used courtesy of Synopsys

To enable this in hardware is no easy task. ADAS systems require extremely low latency computation, especially since driving requires real-time decision-making and is often a life-or-death scenario. 

This requirement means that ADAS systems must acquire, pre-process, process, and make a decision based on camera inputs in a matter of milliseconds. Achieving this is a challenging task, as cameras have a very high throughput, producing enormous amounts of data.

To address this, designers must use specialized computing hardware such as ISPs. 

In general, an ISP is designed to accelerate the data processing and computation required by image-based systems, leading to low power and faster performance processing.

A New Addition to Arm's Mali Series: the Mali-C78AE

Hoping to tackle those ADAS design challenges, Arm's introducing its newest ISP for ADAS: the Mali-C78AE.

The new ISP, built on the same technology as the Mali-C71AE, is explicitly designed for data processing. 

Arm claims it can simultaneously accept inputs from up to four real-time cameras of 16 virtual cameras, with the device supporting up to 1.2 Gigapixels/second of throughput. It hopes that this high throughput, multi-input capability will significantly reduce the complexity of a system design, making for a more efficient system overall.

Internally, the ISP integrates several image processing blocks, including noise reduction and defect pixel correction, which include: 

• Chroma noise reduction
• Purple fringe correction
• Spatial noise reduction
• Local tone mapping functions 

The ISP also offers dynamic range management, featuring a 24-bit dynamic range processing unit. 

Altogether, Arm states that the new ISP has over 280 fault detection circuits built-in.

Functional diagram of the Mali-C78AE. Image used courtesy of Arm

From a systems level, the device is meant to be combined with other Arm offerings, such as Cortex-A78AE and Mali-G78AE. 

According to Arm, the combination of these three units can provide a full ADAS vision pipeline optimized for both performance and safety. So far, the device itself has received an ASIL-B automotive safety rating.

All in all, Arm is hoping to create the next step in evolution for its ADAS offerings. 

The Next Step for ADAS

As ADAS continues to integrate into almost every new vehicle today, ISPs will continue to be necessary. 

With the Mali-C78AE, Arm hopes to provide the industry with an improved solution, increasing overall system safety and performance.

Interested in other ADAS innovations? Catch up on the articles down below.

2022 Revs Up ADAS With Radar, EEL Lasers, and Distance Sensors for Autonomous Vehicles

Intel Takes CES Stage to Announce Processors and ADAS Roadmap

From Inductors to Sensor Testing: 3 Companies Promote ADAS Safety