When it comes to designing automotive Electronic Control Units (ECUs) for vision-based applications, such as Advanced Driver Assist Systems (ADAS), surround view cameras, or automated driving features, the choice between a discrete image signal processor (ISP) and an ISP integrated into the main system-on-chip (SoC) is more than just a technical detail. It is a decision that shapes flexibility, performance, system cost, and scalability.
Understanding the Benefits of a Discrete ISP
Superior Customization and Flexibility
Discrete ISPs typically offer more advanced image processing capabilities than their integrated counterparts. In applications demanding nuanced image tuning, simultaneous optimization for both human and computer vision, or specialized features like advanced high dynamic range (HDR) processing or support for multiple sensor types (e.g., IR, thermal, new color filter array (CFA) patterns), a discrete ISP is often preferable.
Figure 1. Example of a discrete ISP and system-on-chip (SoC)
Performance and Resource Optimization
SoCs used in high-performance automotive systems are tightly integrated with many different components (CPU, GPU, NPU, peripherals etc.). While these components operate independently within the system, they all share the same dynamic random-access memory (DRAM) subsystem and in many systems, access to DRAM becomes the major resource bottleneck. Image processing is a DRAM-intensive task, often requiring multiple reads and writes from and to the DRAM. By offloading this task to a dedicated ISP, the SoC can reserve a bigger share of the DRAM bandwidth for critical compute tasks in the system.
Discrete ISPs, such as the iND880, utilize a line-based ISP processing architecture, which eliminates the need for DRAM to store frames. Processing video line-by-line rather than frame-by-frame reduces ISP latency. Reducing latency delivers data to ADAS algorithms sooner, enabling faster detection in critical functions such as Automatic Emergency Braking (AEB) and Automated Parking Assistance (APA). This early recognition supports prompt emergency responses, including timely braking or collision avoidance.
Scalability for Multi-Camera Setups
As the industry advances toward higher levels of autonomy (Level 2 → Level 3 → Level 4), ADAS architectures are integrating more cameras. According to a report by Mordor Intelligence, leading automotive camera supplier Sony forecasts each vehicle will embed 12 cameras by fiscal 2027, up from 8 today. A typical vehicle configuration with 11 cameras is shown in Figure 2.
Figure 2: Typical vehicle configuration with 11 cameras
The more ADAS cameras an architecture employs, the greater coverage, accuracy, and redundancy is delivered, leading to smarter vehicles with enhanced safety capabilities. However, this presents a significant challenge in synchronizing and processing the video streams, which can overwhelm an integrated ISP. Discrete ISPs, on the other hand, are designed for exactly this kind of multi-channel, high-bandwidth environment – often employing line-based rather than frame-based processing to maintain more precise synchronization. An example is the iND880 ISP with its 1.4 Gpixel/sec throughput can process four 3MP sensors at 60fps or two 8MP sensors at 60fps with bandwidth to spare.
Independence from SoC Roadmaps
With a discrete ISP, an ECU architecture is not restricted by the capabilities or update cycles of a single SoC vendor. This can be a distinct advantage for long automotive design cycles or when the need arises to change SoC for performance/cost or supply chain reasons. The discrete ISP can be optimized and deployed as standard across platforms and vehicle models, present and future, eliminating the need to re-tune the ISP, shortening the design cycle and ensuring state-of-the-art image quality is maintained throughout. Additionally, standardizing on a discrete ISP that pairs with various SoC’s across platforms ensures scalable deployment of AI models across platforms, preserving inference fidelity and eliminating the need for resource intensive retraining.
Advantages of an Integrated ISP Approach
Lower Cost and Power
For many mainstream automotive applications, an integrated ISP in the SoC is “good enough,” meaning that most of the time the video quality is acceptable; the exceptions being automotive-specific corner cases – less common scenarios that are not handled to the same level of quality. ISP integration eliminates the need for a discrete ISP and associated interconnects, reducing component count, lowering power consumption, and reducing system cost.
Simplified Design and Faster Time-to-Market
With the integrated ISP approach, removing the discrete ISP from the design reduces board complexity, allowing for a more compact, lightweight, PCB design and a more streamlined supply chain. With fewer components there is a natural reduction in points of failure, enhancing reliability and functional safety management. For applications where the image processing requirements are standard, integration reduces overall product complexity and can speed up development and deployment. In some cases where the camera video stream is only to be consumed by AI inference algorithms that operate on RAW video streams, there is no need for ISP processing at all. This allows for direct camera interconnect to the SoC. However, it should be noted that running AI algorithms (training and inference) on cleaner ISP-processed data offers performance advantages.
Tight Integration and System Responsiveness
Integrated ISPs can work seamlessly with other SoC modules (e.g., CPU, GPU, NNE, security) through direct on-chip interconnects and a common hardware/software platform. The resulting simplified data flows can improve overall system responsiveness. Additionally, a SoC with an integrated ISP allows processing to take place in centralized compute architectures, which Yole Group projects to grow 36% annually through 2029.
In the table below, we take a closer look at comparing the discrete ISP and Integrated ISP approaches.
Figure 3: Discrete ISP vs SoC-Integrated ISP
Other Considerations
It should be noted that some of the benefits in the above table can vary depending on specific architecture. For example, multi-camera ISPs processing video on the edge (also referred to as distributed architecture) can process video from up to 3-4 cameras/zone and output a single data stream, on a single cable routed to the central compute processor—reducing the cost/weight of cabling.
There is also a trend towards adding more processing power and lightweight NNE’s (AI) to discrete ISP’s, which make them into a form of SoC themselves. These edge-based processing devices can process video and run algorithms and then send just the extracted metadata (e.g., pedestrian detected bounding box coordinates) to the central processing SoC – greatly reducing the transmission bandwidth.
What is the best choice?
With the slowing of pure semiconductor exponential scaling (commonly known as Moore’s Law), the pace of integrating ever more functionality and logic gates into a single, powerful SoC is also decelerating. Simply adding more or faster ISPs to a central SoC may not be the answer going forward. Both SoC-integrated and discrete ISPs will continue to play important roles, as each offers distinct advantages suited to different applications. Designers are likely to adopt innovative architectural approaches that leverage the unique strengths of each solution to best meet the requirements of specific use cases.
For next-generation, feature-rich, multi-camera ADAS or automated driving systems where image quality and flexibility are paramount, a discrete ISP is a powerful tool. It enables pushing the envelope on what cameras can see and how fast the system can react (i.e., ultra-low latency response), without being constrained by the limitations of SoC-integrated ISPs. The indie iND880 is an example of a discrete ISP suited for such designs.
On the other side, for applications prioritizing cost efficiency, lower power consumption, and streamlined designs suitable for mass-market automotive applications, the integrated ISP in modern SoCs is hard to beat. Vendors like Qualcomm and Nvidia are making these solutions increasingly capable.
A third alternative lies between discrete ISPs and SoC-integrated ISPs: edge-based SoCs. These build on the benefits of discrete ISPs by incorporating processing, but not to the scale and complexity of a centralized processing SoC. For instance, edge SoCs may have neural network engines delivering 2–8 TOPS compared to up to nearly 2,000 TOPS in central compute SoCs. This approach enables new systems to achieve more optimal architecture.
Ultimately, selecting the right ISP architecture depends on the system’s ambitions. While maximizing performance is critical to enhancing ADAS safety, this must be balanced against economic considerations. Advances in AI-enabled camera technology are expanding design options, making it possible to achieve high performance cost-effectively. The designer should choose wisely—as the ECU’s vision (literally) depends on it.
Related Products
| Product | Description | |
|---|---|---|
Camera Video Processor |
|
