THE MOVE TOWARDS THE NEXT-GEN SEMICONDUCTOR PACKAGE TECHNOLOGY
Packaging plays an important role in almost all the industry. From cars to rovers, packaging the product before shipping is vital. Doing so cost-effectively is critical to ensure the product cost is not affected.
The same story is valid for the semiconductor industry, where the packaging technology ensures that the silicon die is protected from harsh mechanical to environmental conditions.
In a nutshell, package technology in the semiconductor industry is geared towards providing the following features to the die area:
Thermal: Ensure the heat is dissipated efficiently out of the die to avoid thermal runaway.
Mechanical: To save silicon from any outside impact like vibration or stress.
Connectivity: Enable a pathway for connecting the die to another system via a printed circuit board.
Contamination:Protect the die are from contamination due to long-term usage.
Today, there are numerous package technologies to choose from. Eventually, the decision to decide on the package type is based on both the technical and the business points.
Give the advancement in the semiconductor design and ever-shrinking technology-node, it is becoming a challenge for semiconductor packaging solution providers to come up with innovative packaging solutions that are also low-cost. All this is pushing the need for next-gen – InFO, CoWoS, CoC, FCiP, PoP, and many more – package technology solutions.
The push towards next-gen package technology is driven by several factors:
Die: Semiconductor product development is moving towards More-Then-Moore solutions. This means several new die-based solutions and one of them is chiplets. Chiplets is IP driven design process, where different die come together to form the same system which earlier would have been a single die. Chiplets are proving to be effective both from a cost and yield perspective. To cater to such requirements, package technology solution providers (OSATs) have to keep their innovation team engaged, to ensure not only the packaging solution is at-par with chiplets requirement but are also low-cost due to the stiff competition.
Design: One example of modern semiconductor design change is the heterogeneous architecture. It has provided ways to bring different types of XPU together. This also requires an efficient way to package by considering the thermal profiles of such complex and high-frequency devices.
Size: The shrinking size is also adding to the need to be more innovative in package technology. Not all devices will be small and at the same time also not too big. OSAT is facing a challenge to cater to different markets and while doing so, have to ensure the type or size of the device is not a constraint.
Workload: Emerging workloads are becoming complex year-on-year. These workloads running on the die continuously end up heating the system. To mitigate such issues, new materials and packaging solutions are required. This is also one reason that is pushing OSATs towards next-gen package technology.
OSATs have to continuously provide solutions that cater to both the smaller and larger die areas. This has to be done in coordination with the design houses, to ensure that next-gen package technology meets all customer requirements.
However, lately, OSATs are being pushed from all the corners and are getting beaten in their own game by FAB-LESS to FAB houses.
THE RACE TOWARDS NEXT-GEN SEMICONDUCTOR PACKAGE TECHNOLOGY
The semiconductor package technology is mostly enabled by the OSATs. IDMs also play a crucial role. However, given the rise of the semiconductor packaging and testing business, OSATs are facing stiff competition from FAB and FAB-LESS companies too.
This is leading to both positive and negative impact on all types of semiconductor companies:
OSAT: The primary focus and strength of the OSATs have been the testing and packaging solutions. With growing competition from FAB on the packaging solutions is certainly putting OSAT in a position to not only innovate on the testing to packaging arena but also how to expand the business beyond traditional OSAT services. This certainly diverts the focus of OSATs, as it is more difficult to move towards FAB or FAB-LESS business while also catering to the same customer type.
FAB: FABs all over the world have the biggest advantage due to the availability of high-tech and research facilities that can aid the innovation of next-gen package technology. This is why TSMC and Samsung have been at the forefront of next-gen packaging solutions. At the same time, FABs are also diversifying and entering the OSAT business with a strong packaging portfolio.
IDM: IDM has always enjoyed the best of both world. Intel already has many homegrown packaging solutions for its need, apart from the ability to fabricate, test and assembly in-house. The innovation in the design space is only benefitting IDMs as they can foresee and quickly adapt their business by pushing towards the required semiconductor packaging research and development activity, and quickly turn research into production-worthy solutions.
FAB-LESS: While, majority of the FAB-LESS companies only focus on the design side. The need to have a packaging solution that can cater to their designs is also pushing FAB-LESS to continuously invest time and money in the packaging technology.
Eventually, packaging solution for semiconductor products is driven by cost optimization. The innovative and low-cost packaging solution catering to the next-gen design will win the race. However, to do so massive research and development activity is required. Which certainly demands high CapEx.
To balance the complexity and high CapEx, packaging solution providers (OSATs mainly) will have to be more innovative to ensure the next-gen solution is business-friendly.
The race towards next-gen semiconductor package technology is certainly heating up, however, it is also an opportunity to innovate and create a new market for the emerging solutions.
THE REASON WHY COUNTRIES ARE PUSHING IN-COUNTRY SEMICONDUCTOR GROWTH
The basic and the applied science form the base for any technological advancement. Countries globally have always focused on the importance of these two aspects of science. That is why countries race against each other and invest a lot of time and money to lead science that enables technology.
In the 21st century, the focus on applied science (basic science is not behind, maybe ahead in several aspects) skyrocketed due to the proliferation of digital devices and wireless connectivity. Only a handful of countries quickly realized the importance of leading in a technological solutions that forms that base of a modern digital solution. Some focused heavily on the research and development (design) of novel technologies (WiFi to 5G to autonomous to robotics and beyond), while others emphasized the need to manufacture these for the end customers.
Semiconductors Are Everywhere
The semiconductor industry also can be seen by separating out into design and manufacturing. Countries like the USA focused more on the design aspect of the semiconductor, while Taiwan and China ramped up manufacturing efforts. This worked well until the 5G and COVID raised concerns over the growing inter-dependency, and then started the saga of how leadership in the semiconductor design and manufacturing affects day-to-day technological (from satellites to cars) solutions. This in turn started pushing several other countries (India to Australia and beyond) to focus on the need to design and manufacture semiconductors in-country.
There are several reasons as to why countries (which are behind) should focus on end-to-end in-country semiconductor industry (or at least have enough infrastructure to cater to in-country demand):
Dependency: Relying on other regions for manufacturing or designing semiconductor products is one reason that is pushing countries to focus on the in-country end-to-end semiconductor solutions. This also includes materials to equipment. It is difficult to make everything in-house, hence good enough infrastructure to cater to the local consumer demand and critical national infrastructure of the county is a good way to start.
Import-Export: From smartphones to cars, all are heavily powered by semiconductor solutions. In trade, it is good to keep imports and exports in balance. Countries without vital semiconductor (manufacturing) infrastructure will always end up importing more than exporting. This will put pressure on their forex reserves and will make them fully dependent.
Growth: The semiconductor market is growing due to the growth of semiconductor usage in the modern infrastructure/technology. Automotive is one example, apart from the consumer market, and many other industries where semiconductor solutions are used by default. Countries should capture a portion of the market which will eventually generate employment and business growth. This is critical for long-term financial stability too.
Business: Establishing a high-tech industry like semiconductors (mainly manufacturing, as many countries do have design houses) will not only lead to market growth and employment but will also support other businesses that eventually are the pillars for the semiconductor industry. It can be from equipment manufacturing to raw materials to turn-key (and ever-expanding) civil engineering work to build FABs/OSATs.
Above points fueled with the growing use of semiconductor in all the major infrastructures and technological solutions (that affects every aspect of day-to-day life) is more than enough reasons for countries with lacking semiconductor manufacturing infrastructure to start today for tomorrow’s need.
THE ROADMAP FOR IN-COUNTRY SEMICONDUCTOR INDUSTRY SUCCESS
There are 195 countries and each one of these have their own strengths and weaknesses. It is difficult to create an end-to-end in-house solutions for the semiconductor industry. More so, when the goal is to drive in-country semiconductor growth to the next level.
In reality, there are no roadmaps that countries can follow to gain momentum for an in-country semiconductor to make themselves self-reliant. On other hand, countries today also cannot overlook the importance of having as many points checked from the roadmap that can enable, if not end-to-end, some parts of the semiconductor supply chain. This will allow them to tap into the future market needs and ensure stability in semiconductor market.
The roadmap to success in the semiconductor industry consists of many points and below are the few major ones:
Talent: Nothing can be developed without having the right set of talent (human resources). Countries already have a framework to educate their population. However, the traditional education system is still focusing on core aspects that enable basic training. While there is no harm in doing so, it is about time that countries wanting to engage in in-country semiconductor growth (manufacturing and beyond), needs to start with training programs that focus more on core semiconductor on-field training programs apart from the fundamentals.
Policy: Eventually, investment is a big part of semiconductor growth. While countries looking to steer ahead in the in-country semiconductor solutions have already started to come up with incentives and policies, the approach should be more holistic that focuses not only on the semiconductor growth but also on the regional infrastructure required for the semiconductors. This may be from ensuring airports to logistics to man-power availability, apart from ease of the land and housing facilities.
Infrastructure: The semiconductor supply chain is heavily driven by the turn-key infrastructure. From FAB-LESS to FABs to OSATs, all require support infrastructure to ensure the products reach the market in time. This infrastructure varies from material handling to chemicals transportation to water availability to non-stop electricity. Building FAB/OSAT is one thing and running it non-stop is another. The better the support infrastructure for the semiconductor industry, the more likelihood of semiconductor giants willing to setup future-focused manufacturing setups.
Public-Private: Government support plays a very key role in driving in-country semiconductor growth. This can be from investing in the required infrastructure to providing incentives that can ensure required investment is available to drive manufacturing facilities. A two-way shake hand between the private players and public bodies can drive the needed policies that are friendly to both the businesses and the consumers.
Research: Research is key to long-term semiconductor growth. Countries focusing on the in-country semiconductor ecosystem should provide research funding (mainly countries with lack research funding support) to universities and colleges to focus on the next-gen semiconductor devices and manufacturing solutions. Universities and colleges themselves also can raise funding via industry collaboration with the semiconductor companies.
Academia: Education is vital to every aspect of industrialization. The majority of the countries already have the infrastructure to educate the future workforce. However, the goal to make semiconductor in-country growth requires academic institutes to focus on semiconductor engineering courses apart from the traditional engineering domains.
Cluster: FAB requires billions of dollars before it can run at full capacity and then it takes years to break even. Even then, due to changing technology-node and solutions around it, the process to upgrade FAB is a continuous one. Countries without a FAB wanting to attract/setup one, should focus more on the cluster approach. Cluster-based FAB can be a pooled investment for higher technology-node (older but relevant) that caters to different semiconductor FAB-LESS companies. Such kickstart can then lay the foundation of dedicated sub-10nm foundries.
Outreach: Outreach is about reaching out to other countries and also attracting businesses to showcase what a specific country can offer. This way there is a dialog between the public and private players that can lead to many business opportunities in the semiconductor sector. Countries should do outreach by default, if the goal is to setup the semiconductor manufacturing (FABs to OSATs) in-country.
Future-Tech: Countries that have semiconductor design houses but not manufacturing facilities should set the target on what the world will need in 2040 and not in 2030. This can range from chiplets manufacturing (semiconductor specific) to next-gen 6G wireless solutions to flexible electronics. Doing so will ensure that the companies and countries are creating infrastructure today for tomorrow’s demand. This can give an edge over countries with massive semiconductor manufacturing infrastructure and are not willing to invest further without closing/upgrading existing semiconductor infrastructure.
The above roadmap points cover the majority of the aspect of ensuring in-country success in the semiconductor industry. There can be different points that also are vital for the semiconductor industry growth. In the end, it all boils down to what eventually works and what does not.
THE LONG TERM IMPACT OF IN-COUNTRY SEMICONDUCTOR INDUSTRY
The last two years have shown the growing importance and emphasis on the need for in-country semiconductor industry growth. Countries are putting efforts to lead in every aspect of the semiconductor solutions, mainly in order to make themselves less reliant on other countries while also taking lead in the advanced modern technological solutions.
The in-country semiconductor growth will have two major long term impact:
Self-Reliance: Countries that can take the lead in the semiconductor industry growth will make themselves self-reliant in the long term. Which will prove vital in the long term due to the fact that the consumers solutions to national infrastructure is fully reliant on the semiconductor powered solutions.
Leadership: There is no denying that every country is racing to claim leadership in every aspect of the modern world. Leadership in the semiconductor industry is certainly going to veto on who gets to call itself the superpower.
It is good that the semiconductor industry is getting the focus, but countries will have to be diligent in understanding what works and what does not works as per the demand and supply requirements.
In the end, investment to get semiconductor manufacturing up and running is huge. Any miss-step will put countries behind instead of going ahead. It also takes years of planning to create any kind of massive (and advanced) infrastructure.
Hopefully, the global supply chain the semiconductor industry runs on, stays intact even with in-country semiconductor infrastructure race.
High-Tech manufacturing is not possible without the use of equipment. The equipment ensures that the product being manufactured is defect-free and also meets the quality requirement for the target market.
The same fundamentals apply to the semiconductor industry, where equipment plays an important role. Given the need to fabricate semiconductor products without even touching the wafer/die area, requires equipment that can do the task and that too without adding delay or defects.
Every stage of the semiconductor product development requires high-tech equipment, that is why semiconductor fabrication, testing, and assembly facility is 90% occupied by the equipment. These advanced equipment require only a set of commands (called recipes) to complete the task they are assigned. The automation achieved with this process ensures the FAB/OSAT are running at full capacity 24x7x365. This is why it is often a challenge for the semiconductor FAB and OSAT to ensure the equipment never goes down and this requires periodic maintenance that is often driven by data collection and analysis.
Semiconductor FAB and OSAT often have two different types of equipment:
Advanced: Advanced equipment is often robotic driven. This includes moving the wafers and other materials (mainly reticle). Apart from robotic capabilities, advanced equipment often has data capturing capability that is then stored for further wafer/die analysis.
Non-Advanced: These types of equipment in the semiconductor process are confined to analysis or lot transferring, and do not have the robotic capability and are more useful for manual inspection or testing.
Majority of the equipment in the semiconductor FAB/OSAT are advanced and are designed for the following goals:
Yield: Primary target of any semiconductor FAB/OSAT to ensure there is minimal to zero yield loss. This is only possible if the process of handling and using materials (reticle, wafer, etc.) adheres to the standards that ensure high-yield. This is where equipment come into the pictures, and the materials are handled and processed in a manner that does not negatively affect the semiconductor wafer, thus ensuring minimal yield loss.
Quality: Quality is the primary de-facto target of every semiconductor product. Equipment can be handy to capture any issues/variations/excursion during the semiconductor process. Whether it is during the lithography stage or automated inspection, equipment ensures the process being executed is eventually leading to correct wafer/die formation.
Capacity: The primary target of FAB/OSAT is to ensure the facility is 100% utilized. This is only possible if there is an advanced equipment that is working non-stop day in and day out. Without investing in the equipment to increase efficiency, it is not possible to utilize FAB/OSAT fully.
Accuracy: Advanced equipment is also important from an accuracy point of view. They ensure the target design is fabricated without any gaps. From etching to deposition to even assembly, accuracy/alignment can only be achieved if the equipment is used. If the manual process is used, then it will not only lead to errors but also bring the FAB/OSAT capacity down.
Throughput: Semiconductor products get utilized by different industries and markets. To timely provide them with the products requires high efficiency. This is made possible by advanced equipment that can process wafers after wafers to keep up with the demand. On top of this, any given FAB has numerous parallel equipment doing the batch processing, thus ensuring the throughput is always at the maximum level possible.
The semiconductor FAB/OSAT are utilizing advanced equipment to keep the facility running that eventually drives revenue. However, ever changing technology-node and device structure is also putting pressure on the FAB/OSAT to upgrade their equipment often.
While the majority of the advanced equipment has a plug-and-play feature for easy upgrades, CapEx and maintenance teams are required to keep the equipment running which in turn runs the FAB/OSAT.
Irrespective of the features or the CapEx hurdles, advanced semiconductor equipment is vital for high-quality semiconductor product development.
There are different types of advanced semiconductor equipment that are used by the FAB/OSAT. They often get categorized based on which part of the wafer fabrication the equipment is used for.
Front-End: The advanced equipment for the Front-End process is used in a semiconductor FAB. There are different advanced equipment that does the task as per the process flow they are part of.
Cleaning: This process utilizes equipment to clean wafers/lots post a wafer stage. Whether cleaning is required or not depends on the recipe that semiconductor engineers design for a given product based on its features and requirements.
Device Fabrication: This stage (the most critical one) requires different equipment to carry out different device fabrication steps from lithography to etching to interconnection to diffusion. Using equipment for device fabrication, the end goal of the wafer with dies having FETs is achieved.
Automated Material Handling: This is the most vital part of the process where the equipment is used to deliver a lot (using FOUP) from one section of the FAB to another. The equipment to do this task eventually forms a semiconductor highway inside the FAB, which autonomous vehicles use to retrieve/deliver lots from one stage to another.
Material Control: Equipment used for this process is focused on controlling the materials like chemicals etc., to ensure the equipment/process needing it like it when required. It is vital to ensure it is done with proper safety, otherwise, a small leakage can cause an emergency.
Outgoing Inspection: The last stage of the wafer fabrication is often to ensure the outgoing lot (for the OSAT), is defect-free. This requires the usage of highly automated robotic inspection equipment that ensures any defect is captured (reported too) and the final wafer map is generated accordingly.
After the wafers have been shipped out of the FAB, the next stop is OSAT. Which requires different (some similar) equipment to drive the semiconductor process.
Back-End: The advanced equipment used for the Back-End process is designed for OSAT.
Incoming Inspection: Equipment to perform incoming inspection is similar to the one used for outgoing inspection. The goal is to remove any die defect that occurred during the shipping process. Inspection equipment is high on speed and the data generated is analyzed and stored for the future use.
Lot Transfer: The shipping box that arrives at OSAT eventually has to get transferred to a different FOUP. This is where equipment comes in handy that can take all the wafers in the lot and transfer it to another FOUP without human intervention. This ensures there are no particles or human errors. Tracking is done via lot/wafer id.
Automated Test: Automated Test Equipment (ATE) is loaded with the wafer (along with the probe hardware) and then a test program runs on every die of the wafer. The test is required to capture electrical characteristics, which is part of the process to ensure the defective die (root cause can be many) are not processed further.
Assembly: Depending on the assembly recipe, the equipment takes the wafer and assembles the parts. This stage can involve bumping to dicing. Each of which requires its own set of equipment. This process certainly cannot be done by humans and hence advanced equipment is a must.
Failure Analysis: Eventually, if multiple materials have set failing patterns, then failure analysis needs to be carried out. The advanced equipment required for failure analysis often has biasing capability apart from features to dive inside the die at the gate level with the help of highly advanced X-Ray techniques.
Both FAB and OSAT are heavy equipment driven facilities. While there are opportunities for the equipment industry to enable new solutions, at the same time it requires a balance of equipment features and the usage cost.
THE OPPORTUNITIES FOR ADVANCED SEMICONDUCTOR EQUIPMENT
The advanced semiconductor equipment market is poised to grow mainly due to the race to set up new FAB and OSAT by countries wanting to establish the lead in the semiconductor industry. This certainly presents opportunities for advanced semiconductor equipment manufacturers.
However, given that the semiconductor equipment market itself is highly competitive, the new equipment solutions need to focus more on features that can make the equipment stand apart in the market.
There are many features/areas that advanced equipment manufacturers can explore apart from the following two:
Automation: While the semiconductor equipment is already highly advanced and automated, there are still parts that can be automated. This might be from materials handling to/from the equipment. Maybe a more data-driven approach to finding how to optimize the lot handling process.
Data: Capturing more data and analyzing it to focus on the equipment wellbeing. This can be from understanding maintenance requirements to predicting when the machine can breakdown. Any predictive information can aid in balancing the cost and productivity by ensuring equipment are up and running all the time.
The top players providing advanced semiconductor equipment solutions are increasing market share every year. This is putting pressure on small scale equipment makers that are trying to competing against these top players. Given the increasing focuses on the semiconductor industry due to the shortage, it is about time the equipment makers (small scale enterprise) take note of the gaps and provide solutions.
Apart from the new equipment market, the used marketplace is also getting a lot of traction. Emerging companies are now providing FAB/OSAT with the opportunities to tap into the low-cost equipment that is not new but certainly can provide the same defect-free features/processing as the new ones.
Due to the semiconductor design and manufacturing race to lead the FAB/OSAT market, the semiconductor equipment market is also heating up.
The computer architecture design to cater the personal computing to data center needs has seen many changes over the last four decades. Computer architects have pushed the boundaries of designing novel ways to process data, which has lead to the manufacturing of advanced processing units (XPU) that are powering low to high-performance computing systems.
The growth of XPU is not only driven by the novel designs but also due to the shrinking transistor size. The advanced technology-node has allowed computer architects to take advantage of billions of transistors in the smaller area possible. With the ability to pack more transistors, the processing speed has increased, which has reduced the time taken to process user requests. All these hardware features combined with an elegant software system is powering the system to drive a rich user experience.
However, packing more transistors in the same area is hitting the wall. The benefits that Moore’s law provided a decade ago will not be applicable ten years down the line. Packing more transistors is not only leading to area-wall but is also adding power-wall, performance-wall, and thermal-wall. As more users come online, the expectation to provide real-time data processing will grow and this demands a new design and manufacturing approach for the next-gen XPU. The next-gen innovation should not only drive the required market growth and but must also ensure that the semiconductor industry never hits the usage-wall.
Heterogeneous Integration Is For Next-Gen XPU
On top of all this, the transistor size can not keep decreasing forever. The new technology-node also demands massive investment in the existing and new semiconductor fabrication facilities, thus pushing the semiconductor industry towards new XPU designs that uses a novel combination of technology-node and packaging solutions. These new types of CPUs are/will use heterogeneous integration for semiconductor designing and manufacturing.
There are two aspects of heterogeneous integration that any new emerging solution will have to consider before heterogeneous integration can be used to fabricate the next-gen XPU:
Design: Developing XPU with a heterogeneous integration approach needs different dies (sub-systems) to work in harmony. This demands cycle/time accurate communication between dies/blocks that may or may not come out of the same design team/company. To ensure all the dis-integrated systems can work in an integrated manner, requires a detailed design, verification, and testing approach. Without ensuring all the systems within the heterogeneous integration powered XPU working in synchronization, the end-product might not meet the expectation of the end-customers.
Manufacturing: Fabricating, packaging, and testing all the components of an XPU on a single die is a proven method. With heterogeneous integration, the dies (fabricated individually) will get integrated to form a single system. This will require a novel approach that is cost-effective and does not add a new bottleneck from a manufacturing point of view.
Heterogeneous integration and solutions around it are not new but will be when the industry moves towards large-scale adoption to design chips that follow the heterogeneous integration approach. The main reason is the bottlenecks and issues semiconductor companies without prior heterogeneous integration experience will face when they decide to venture into this territory.
Design and manufacturing are the two key areas that the emerging heterogeneous inspired XPU will have to focus on. However, there are several other challenges that both the existing and emerging solutions around heterogeneous integration will face.
Heterogeneous integration is touted as one of the More-Than-Moore solutions that can take away the need of shrinking transistor size further. However, there are few challenges to overcome when the XPUs are designed using a heterogeneous integration approach:
Data: The goal of any XPU is to ensure the data being processed flows across the blocks in the fastest possible time with near-zero latency. If the systems are going to be more heterogeneous and are stacked in such a way that they eventually have to communicate via a bus, then this may very well add data flow bottlenecks. This challenge can be overcomes via an optical interconnect system, but such solutions are not fully tested for a large-scale computing usage.
Power: Heterogeneous integration certainly provides avenues to overcome the challenges shrinking transistor size brings. However, the focus should also be on Performance-Per-Watt (PPW) to justify XPU designs that uses a heterogeneous integration approach. In the end, the goal of any processing unit like XPU is to provide maximum benefits at the lowest cost possible. Given, the demand to run more sophisticated graphics driven applications, the need to provide better PPW should be the priority of any heterogeneously integrated XPU.
Quality: XPU that will use heterogeneous integration will require qualification before production. In a monolithic chip, all the components are designed and fabricated together, which also gets qualified (JEDEC/AEC, etc.) for production use. However, with a heterogeneous approach, the singulated dies that form an integrated system will have to get qualified individually. This will increase the cost and time taken to execute the qualification plan, and might also raise the question on how different qualification criteria can be applied to integrated XPU.
Thermal: Controlling temperature ensures that there is no thermal runaway in any type of XPU. If heterogeneous integration will take the 3D approach, then managing die-to-die heat transfer can be a challenge. Fast heating will mean throttling the system often and will eventually impact the user experience.
Protocols: Heterogeneous integration also means integrating two different IPs. If the protocols to ensure smooth communication between these multiple dies are not defined and standardized across the industry, it might hinder the progress of next-gen XPU.
Packaging: Integrating all different die to eventually form a single system demands innovative packaging. While System-In-Package (SiP) based solutions are already out in the market, there is a need to test a different approach to packaging that ensures the area and thermal constraints are not leading to bottlenecks.
Above are few challenges that the next-gen XPU powered using heterogeneous integration may face. As more and more XPU design follow heterogeneous integration approach, the challenges can be overcome but might also lead to newer ones.
Above are the major challenges that the next-gen XPU powered using heterogeneous integration may face. However, there are several opportunities also that the heterogeneous integration provides.
The two major opportunities for the semiconductor industry if heterogeneous integration is adopted at a large scale:
Emerging Solutions: The solutions that heterogeneous integration can enable will be novel that can take the best of the IPs and integrate them to form an efficient computing system. These systems might very well provide high-performance computing to the general-purpose domain. The area to power wall can be overcome using heterogeneous integration and will drive design to packaging solutions. The only key is large scale usage as is the case with monolithic chips.
Cost Optimization: Yield is one of the major concerns when a single die is fabricated to manufacture XPU. With heterogeneous integration, there is a possibility to design and manufacture dies separately. It can bring down the wastage and improve yield, which in-turn will lower the cost and will certainly improve manufacturing efficiency.
The time of heterogeneous integration has come. Emerging solutions have started to embrace the possibilities heterogeneous integration provides. It will be interesting to see how general-purpose computing changes with heterogeneous integration design and manufacturing.
The use of virtualization techniques in the software and the hardware industry has enabled several technology developments. Virtualization provides developers the infrastructure to develop, test and deploy solutions catering both to the consumers and enterprises.
The three major types of virtualization occur at the application, operating system, and architecture levels:
Application: At the application level, virtualization provides a way to run an application inside a guest operating system that runs on top of a host operating system. It requires large memory and high-speed processors to ensure the memory level translation are fast enough to minimize latency when using the application inside a guest operating system.
Operating System: The operating system virtualization allows the developers to create containers to minimize (and almost eliminate) application dependencies when moving the application from one environment to another.
Architecture: The architecture level virtualization emulates guest instruction set architecture (ISA) on top of a host ISA, thus providing developers hardware features to test application behaviors on different architectures using one single target platform.
The above three virtualization techniques are post-silicon (majorly software) activities, which do not provide virtualization for silicon at the transistor level.
Wafer Virtualization Enables Pre-Silicon To Post-Silicon Die Level Semiconductor Design And Manufacturing Process Modelling And Simulation On Virtual Wafers.
Eventually, software/hardware virtualization will require a piece of silicon to run on. The development time and the cost involved to create the perfect semiconductor design and manufacturing recipes are way too high. To develop next-gen silicon-powered solutions also means a new approach to test and validate the technology-node and process (pre-silicon to the post-silicon) around it. On top of that, the silicon development technique should be affordable and drives faster development. The solution to the problem is wafer virtualization.
Wafer Virtualization: A virtualization technique that is driven by software modeling and simulation. It replicates pre-silicon to post-silicon activities at die level to create virtual wafers to validate end-to-end design and manufacturing process for the fast-paced technology-node development.
Wafer virtualization has numerous benefits. However, reducing cost and time-to-market are the two reasons for the semiconductor industry to invest more in the virtualized way of designing and manufacturing.
Cost: The cost of manufacturing using advanced technology-node is skyrocketing. The semiconductor companies often face challenges in opting for the next-gen technology-node and processes around it. No company wants to lose out on the benefits new technology-nodes have to offer. However, to fabricate samples for a new product using new technology-nodes, time and capital is required. While semiconductor manufacturing is a tightly controlled process, it often happens that there are device-level escapes that end up costing more when manufacturing proceeds without die/wafer level simulation. Process Design Kit is tightly controlled but does not offer a sneak peek at how the thousands of die on a wafer will behave.
Time-To-Market: The semiconductor market has stiff competition. The early launch advantage is the key. Semiconductor products utilizing the latest technology-nodes without prior validation often face challenges. More so, when the manufacturing flow requires masking to equipment level investment. Understanding the effect of a new technology-node or device/transistor changes on manufacturing is a time-consuming process. It is why a faster approach like wafer virtualization is required. It can help in capturing the effects of design changes on manufacturing. Eventually, it helps in reducing the time-to-market of new products.
The above two reasons make wafer virtualization a technique that should see large scale adoption. While there are already few companies that offer wafer virtualization, the solution is yet to get widely adopted for new product design utilizing new technology-node and devices/transistors.
Wafer virtualization will drive not only modeling and simulation of the design stages but will also enable virtual wafer fabrication. It can help with understanding how the process modulation (apart from electrical testing) impacts the design.
Wafer virtualization has the potential to reduce cost and time by enabling near-perfect data-driven design and manufacturing. It can provide insights to designers on how the circuit and internal components behave with the new technology-node or device/transistor changes. Wafer virtualization can also enable process level modulation that can drive iterative analysis of different design methodology.
However, wafer virtualization does have drawbacks while also providing several benefits:
Positive:
Early Look Ahead: Important advantage of wafer virtualization is the ability to provide early look data. This data can help in understanding the impact of the technology-node on design. It can also drive experiments to understand the effect of fabrication processes on the product design. Wafer virtualization can also help in understanding how the die and the full wafer behave to a different temperature to process modulation.
Multiple Iteration: Wafer virtualization provides numerous data analysis approach by tweaking devices or blocks within the product. The data gathered can provide insights on how the semiconductor product will behave due to minor/major process changes.
Low-Cost: The use of wafer virtualization guarantees that the capital and time invested, is utilized towards more cost-friendly analysis that can provide confidence to design and the manufacturing teams on product behavior. Companies can save time and thus can test/validate different products in the same given time.
New Methodology Testing: Designers often come up with novel approaches to design circuits. Wafer virtualization can allow innovative development to speed up the semiconductor product fabrication by quickly understanding the impact of design methodology on semiconductor fabrication.
Faster Iterative Process:Testing the same product with a different process is costly and time-consuming. Wafer virtualization enables the iterative approach to product development but from a die/wafer manufacturing point of view. Iteration on the design side is not easy. A middle path that strikes the balance of iterative design with manufacturing using wafer virtualization can aid the new way to design and develop.
Apart from above all the positive points about wafer virtualization, there are few negative/drawbacks of wafer virtualization.
Negative:
Validation Requires Silicon: Eventually, even after numerous unlimited number of simulation and modeling, the design needs to be fabricated into real silicon. Design is sent for fabrication only when the designers are satisfied with the data that wafer virtualization provides after all the variations and changes. It means a FAB and OSAT are eventually needed to bring the data into reality.
Modeling Errors: Wafer virtualization can go wrong if there are errors in the data due to incorrect analysis. Any type of modeling errors can be costly when the processes are locked and more so when the design files are already to FAB for fabrication. Another iteration will add to the cost.
Costly Data: Eventually, the data needs to get generated before it gets used for wafer virtualization. Some parts of the simulation leading to wafer virtualization can be via statistical methods. However, the transistor level functioning still requires validation with the real die/wafer-level data. It again requires investment in equipment and FABs that can provide the silicon sample data before the virtualization starts generating on its own.
FAB/OSAT Required: Wafer virtualization does not mean products being developed virtually for the customer use case. The final locked product will require FAB and OSAT to validate the virtualized wafer. Indeed, the gaps closed due to wafer virtualization can enable fewer errors than ever before.
Increase Development Cost: Investing in wafer virtualization also means added CapEx. For a FAB-LESS company, wafer virtualization can be key, but for an IDM it may mean added cost to the existing investment.
While there are both positive and negative aspects of wafer virtualization, in the long run, the benefits of wafer virtualization due to data provided that is very close to the manufacturing process is vital.
Given the numerous use cases of wafer virtualization in the end-to-end semiconductor design and manufacturing process, the impact it can have on semiconductor device development can push the boundaries of next-gen device/transistor design.
The impact of wafer virtualization use can be seen both on academia and the industry:
Academia: Researchers can implement modeling and simulation for wafer virtualization to drive the development of die/wafer-level solutions. It can provide data on how the design changes at the device/transistor level can impact the post-silicon process. Researchers can develop a new device/transistor design and see how it performs across different technology-node. The promising solutions can then be used by industry to enable next-gen semiconductor products.
Industry: Apart from balancing cost and time-to-market, wafer virtualization can drive internal research and development activities are geared towards the market a few years down the line. The semiconductor companies can benefit immensely by leveraging data that ensure robust More-Than-Moore solutions.
Wafer virtualization is a promising solution for an industry that requires products to meet strict standards. The requirements to meet from pre-silicon to post-silicon stage are vital too. With the face-paced technology-node development, wafer virtualization can provide a new way to design and manufacture semiconductor products.
The semiconductor product development requires many businesses (from materials to logistics) to work in harmony. The complex process of fabricating semiconductor devices has also given rise to the cost of development. To lower the expenditure and to survive the semiconductor business, companies have leveraged different business models.
The three types of semiconductor business model that define the design and the fabrication houses have been in use for several decades:
Pure-Play FAB: In Pure-Play FAB (fabrication) model, the semiconductor companies invest only in semiconductor fabrication plants to turn the design into silicon products. These Pure-Play FABs in the last few years have also started investing in OSATs to leverage the growing assembly and testing demand. Examples: TSMC and GlobalFoundries.
FAB-LESS: FAB-LESS model allows companies to focus purely on the design aspect of the semiconductor product. FAB-LESS companies either have sold all their semiconductor fabrication plants (AMD) or never had any. FAB-LESS model was a game-changer for the semiconductor industry. FAB-LESS model allowed emerging companies to make use of the semiconductor manufacturing capacity built by the Pure-Play FAB. Even today, the majority of the emerging semiconductor companies are FAB-LESS focusing on innovative solutions and designs. Examples: Qualcomm and AMD
Integrated Device Manufacturer (IDM): IDM is a mix of Pure-Play and FAB-LESS. It allows companies to own semiconductor fabrication and testing facilities that cater to an in-house developed product. In the rarest of the cases, the IDM works with outside vendors to allow in-house capacity usage. Examples: Intel, Samsung, and Texas Instruments.
The majority of the emerging semiconductor companies are FAB-LESS due to the high cost of semiconductor fabrication plants and the time it takes to breakeven. Some Pure-Play FAB companies are also getting into the semiconductor design, but not at the scale at which FAB-LESS companies operate. IDMs have enjoyed the best of both worlds. However, the high CapEx to build/run the MEGA/GIGA FAB is now pushing IDMs to innovate their business model.
FAB-LITE Model Provides Low-Cost Solution For Semiconductor Dependent Companies Without Foundry To Enter Semiconductor Manufacturing
The semiconductor chip shortage in the automotive industry is another game-changing event for the semiconductor industry. It is driving automotive companies to think hard about the growing need for semiconductor products in the automotive solutions and how dependent the future EV (or Alternate-Fuel) market driven by high-tech semiconductor solution is going to be.
While there is a large number of FAB-LESS companies that can cater to the automotive industry to design innovative automotive semiconductor products, on the other hand, the capacity to cater to such high demand is not. The majority of the semiconductor manufacturing capacity available today is for advanced technology-node. Given that the smartphone to server market also requires advanced technology-node, the process to reserve the resources is becoming difficult for an industry like automotive. On top of that, the new advanced technology-node facilities under development will take many years to develop, and there on will also take months and years to run at the full capacity.
All of this is requires the need to re-invent the semiconductor business model that caters specifically to the need for an industry like automotive that is facing an uphill battle to reserve semiconductor fabrication capacity against the top computing OEMs.
What is the solution for an industry that requires semiconductor fabrication but is facing capacity constraints? The answer to this question may very well lie in the FAB-LITE model.
FAB-LITE:
– A semiconductor manufacturing model that enables in-house semiconductor manufacturing that caters to specific low-cost higher technology-node that are still in high demand in industries like aerospace, automotive, shipping, defense. etc.
– FAB-LITE model differs from the other three semiconductor business models as it purely focuses on cost optimization, either by building FABs by the companies that never ventured into semiconductor manufacturing but are heavily dependent on semiconductors to meet day-to-day production, OR by acquiring older FABs for in-house semiconductor need. For example, Ford may want to buy/build a semiconductor manufacturing facility that caters to its automotive needs.
FAB-LITE is not geared towards advanced technology-node but can be upgraded as the market moves from higher to lower technology-node. In automotive, the products can be in use for decades. The majority of these active semiconductor chips, do not use today’s latest and greatest technology-node, but instead are dependent on yesterday’s older technology-node.
FAB-LITE model is perfect for an industry that never owned semiconductor manufacturing but has always relied on semiconductor fabrication. It allows lost-cost technology-node operations to ensure they never run out of semiconductor capacity and chips.
The FAB-LITE model need is because the high-tech industry is becoming increasingly dependent on semiconductor solutions.
The vital piece to ensure the unlimited supply of semiconductor products is the fabrication, without which, the end-product of all the high-tech industry will face constraints like the automotive and consumer industry is facing in 2021.
Following are the major factors that drive the need for a FAB-LITE mode:
Cost: Continues improvement enables making the FAB-LITE model more cost-friendly. It can be either by sharing the resource with the competitor in the same industry with strict confidentiality.
Control: Owning a semiconductor fabrication that is FAB-LITE allows stricter control over the quality and reliability of semiconductor products being products. It allows prioritizing products that eliminate product launch delays.
Quality: Having an in-house fab that is FAB-LITE also ensures that the quality of the product can be strictly controlled. Thus enabling better products that meet the requirements of the end product. Such control is very crucial for an industry like automotive.
Shortage: Companies that are not into semiconductors but into the manufacturing of products that require semiconductors at a large scale cannot afford the shortage. The automotive industry today is one such example. With growing electronics in automotive solutions. the need for semiconductors will keep growing. FAB-LITE’s low-cost approach eliminates the shortage of in-house production needs.
Dependency:FAB-LITE business model is all about eliminating the dependency on large Pure-Play FAB. The majority of the Pure-Play FABs provide advanced technology-node. Many semiconductor driven products in the market still rely on older and high technology-node (automotive, defense, aerospace, etc.). To serve such solutions, the FAB-LITE model is ideal, given it focuses on technology-node that are in high demand and are less costly to develop (infrastructure to process lines) compared to advanced technology-node.
FAB-LITE is not limited to non-semiconductor companies. Even FAB-LESS can take the help of the FAB-LITE model to ensure they never become 100% dependent on their semiconductor manufacturing needs. FAB-LITE is also suitable for emerging markets like India, where the semiconductor fabrication is limited to infrastructure (like space and defense) of national interests only.
The above benefits play right into the hands of the semiconductor dependent companies that can invest today for the future semiconductor manufacturing needs by lowering the cost of the processes being used.
Like the other three actively used semiconductor foundry models, FAB-LITE also has many benefits. These benefits balance the best of the need and the cost to develop the semiconductor products in-house.
Cycle Time: FAB-LITE enables in-house production that ensures no delay in products as there are no conflicts with the outside competition. FAB-LITE keeps the product development cycle time in line with the companies expectation and execution plan. If any impact ever occurs, then the capacity can be quickly increased to minimize future constraints.
Low CAPEX: FAB-LITE focuses on older and higher technology-node (> 14nm). These require less capital to run compared to the advanced technology-node. There are already several old FABs that can be acquired to drive in-house requirements. These FABs later can be upgraded as the market requirement changes. Such a solution is perfect for automotive and defense solutions.
Supply Chain: In-house FAB-LITE provides the option to control the inventory. It enables an effective supply-chain and lowers the expenditure of in-house fabrication can also aid in eliminating waste.
Global Competition:FAB-LITE model is for the world that will run everything on the semiconductor. Having a FAB in-house allows semiconductor dependent companies to strictly control product development in the market where any slip in product launch can amount to huge losses.
Process Development: FAB-LITE model drives in-house technology-node related research and development. It can also lead to the development of new low-cost processes that can further lower the capital expenditure. If tomorrow’s automotive market will be semiconductor driven (which it certainly will be), then companies need to adapt starting today.
High Demand And Capacity:FAB-LITE can easily cater to the high demand and capacity requirement of products that require semiconductor solutions. Smartphones and other products are shrinking in size, and the high-tech OEMs are reserving all the available advanced technology-node capacity. Reserving full FAB capacity impacts the older/higher technology-node required by other industries. By enabling the FAB-LITE in-house model, companies can balance the high demand for semiconductor solutions by strategically implementing the FAB-LITE model.
Given how hard the automotive industry (and many other industries dependent on semiconductors) got hit by the semiconductor chip shortage, it will be interesting to see if the learning from such chip shortage can drive in-house semiconductor fabrication via the FAB-LITE model.
Every software application eventually has to get executed on a hardware system. Whether the software application is running on a smartphone or a data center, the data processing request has to get decoded (binary instructions) before the hardware system can process the request successfully. This seamless exchange of processes between software and hardware forms the base for a computer system.
Software form factor, user interface, and the speed might have changed over the years. However, the need to have a processing unit that can execute all the software code has not. Over the last two decades, the de-facto processing unit – the Central Processing Unit (CPU) – has seen several semiconductor and computer architecture backed technological advancement, that has taken computing to the next level.
As the software layers (application, presentation, session, and transport) are becoming model-driven (more pro-active, than re-active), so is the need to process the unique data/compute/memory intensive requests at the hardware-level.
Traditional CPU earlier was designed to handle a single request at a time. Then, the computing world moved towards multi-CPU (multi-cores) to cater to the parallel computing demand. Today, the de-facto is a System-On-A-Chip (SoC) that packs the CPU, the Graphics Processing Unit (GPU), and other processing units to process the high-resolution, high-speed, highly-data intensively requests in the shortest possible time.
Semiconductor Powered XPU (X Processing Unit) Are More Application-Specific Than General-Purpose
The integrated system (mainly CPU + GPU on an SoC) has not been able to keep up with the computing world’s data processing demand. It has pushed the computer architects to design new types of processing units (apart from high-bandwidth memory, cache coherence, and smarter interconnect topology) that are more application-specific than general-purpose.
This race to come up with the new type of processing units has given rise to XPU:
X = Application Domain — Vision, Graphic, Neural, X-Reality, Video, Audio, and so on
P = Processing
U = Unit
XPU is different than CPU and GPU as it caters to the specific needs of the application. XPU is more application-specific, and it can work standalone or as a co-processor/co-unit alongside the CPU and GPU. XPU is geared towards throughput and speedy data management that takes the best out of the CPU and GPU design methodology to enable application-specific needs. XPU is not only Application-Specific Integrated Circuit (ASIC) due to the workload it is designed to cater to, but can also be classified asApplication-Specific Standard Product (ASSP).
The rise of XPU is enabling a new era in computing. The hardware and the semiconductor market are enjoying different challenges and solutions the XPU brings. Intel is betting big on it, and so is AMD. Apart from these two semiconductor giants, there are numerous innovative startups (and academic research) that have XPU powered solutions and are pushing the computing industry towards next-gen data processing.
Several types of XPU are available. It is vital to understand how these differ from each other apart from the two processing giants – the CPU and the GPU.
There are numerous XPU powered examples in the market. Many of these are still in a nascent stage and yet to prove the solution in the market. Given the growth and demand for new AI workloads, the XPU catalog will keep growing.
Below are the major semiconductor powered XPU:
AIPU – AI Processing Unit — AIPU is targeted towards an Artificial Intelligence solution and is mainly designed to cater to the Edge AI market. MediaTek Helio series of SoC is an example of AIPU integrated with CPU and GPU. Even RAIP – Real AIProcessing Unit (RAIPU), IPU – Intelligence Processing Unit or Image Processing Unit, EPU – Emotion Processing Unit – fall under IPU with the only difference being the change of name. The goal of AIPU, RAIPU, EPU, and IPU is the same – to process data to make a decision that is at-level or at-part with human intelligence.
APU – Accelerated Processing Unit — APU design requires fusing the CPU and GPU into a single die. AMD’s A-Series processor is a perfect example of an APU. APU is also capable of running a heterogeneous system by utilizing system-level architecture and software features.
AMPU – Analog Matrix Processing Unit — AMPU is designed to cater to the data training needs that are often large parameters that execute multiple matrix multiplications. AMPU custom feature to handle such parameters and matrix operations allows speedy training without relying on the external memory. MYTHIC’s Analog Matrix Processor is an example of AMPU.
BPU – Brain Processing Unit — BPU is envisioned to minim the human brain as it is. The processing unit is capable of performing multiple TOPS. Horizon’s Journey line of products is considered the first BPU ever and is designed in close collaboration with the Baidu Institute of Deep Learning. BPU may also form the base for Level 5 automation.
CPU – Central Processing Unit — CPU is the de-facto processing unit and is vital for general to specific purpose computing needs. Intel and AMD are leading the CPU innovation along with ARM. CPU is here to stay and will keep providing the much-needed multi-tasking capability for consumer applications. On top, Academia is still engaging in research to make cache and data pipeline more energy efficient.
DPU – Dataflow Processing Unit | Data Processing Unit | Data Parallel Unit — DPU focuses on speeding up the data movement between the cores and the memory. It requires a new interconnect topology apart from the smart placement of sub-blocks to minimize bottlenecks. The instruction set allows for faster memory and compute-intensive processing. Intel already several RISC (AVX512) instructions set to optimize processing for x64. DPU takes it to the next level with its highly optimized data pipeline, which enables massive parallelism. Fungible’s DPU is one such example apart from Deep Vision’s ARA series. RDPU – Reconfigurable Dataflow Processing Unit – is also a type of DPU.
DLPU – Deep Learning Processing Unit — A DLPU finds similarity with DPU. It is a domain-specific solution to enable faster training. DLPU finds use in Edge AI and similar applications. Researchers first showcase DLPU with DianNao paper, and recently in Cambricon-F paper.
GPU – Graphics Processing Unit – Like CPU, GPU has been in the market for a long time and is designed to cater to graphics applications. In the last few years, GPU has also found its way into AI/ML/DL applications too. Highly parallel design (with multiple core and large memory) of GPUs enables faster computation. GPU’s efficient programmability also allows faster training of dataset. NVIDIA by far is the leader in the GPU domain along with AMD, which is not far behind. Depending on how the GPU is fabricated in the computer system, it also gets classified as = DGPU – Discrete Graphic Processing Unit, GPGPU – General Purpose Graphic Processing Unit, EGPU – External Graphic Processing Unit, IGPU – Integrated Graphic Processing Unit.
HPU – Holographic Processing Unit — Coming out of Microsoft, HPU is designed for X-Reality. It incorporates design to process rich information that is generated by the sensors and cameras on the X-Reality device. HPU incorporates processing units to implement DNNs.
MCPU – Micro Controller Processing Unit — MCPU is not used to run operating systems or frameworks, but instead are geared to run Real-Time Operating System (RTOS) powered solution. MCPU find use in automotive, remote devices and even laptops desktop to offload non-critical tasks. ARM, Texas Instruments, and others have been providing MCPU solutions for a few decades. In the AI-powered world, the solution around MCPU is increasing, and architecture design is adapting to it.
NPU – Neural Processing Unit — NPU implements all the required blocks to enable faster data modeling using neural networks. Alibaba’s Ali-NPU is one such example. There are different types of NPU: NDPU – Neural Decision Processing Unit, NNPU – Neural Network Processing Unit, NDPU – Neural Decision Processing Unit. Eventually, the end-goal of the types of NPUs is the same – train the data faster using a neural network and framework.
QPU – Quantum Processing Unit — QPU is a computational unit that uses quantum computing principles to perform a task. The physics used in QPU is drastically different than the general-purpose CPU. D-Wave Systems is the leader in QPU, and their QPU follows superconducting computing. Google, IBM, and Intel have QPU based on a quantum circuit. QPU is massive and not designed for mass-market and are supposed to compete against AIPU.
SPU – Streaming Processing Unit — SPU is useful to process streamed data. The structured data enables placing the cores and memory to minimize the delay in bringing the new data to process. Google’s TSPU – Tensor Streaming Processing Unit – is an example of SPU. There are different types of SPU available in the market apart from TSPU: GSPU – Graph Streaming Processing Unit, TMPU – Texture Mapping Processing Unit, TPU – Tensor Processing Unit
VPU – Vision/Visual/Video Processing Unit — VPU is coming out in the market due to the demand for providing a rich visual experience. VPU consists of more video encoding and decoding units to enabled faster 3D processing. X-Reality is an application area where VPU finds use. Intel’s Movidius VPU is a perfect example of how to use processing units to process video applications with low-latency.
XPU is unique and solves niche problems. The challenges that come with it are many. Below are the hurdles and challenges that the semiconductor powered XPU face:
Cost: Designing a new processing unit not only enables new features to run the workload in the most optimized way possible. But it also adds the cost of design, development, and manufacturing. Balancing the CapEx is always the de-facto goal of any organization, and in the case of XPU, the stakes are higher given the stiff market competition. Companies looking to come out with more XPU based designs and solutions will have to make the process cost-effective to breakeven faster.
Features: XPU is feature-specific. Deciding which problem the XPU is going to solve is difficult to find. The semiconductor industry has launched different types of XPU that cater to almost all the possible computing domains and problems possible. Upcoming XPU will have to beat the existing XPU not only from the design point of view but also with respect to the features that make the new XPU sustainable in terms of power, performance, area, and cost.
Application: Defining use cases and features for XPU is another challenge. It requires figuring out the bottleneck in the existing applications/workloads and then designing the solution at the silicon level to solve it. TSU is one such example, which removes the bottleneck to train the data. Such a unique feature-based XPU is going to make it stand out in the market.
XPU Will Drive Innovation Along With General-Purpose Processing Units
Manufacturing: XPU either needs to be fabricated along with the CPU or as another block with the CPU inside the SoC or as a separate co-processing unit. In any of the three cases, it takes a unique semiconductor manufacturing process to ensure there is no process, quality, package, and reliability issues. Multi-Die Multi-Chip powered chiplets can be one way to ensure future XPU designs for manufacturability. The technology node and the packaging technology needs to be thoroughly tested before making the XPU with it. The goal of reducing the cost of manufacturing for XPU will be another challenge.
Programmability: Hardware is designed to run the software. Developing and running applications on any piece of silicon (mainly those designed to run operating systems and frameworks) requires the support of system-level libraries to ensure the data being processed can make use of all the internal features. Semiconductor companies developing different types of XPUs will have to provide APIs and system-level hardware interface libraries. This requires time and cost to develop. To keep developers engaged dedicated continuous API development teams are required. Providing such software features is vital and also a challenge to the XPU design team.
Research And Development: Continuous R&D is a vital factor that leads to a new type of XPU. It is critical to ensure that the R&D team can collaborate with academia to innovate new processing blocks. Organizing a dedicated top-notch R&D team is still a challenge given the competition in the semiconductor industry. Investment is another differentiating criteria when it comes to advancing new XPU designs.
It is an exciting time to be in the XPU domain. General-purpose CPU and GPU are here to stay, and along with XPU will enable a new, powerful and efficient way to solve the data problems.
However, the need to innovate in the competitive semiconductor industry will be the race to watch out. Companies like Intel, Apple, AMD, Microsoft, Google, Amazon, and Facebook will play a crucial role in pushing the market for semiconductor powered XPU solutions, alongside promising FAB-LESS startups worldwide.
The automotive industry is going through a significant transformation. Countries all over the world are pressing for greener and eco-friendly vehicles. Government policies are getting stricter and demand vehicular technologies that require smarter software and hardware.
The de-facto transformation is towards electric vehicles. However, other vehicular technologies (hybrid, autonomous, and alternate-fuel) are also driving the change in the automotive industry, and all this is pushing automotive companies to innovate. To keep up with the smart and safe automotive features, the share of software and hardware in automotive is increasing tremendously, thus leading to new automotive technology companies are also emerging. The emerging automotive companies work in collaboration with the established automotive companies or are launching their own smarter electric, hybrid, autonomous, and alternate-fuelautomotive products.
The Semiconductor Industry And Products Are Key To Developing Smarter Hardware To Drive The Automotive Industry Using Smarter Software
The need for semiconductor solutions in automotive is also pushing the innovation and development of smarter automotive chips to drive vehicles safely. The increasing need to develop error-prone software that runs on efficient hardware is vital and is driving the automotive industry to re-invent automotive solutions from a semiconductor point of view.
It is not that semiconductor products started getting used by the automotive industry in 2020. The automotive industry has relied on semiconductor products for decades. From airbags to infotainment and many other solutions have always required defect-free semiconductor products. However, the landscape of semiconductors in automotive is changing from individual silicon components to more centralized silicon systems. The semiconductor products work excellent for solo operations.
However, when the goal is to connect the system for level 5 autonomy or increasing hybrid efficiency, a centralized automotive-specific system-on-a-chip (SoC) is required. The centralized automotive semiconductor-based solution still relies on individual semiconductor components and is leading to innovative work from both the established automotive firms and the new semiconductor FAB-LESS companies looking to eat into the automotive and semiconductor market.
A centralized siliconsystem needs to cater to the following uses cases for autonomous automotive solution:
Capture Information: Ability to capture information with the help of monitoring sensors, LIDAR, and RADAR
Process Data: Information captured should is processed without delay
Take Decision: The processed data is used to take accurate actions
Autonomy: Decide on behalf of the driver to enable a safer experience
Management: Track electrical and mechanical activities to provide vehicle health
Infotainment: Display, audio, and video system for entertainment
Connectivity: On-the-go navigation, Bluetooth, 5G, and WiFi connectivity
Safety: Ensure critical components and features are working and alert when maintenance is required
The above use cases of semiconductors are valid for any kind of two/three/four automotive solution, including passenger, commercial, motorcycles, and industrial automobiles.
Developing semiconductor solutions for automotive requires strong collaboration with the semiconductor design and manufacturing industry. In some cases, the semiconductor solutions are also being developed in-house by automotive companies. It is why the automakers are investing or building R&D facilities to come up with a high precision silicon need for future vehicles.
It is critical to make use of a high-level intelligence system to make automotive products smarter. The growing share of technology-powered features in modern vehicles is prompting the automotive industry to invest in software and hardware capabilities in-house. In many cases, automotive companies are also collaborating and investing by out-sourcing many of the vital semiconductor solutions from silicon chips to sensors systems.
Hyundai: Hyundai has a long-term plan to remove the dependency on automotive products. In line with its vision, Hyundai has launched a semiconductor lab to develop semiconductor products for electric/hybrid vehicle powertrain controllers.
There are numerous examples of big automotive giants already owning or investing in semiconductor chip development. Given the decreasing cost of developing an automotive solution using advanced intelligence techniques, several startups and newcomers are also shaking the automotive market by providing semiconductor solutions.
New comers in automotivesilicon chip:
Autotalks: A FAB-LESS startup providing semiconductor solutions for vehicle-to-everything (V2X) communications for the automotive industry.
Tesla: While Tesla is not new to the automotive market, its silicon solution is certainly is. It aims to provide a silicon sandbox that is going to make any vehicle an autonomous one. Tesla AI chip is still under in-house use, but will certainly open up the market if sold separately.
Waymo: Owned by Google, Waymo, for now, is using Intel’s technology but in-house is also planning to develop its silicon chip.
The list of newcomers and startups in automotive semiconductors is going to increase. It will be interesting to see how the semiconductors market adapts to the solutions from the smaller companies.
It is for sure going to increase the importance of semiconductor design and development. However, there are still major challenges to overcome before automotive companies can make it big in the semiconductor industry.
Automotive companies which have been in the industry can establish the semiconductor business unit to meet its need. The challenges arise for newcomers wanting to cater to the automotive industry.
Five major challenges might hinder the progress of newcomers from providing elegant automotive semiconductor solutions:
Cost: Even though the share of semiconductor cost in new vehicles is on the rise, the challenge remains on optimizing the development and manufacturing cost to reach the breakeven point. Optimizing the process to make a low-cost product for large scale consumption is still a challenge. On top of it, proving the smarter semiconductor driven solution to the market is getting tougher. Autonomous (not specifically self-driving) technologies take years to test on the road before they can be used by the mass market. All this adds to the cost.
Talent: Acquiring relevant software and hardware talent to bring silicon solutions to bring innovation to the market is another challenge. Automotive companies getting into self-driving, electric vehicles, and alternate-fuel solutions are doing all they can to form the best team. In some cases, the practice is not as per everyone’s liking. Bringing new talents on-board and training them takes years too. Challenges also remain with universities to launch programs that cater to the new-age automotive and semiconductor market that requires different skills and demands. An interdisciplinary study that combines mechanical, computer, electrical, and semiconductor engineering is the need of the hour.
Policies: The lives of many on the road are at stake, and Government policies play a crucial role in the automotive industry. Not all countries or states enable the framework to test autonomous solutions out on the road. Dedicated infrastructure is required to test solutions like lane guidance and autonomous driving. It is difficult for states to grant permission to test the solution on public roads due to dangerous un-known consequences. Only a few examples of how state policy can strike the balance of technological progress and road safety. Arizona is one such example. The policy has to lead Chalder in Arizona to become the hub for self-driving design and development. Waymo to Uber to Cruise are all testing their solutions out on the road. It shows the growing need to make policies that strike the balance of safety and future market needs.
The Semiconductor Automotive Product Development Demands Talent Pool Trained With Interdisciplinary Curriculum Covering Mechanical, Computer, Electrical And Semiconductor Engineering.
Reliability: Semiconductor automotive products have to go through stringent qualification, testing, and reliability criteria. Automotive Electronics Council (AEC) and a few other standards provide the guidelines for qualifying the semiconductor products. The guidelines require the temperature to reliability stress testing. All these require resources, time, cost, and talent to execute. A chip not functioning during a crash can lead to fatality. Making automotive semiconductor product reliable is a key concern.
Manufacturing: Automotive semiconductor products eventually have to get manufactured in the same arena where smartphone semiconductor products get manufactured. A semiconductor FAB and OSAT have to ensure strict control over the process to ensure zero variation between lots of the same product. Controlling such requirement demands strong industry flow that enables defect-free products. FAB-LESS companies developing semiconductor products for the automotive industry have to engage and invest with semiconductor manufacturing teams.
The challenges are many, but so are the opportunities. Companies have to increase focus on semiconductor products in the automotive industry at the chip level. The automotive industry has leverage semiconductor solutions for decades.
The next decade is going to be a game-changer. The share of semiconductor products and costs in automotive is only going to quadruple. All this will open tremendous opportunities both for the semiconductor and the automotive industry.
THE GROWING NEED FOR HETEROGENEITY IN SEMICONDUCTOR
The computing world today is all about processing data in real-time. Developers expect their code to compile in milliseconds. Consumers expect applications to respond with zero-delay. All of this requires a seamless communication of different computing components is a must, mainly the software (code) and the hardware (chip) is required.
The computing world pitched against the human brain. The ultimate goal is to outperform the human brain’s ability to sense, think, and act. While computers are outpacing humans, the desire for the silicon brain is still ongoing. To eventually mimic the human brain’s capabilities (mainly – sense, think, and act) demands much more computational speed and optimization than available today.
To reduce the time to run the compiled code on the chip has pushed both the software and the hardware (semiconductor) industry.
The software industry has been consistently coming up with unique ways to handle the data to avoid thrashing. Efficient use of parallel programming to split the single process/task into threads has been one major factor. Software developers (mainly frameworks and programming ones) have also been pro-actively utilizing all the hardware features (SVMS – Scalar, Vector, Matrix, Spatial) to enable a rich user experience by processing the data faster.
To Reduce Time To Completion While Balancing Performance-Per-Watt Is Pushing The Need For Heterogeneous SystemArchitecture
On another side, the hardware (semiconductor) industry is also innovating consistently (and trying to keep up with the software industry’s demand) to provide more performance-per-watt (PPW) that ensures the complex applications/workloads run efficiently. CPU/GPU/FPGA/ASIC design has seen not only architecture level innovations but also transistor-level. Shrinking transistor size made it possible to fabricate System-On-A-Chip (SoC) with billions/trillions of transistors in it. Transistor-level innovation has also allowed AI workloads to thrive. However, the SoC is hitting the design-wall and demands an innovative approach to cater to future workloads.
The SoC hitting the design-wall has pushed the semiconductor industry towards the Heterogeneous System Architecture (a.k.a. heterogeneous integration), which combines the best of the hardware capabilities to form a unique computing system. It allows the workloads to reduce time to completion while balancing the power-to-performance ratio.
Heterogeneous System Architecture requires unique semiconductor techniques that enable processing unit designs built using the best innovation out of the CPU, GPU, FPGA, and ASIC designs. It also drives the manufacturing process towards advanced technology nodes, packaging technology, and novel equipment.
THE SEMICONDUCTOR INNOVATION TO ENABLE HETEROGENEITY
Academia and the industry has been putting forward ways to design and manufacture architectures that can fit the demand for heterogeneous system architecture.
The heterogeneous system architecture can be classified into three categories:
Synchronous: Synchronous heterogeneous system architecture uses a single voltage, frequency, clock, and power domain for all of its processing units/cores. Multiple clusters with cores can exist, with each cluster designed using a unique data pipeline technique such that clusters are capable of operating at different speeds/frequencies. However, the processing units/cores within a cluster, always run under the same voltage to the power scheme. Apart from the CPU, the GPU (running on separate voltage to power domain) is the only other type of processing unit that is part of the synchronous heterogeneous architecture system. ARM big.LITTLE is one such example.
Asynchronous: Asynchronous heterogeneous system architecture borrows everything from the synchronous one, but it may also allow processing units/cores level voltage, frequency, clock, and power scaling. The helps in fusing cores/units on the same SoC that improves PPW. Qualcomm’s Snapdragon is one such example. However, the data pipeline of all the cores in the Snapdragon is the same. In reality, an asynchronous heterogeneous system architecture is not yet available as it demands innovative transistor-level techniques to drive per core level power domain apart from different core designs. There are thermal challenges too. Asynchronous heterogeneous system architecture often has another type of processing units/cores (FPGA, ASIC, GPU) apart from the CPU.
Fusion: In many architectures that form heterogeneous system architecture, per-core/unit level power management is not available. Instead, a fusion heterogeneous system architecture technique of combining different types (CPU, GPU, FPGA, ASIC, NPU, XPU, and so on) of processing units/cores are used. Each of these processing units may have separate power management. AMD’s Accelerated Processing Unit is an example of fusion without per processing unit/core power management. Fusion-based heterogeneous system architecture demand advanced technology nodes, packaging technology, and novel equipment
Whether one is designing synchronous or asynchronous or fusion heterogeneous system architecture, below are the five pillars of heterogeneous system architecture:
Technology Node: Integrating different processing cores/units to create a heterogeneous system architecture requires advanced technology nodes. The dies or the cores/units that get fused often have to get fabricated with the smallest possible transistor size. A true heterogeneous system architecture is supposed to make use of different technology nodes for the same integrated system. Example: A CPU inside a heterogeneous system architecture maybe 5nm, and the GPU might note be 5nm, and this pushes fabrication semiconductor companies to keep innovating on the transistor size and also on the device/transistor design/type (Planner FET, FinFET, GAAFET, MBCFET, etc.) Investing in Process Design Kit (PDK) and Electronic Design Automation (EDA) tools that can aid defect-free design is also required apart from developing fabrication facilities that can turn designs into silicon chips. All this puts pressure on the fabrication part of the semiconductor manufacturing process.
Interconnect: Faster data movement is key to enabling optimization on heterogeneous system architecture. Whether it is within the processing core/unit or in-between two or more, high-speed bandwidth is vital. Many are proposing silicon photonics-based interconnection that provides a high-speed interface. There are still open questions about the power requirements for silicon photonics-based solutions. It might be possible to make use of a photonics-based solution with electrical interconnects. Researchers have also proposed several solutions for heterogeneous silicon photonic interconnects. Eventually, continuous research and development is a must, as tomorrow’s heterogeneous system architecture will be highly complex than today’s.
Memory: Like the high-speed interconnect, high-bandwidth memory is also vital for heterogeneous integration to enable faster read/write for processing core-to-core (unit-to-unit). It may also act as a memory side cache. Intel’s MCDRAM is one such example. AMD also has hUMA that provides Heterogeneous Uniform Memory Access (hUMA) for fusion-based heterogeneous system architecture. Recently, Micron launched 176-Layer NAND, which delivers high performance and density. Similar techniques are required to enable faster input/output in heterogeneous system architecture.
Software: Efficiently scheduling tasks on a heterogeneous platform needs APIs, as it allows developers to map software on the target heterogeneous system architecture efficiently that allows access to internal functional units and drivers. Intel’s oneAPI provides exactly such an interface for its heterogeneous platform. Another approach is utilizing the Heterogeneous System Architecture Intermediate Language (HSAIL) which acts as an ISA for parallel compute routines. Software developers need to also make use of all the internal hardware features to drive the fastest time to completion.
Irrespective of the type of heterogeneous system architecture used – synchronous, asynchronous, or fusion – the above five components are crucial to take full advantage of the heterogeneous system architecture and its capabilities.
THE IMPACT OF HETEROGENEOUS SYSTEM ARCHITECTURE AND INTEGRATION
The impact of deploying heterogeneous system architecture is largely on the semiconductor manufacturing process due to the highly complex nature of fabrication, testing, and assembling the different types of sub-units using heterogeneous integration technology.
FAB: Semiconductor FABs have to always keep innovating new types of transistor devices, interconnects (TSV Interposers, etc.), and most importantly the advanced technology nodes. The demand and pressure to produce zero-defect products while the transistor size decreases is a challenge in itself. FABs like TSMC and Samsung have already started work on 3nm well before large scale production of 5nm, which is a challenging task. Lower technology nodes will enable highly complex silicon that is most likely be part of the heterogeneous system architecture. Fabricating 7/5/3 nm and beyond not only requires massive investment (upward of $10+ Billion) but demands continuous research and development too in close collaboration with academia. Heterogeneous integration is a vital market and is pushing the FABs to get into the packaging domain. TSMC already has a TSMC-SoIC solution for heterogeneous chiplets integration. Soon Samsung and others follow the suit.
Heterogeneous System Architecture Is Pushing Semiconductor Manufacturing To Innovate
EQUIPMENT: The equipment required to not only enable accurate testing but also assembling (Chiplets, Multi-Chip Multi-Die Modules, and SiP) without compromising on the specification puts pressure on the suppliers. The majority of the OSATs providing heterogeneous solutions will have to either upgrade their infrastructure or invest in new facilities. It is directly pushing semiconductor equipment providers to come up with new solutions.
COST: Eventually, aligning FAB to OSAT to equipment for heterogeneous integration requires CapEx. The added cost to design, fabricate, test, and assemble will increase the cost of development. It might directly affect the cost of goods sold, and semiconductor companies will have to come up with new techniques for viable product development to breakeven.
YIELD: All of the above factors eventually impacts the yield. The more complex the product is, the difficult it is to keep the yield high. Maintaining a high yield becomes a challenge due to the new way to test the system. This challenge is due to the complex fabrication and assembly process brings due to the integrated approach. It also means investing in new test hardware, probe cards, and automated test machines to handle heterogeneous testing.
The heterogeneous system architecture is pushing the manufacturing and the design semiconductor industry to new possibilities. It will be crucial to see how both the FAB and OSAT innovate and work in close collaboration with EDA and FAB-LESS/IDM houses to drive the era of heterogeneous integration.
Multi-Core Processor (MCP) or Chip Multi-Processor (CMP) revolutionized the computing industry. MCP/CMP came up with advanced execution and parallelism techniques. Software took the opportunity provided by the multiple processors fused into the single System-On-A-Chip (SoC).
MCP also provided the advantage of Out-of-Order Execution (OoOE), instructions-level parallelism (ILP), thread-level parallelism (TLP), and interleaved Simultaneous Multithreading (SMT), and allowed multiple applications to run on the same processor or multiple cores in the same SoC. Soon, the Single Instruction Stream, Single Data Stream (SISD) evolved into Multiple Instruction Streams, Multiple Data Streams (MIMD). MIMD gave a new experience to the data-intensive applications in the post-internet era.
The semiconductor and computing industry took advantage of MCP for over a decade by incorporating different core/processing units into a single SoC. Multi-Processor/Core System-On-A-Chip (MPSoC/MCSoC) became the heart of the new data and memory-intensive application. MPSoC/MCSoC starting to come up with dedicated processing blocks for data related to the graphic (GPU), digital (DSP), vector/vision (VPU), neural (NPU), and High-Bandwidth Memory (HBM).
The Artificial Intelligence System-On-A-Chip (AISoC) Is The Need Of The Future AI-Driven Workloads And Applications
The software computing industry is demanding data be processed faster than ever from semiconductor chips. Shrinking the transistor size further is not allowing the data and memory-intensive AI/ML/DL workloads to make the best of the MPSoC/MCSoC. Even though there are many opportunities to improve and innovate by proposing smarter data management techniques (cache, memory, and threading), MPSoC/MCSoC seems to have hit the memory wall, area wall, power wall, thermal wall, and performance wall. The data centers that should be shrinking in size and space due to the technology node advancement are instead becoming large by churning out massively distributed systems with a large number of MPSoC/MCSoC connected with large memory (NUMA).
The data-intensive, compute-intensive and memory-intensive artificial intelligence applications/workloads demand SoC that is:
Low Cost:
Affordable to manufacture
Efficient:
Improved performance-per-watt (PPW)
High Parallelism:
Massively parallel execution without stalling
Smart:
Ability to generate/store/predict models on the go that reside closer to the cores
Zero Bottlenecks:
Processes the data without memory/interconnect bottlenecks with or without a co-processor
Adaptive Software:
Ability to get programmed with minimal high-level programming effort and adapts on the go
High-Speed Memory:
Provides a large amount of high bandwidth memory across different memory levels/hierarchies
Technology Node:
Works efficiently irrespective of the advanced technology nodes used
The above eight-point feature is what will pay the way for Artificial Intelligence System-On-A-Chip (AISoC). These AISoC will be critical for the next generation of advanced solutions that will find use in the growing autonomous world. AISoC can be used in all devices and not only in the data centers. AISoC can speed up the fast-changing automotive to the satellite industry.
To cater to the growing demand for the semiconductor SoC chips for Artificial Intelligence) and to also balance the complexity, cost, and time to market, the semiconductor industry has already started to move away from general-purpose cores to specialized cores.
While the semiconductor industry is not labeling these new SoC as AISoC, but the features offered are of the AISoC world. Not all the AISoC solution adhere strictly to the eight-point features discussed above, but the solutions offered by different semiconductor companies is a step in the right direction.
Leadership in AI semiconductor chips is vital. Countries across the world are competing to bring the best homegrown solution to establish the lead. Governments are also funding the semiconductor chip business with the hope of leading the semiconductor race and mainly the AI solution one.
Apart from governments, companies across the globe are also racing against each other. From software giants to hardware leaders, all companies are investing zillions of time and money to come up with AI semiconductor chips out in the market.
The Artificial Intelligence System-On-A-Chip (AISoC) development is happening in two parallel worlds:
Established companies building in-house AI semiconductor chips
Startups providing a new architectural solution to drive AI semiconductor chips market
Below is the snapshot of the world’s top established companies racing against time to bring AISoC not only for their consumption but also for the market:
Alibaba: Alibaba competes directly with e-commerce giants and mainly Amazon. It provides web services similar to Amazon Web Services. To cater to enterprise needs, Alibaba last year launched Hanguang 800 is capable of processing 78,563 images per second. Alibaba introduced XuanTie 910 in 2019 provides 40% more performance than reference ISA RISC-V. These two AISoC are only a handful of examples. Alibaba’s DAMO Academy is continuously innovating and is going to launch much more surprising products in the AISoC domain.
Amazon: Amazon caters to more than 200 million visitors per month. Every visitor provides Amazon business and also the data on his/her shopping behavior. To process and make use of such unique data and to also provide enterprises the efficient web services, Amazon has been investing in AI-driven chips for a long time. Amazon’s Inferentia is the first step towards conquering the server market that is AISoC powered. The growing Alexa line of products pushed Amazon to in-house AISoC development and the results are already been seen in form of smarter voice-assisted devices.
AMD: AMD is another established semiconductor company with AISoC products. With AMD Instinct and AMD EPYC line of products, AMD has been steadily growing its market share in AI-enabled devices. AMD is also making most of the semiconductor chiplets technology to bring more innovation at the silicon level. Its acquisition of Xilinx is only going to help bring more AISoC solutions to the market. AMD is not deep into the mobile space, but they can certainly take advantage of the growing gaming industry to compensate. AMD CDNA is also another breakthrough architecture design to speed up high-performance computing.
Apple: With the launch of M1, Apple has shown the world its next target is going to be more in-house Macbook and iPhone/iPad processors. M1 has a NPU that allows faster predictive actions for its users. Apple is also planning to launch X-Reality products, which will require elegant AISoC, for which Apple has already started the work.
ARM: ARM IP has been critical for the smartphone industry and has taken the lead in providing AI-powered chips for mobile and also the data centers. With TOP500 won by ARM-powered supercomputers, ARM is ready to come up with more AISoC solution. Smart homes, wearables, and smartphones will see a massive use of AI Chip that will be powered by ARM. With Apple going all-in for ARM processors, and it will also help ARM innovate on the AISoC front.
Infineon Technologies: Infineon Technologies is going big in the AI Chip domain. It has established an AI development center in Singapore and also has a series of MCU designed with AI in mind. Low-cost MCU running with AI capability is the perfect solution for portable smart devices like cameras, drones, and smart speakers. AISoC with inbuilt MCU is another avenue Infineon is capable of exploring.
Marvell Technology Group: Marvell has launched a series of ASIC-based accelerators to cater to the AI data demand. The custom ASIC solutions used high speed interconnects and innovative packaging to optimize performance and cost. On top, Marvell has a strong collaboration with TSMC to provide 5/7/14 nm AI ASIC that allows it to pitch a wide range of portfolio to the growing AISoC market.
Qualcomm: As mobile AI is growing, Qualcomm is taking advantage of it by providing On-Device AI accelerators. Qualcomm has also taken steps towards a cloud AI Chip solution. It launched Cloud AI 100 chips to showcase its new architecture design for AI and data centers. Stronghold on mobile business with already out AI chips, Qualcomm can spring a surprise and enable new data centers that are not only AI-enabled but are also low-power and efficient AISoC.
Samsung: Samsung has fingers in many pies. From the design of chips in-house to the capability of manufacturing chips for its products and the world. Like Qualcomm, Samsung has been pushing for an AI chip to enabled On-Device AI. It has also collaborated with Baidu to develop a server-class of AISoC. The advantage of owning a foundry allows Samsung to innovate end-to-end and will be vital to see if it goes in the data centers’ AISoC chip design and development.
Texas Instruments: Like NXP and Infineon, TI is also providing Edge AI chips that cater to the 5G market. TI’s manufacturing capability fueled with low power techniques is going to provide a way forward to the industry on how to innovate AISoC with low power consumption.
The above summary shows how established companies are innovating and launching AISoC. The cost to establish a FAB-LESS semiconductor startup has gone down. The advanced EDA tools provide the ability to test ideas in the shortest possible time. RISC-V open ISA is also helping innovate without investing in royalty based ISA.
All this has lead to an increase in the number of FAB-LESS semiconductor startups that are coming up with new semiconductor chip designs and solutions to cater to the AISoC market. These startups have already got traction and some are even collaborating with established companies to test the solutions.
Below is the list of some of the top startups coming up with silicon level technology to drive AISoC design:
AlphaICs: AlphaICs is focusing on Edge AI and has designed an AI Processor that finds application as both the mobile and the data center solution. AlphaICs call their AISoC as Real AI Processor (RAP)
Alphawave: Alphawave provides Digital Signal Processor (DSP) solutions that are suited for high-speed performance and are low on power consumption. DSP provides audio/video processing and with Alphawave’s AppolloCORE IP semiconductor companies can build AISoC with an onboard accelerator. Alphawave was also the winner of TSMC’s Awards for Excellence in Accelerating Silicon Innovation
EdgeQ: EdgeQ is taking a different approach to Edge and 5G by fusing both into a single AI-powered chip. This will massively off-load the tas from data centers to Edge Computing. With 5G rollout already in progress worldwide, the solution is at the right time for the right market.
Horizon Robotics:Journey and Sunrise processor architecture from Horizon Robotics is designed to provide an AI-enabled Brain Processing Unit (BPU). Journey BPU is designed for the automotive industry, while Sunrise is for the IoT market.
Both established companies and startups are showcasing the world’s new way to design chips and drive data processing. All this is making software development, training, testing, and data analytics faster. The AISoC from all these vendors is also providing avenues for low-cost AI-powered mobile and data centers.
The majority of the challenges the AISoC face are still the same old problems faced by general-purpose CPU and GPU as the technology at the silicon level advanced. The new AISoC solution from both the established companies and startups are eventually going to hit with these challenges.
Cost: Designing and establishing AISoC proof-of-concept using the software simulator demands resource and pushes the cost of development from FAB to OSAT. The cost of owning smartphones and running data centers is already high. On top of it, any new solution with AI-power will add cost to the customer. The technology node required to enable a high number of processing units to speed up the training and inference is eventually going to cost money. AISoC vendors need to balance the cost of manufacturing in order to breakeven the market. On top of all this, the amount of competition in developing new AISoC means time to market is vital than ever.
Bottleneck: The reason to move away from general-purpose CPU and GPU was memory and interconnect bottleneck. There are few startups listed above that are trying to remove these bottlenecks. However, with the speed with which new AI-workload are getting generated, there is a high chance that bottlenecks will still exist. It will be vital to ensure that the new type of AISoC that both the established companies and startups are envisioning does not have any bottlenecks.
Bandwidth: Bringing the data closer to the processing units (any type) is the key to processing AI data faster. However, for such a task high-speed memory with large bandwidth is required. The new AISoC are incorporating new processing units like RAP, GSP, TSP, BPU, AMP, RDU, NDP, and so on, but there is no clear strategy and details on how the data communication bandwidth is improved. May be such details are proprietary.
Programming: In the end, any AISoC cannot process the data efficiently if the workload is not optimized for the target architecture. While few AISoC is pitching their products as no need to change the data or framework before running it on their architecture, however, the reality is that every architecture ends up needing some or other form of optimization. All this adds to the time to develop data solutions.
Manufacturing: As the new AISoCs come out in the market, many of these will end up using advanced nodes beyond 7nm to provide high speed. Advanced packaging technology also is required to operate the AISoC within the thermal budget. Both the complex technology node and package technology will drive a high manufacturing cost. Apart from this, balancing yield and cost will be essential to ensure AISoC development is viable.
Power Consumption: AISoC requires zillions of transistors that require faster cooling. The majority of the AISoC can do with liquid cooling but when such AISoC is connected together to form data centers then the cost to run data centers goes high. Hopefully, greener technologies will be able to run such data centers. However, the AISoC will get challenged to overcome the area, power, and thermal wall.
No matter what, AISoC in coming years is going to be the semiconductor domain that will innovate and provide elegant semiconductor solutions that will challenge the end-to-end semiconductor design and manufacturing.
