#chetanpatil – Chetan Arvind Patil

Category: SEMICONDUCTOR

  • The Race For AI Semiconductor Chips

    The Race For AI Semiconductor Chips

    Photo by david latorre romero on Unsplash

    THE NEED FOR AI SEMICONDUCTOR CHIPS

    Multi-Core Processor (MCP) or Chip Multi-Processor (CMP) revolutionized the computing industry. MCP/CMP came up with advanced execution and parallelism techniques. Software took the opportunity provided by the multiple processors fused into the single System-On-A-Chip (SoC).

    MCP also provided the advantage of Out-of-Order Execution (OoOE), instructions-level parallelism (ILP), thread-level parallelism (TLP), and interleaved Simultaneous Multithreading (SMT), and allowed multiple applications to run on the same processor or multiple cores in the same SoC. Soon, the Single Instruction Stream, Single Data Stream (SISD) evolved into Multiple Instruction Streams, Multiple Data Streams (MIMD). MIMD gave a new experience to the data-intensive applications in the post-internet era.

    The semiconductor and computing industry took advantage of MCP for over a decade by incorporating different core/processing units into a single SoC. Multi-Processor/Core System-On-A-Chip (MPSoC/MCSoC) became the heart of the new data and memory-intensive application. MPSoC/MCSoC starting to come up with dedicated processing blocks for data related to the graphic (GPU), digital (DSP), vector/vision (VPU), neural (NPU), and High-Bandwidth Memory (HBM).

    The Artificial Intelligence System-On-A-Chip (AISoC) Is The Need Of The Future AI-Driven Workloads And Applications

    The software computing industry is demanding data be processed faster than ever from semiconductor chips. Shrinking the transistor size further is not allowing the data and memory-intensive AI/ML/DL workloads to make the best of the MPSoC/MCSoC. Even though there are many opportunities to improve and innovate by proposing smarter data management techniques (cache, memory, and threading), MPSoC/MCSoC seems to have hit the memory wall, area wall, power wall, thermal wall, and performance wall. The data centers that should be shrinking in size and space due to the technology node advancement are instead becoming large by churning out massively distributed systems with a large number of MPSoC/MCSoC connected with large memory (NUMA). 

    The data-intensive, compute-intensive and memory-intensive artificial intelligence applications/workloads demand SoC that is:

    • Low Cost:
      • Affordable to manufacture
    • Efficient:
      • Improved performance-per-watt (PPW)
    • High Parallelism:
      • Massively parallel execution without stalling
    • Smart:
      • Ability to generate/store/predict models on the go that reside closer to the cores
    • Zero Bottlenecks:
      • Processes the data without memory/interconnect bottlenecks with or without a co-processor
    • Adaptive Software:
      • Ability to get programmed with minimal high-level programming effort and adapts on the go
    • High-Speed Memory:
      • Provides a large amount of high bandwidth memory across different memory levels/hierarchies
    • Technology Node:
      • Works efficiently irrespective of the advanced technology nodes used

    The above eight-point feature is what will pay the way for Artificial Intelligence System-On-A-Chip (AISoC). These AISoC will be critical for the next generation of advanced solutions that will find use in the growing autonomous world. AISoC can be used in all devices and not only in the data centers. AISoC can speed up the fast-changing automotive to the satellite industry.

    To cater to the growing demand for the semiconductor SoC chips for Artificial Intelligence) and to also balance the complexity, cost, and time to market, the semiconductor industry has already started to move away from general-purpose cores to specialized cores.

    While the semiconductor industry is not labeling these new SoC as AISoC, but the features offered are of the AISoC world. Not all the AISoC solution adhere strictly to the eight-point features discussed above, but the solutions offered by different semiconductor companies is a step in the right direction.

    Picture By Chetan Arvind Patil

    THE STATUS OF AI SEMICONDUCTOR CHIPS RACE

    Leadership in AI semiconductor chips is vital. Countries across the world are competing to bring the best homegrown solution to establish the lead. Governments are also funding the semiconductor chip business with the hope of leading the semiconductor race and mainly the AI solution one.

    Apart from governments, companies across the globe are also racing against each other. From software giants to hardware leaders, all companies are investing zillions of time and money to come up with AI semiconductor chips out in the market.

    The Artificial Intelligence System-On-A-Chip (AISoC) development is happening in two parallel worlds:

    • Established companies building in-house AI semiconductor chips
    • Startups providing a new architectural solution to drive AI semiconductor chips market

    Below is the snapshot of the world’s top established companies racing against time to bring AISoC not only for their consumption but also for the market:

    Alibaba: Alibaba competes directly with e-commerce giants and mainly Amazon. It provides web services similar to Amazon Web Services. To cater to enterprise needs, Alibaba last year launched Hanguang 800 is capable of processing 78,563 images per second. Alibaba introduced XuanTie 910 in 2019 provides 40% more performance than reference ISA RISC-V. These two AISoC are only a handful of examples. Alibaba’s DAMO Academy is continuously innovating and is going to launch much more surprising products in the AISoC domain.

    Alphabet/Google: Alphabet’s Google arm has always been into hardware research and development. Google’s Brain and Hardware and Architecture team has been continuously providing solutions to make AI algorithms faster and smarter. Several AI-related hardware solutions have come out of Google. Google Cloud’s TPU is already becoming a benchmark for the AI industry. There are already many solutions that promise to improve the time to train networks using large data sets. Google is also pushing the envelope by taking the help of AI to design AI chips. With Pixel’s line of products, there is more room to innovate. In the coming years, Google will showcase innovative AISoC solutions.

    Amazon: Amazon caters to more than 200 million visitors per month. Every visitor provides Amazon business and also the data on his/her shopping behavior. To process and make use of such unique data and to also provide enterprises the efficient web services, Amazon has been investing in AI-driven chips for a long time. Amazon’s Inferentia is the first step towards conquering the server market that is AISoC powered. The growing Alexa line of products pushed Amazon to in-house AISoC development and the results are already been seen in form of smarter voice-assisted devices.

    AMD: AMD is another established semiconductor company with AISoC products. With AMD Instinct and AMD EPYC line of products, AMD has been steadily growing its market share in AI-enabled devices. AMD is also making most of the semiconductor chiplets technology to bring more innovation at the silicon level. Its acquisition of Xilinx is only going to help bring more AISoC solutions to the market. AMD is not deep into the mobile space, but they can certainly take advantage of the growing gaming industry to compensate. AMD CDNA is also another breakthrough architecture design to speed up high-performance computing.

    Apple: With the launch of M1, Apple has shown the world its next target is going to be more in-house Macbook and iPhone/iPad processors. M1 has a NPU that allows faster predictive actions for its users. Apple is also planning to launch X-Reality products, which will require elegant AISoC, for which Apple has already started the work.

    ARM: ARM IP has been critical for the smartphone industry and has taken the lead in providing AI-powered chips for mobile and also the data centers. With TOP500 won by ARM-powered supercomputers, ARM is ready to come up with more AISoC solution. Smart homes, wearables, and smartphones will see a massive use of AI Chip that will be powered by ARM. With Apple going all-in for ARM processors, and it will also help ARM innovate on the AISoC front.

    Baidu: Baidu is a giant in China and competes worldwide with Google, Amazon, and Alibaba. Baidu showcased the Kunlun AI processor during HotChips 2020 that is designed and produced in collaboration with Samsung Electronics. Kunlun is capable of catering to diverse AI workloads and claims to have three times more performance than NVIDIA AISoCs. It will be interesting to see how Baidu goes all in-house with AISoC designs.

    Facebook: There are 2.7 billion people that used Facebook every month. To serve the growing requests, Facebook has been developing in-house silicon that takes advantage of AI to provide faster training. Zion, Kings Canyon, and Mount Shasta are three major AISoC that Facebook has innovated to run its hardware infrastructure efficiently. It has ramped up its effort to develop more in-house AISoC, and the results will be out in the coming years.

    Huawei/HiSilicon: A subsidiary of Huawei, HiSilicion has been innovating fast to cater not only to the AI smartphone and data center market in China but also in the majority of the developing nations. The Kirin and Ascend line of products has done wonders for Huawei devices. Huawei has also launched AISoC for data centers. It will be vital to see how Huawei and HiSilicon innovate in the next few years and expand their AISoC portfolio.

    IBM: IBM has been a quiet leader in smart technologies. Watson has done wonders for the AI industry and also pushed other companies to innovate faster. IBM has innovated to accelerate DNN training with the help of CMOS and new AI Cores. IBM has been focusing on Analog and Digital AI cores that enable dynamic and hybrid cloud systems. IBM is one of the few companies that not only provide AISoC based solutions but also innovates at the transistor level. The combination of two allows it to provide more efficient AI solutions than others.

    Imagination Technologies: Imagination Technologies has also ramped up its AI chip efforts. Recently, it launched a new AI-powered BXT series of chips for data centers. PowerVR backed line of products have helped Imagination Technologies establish its foot in the vision processing domain. PowerVR solution combined with Neural Network Accelerators (NNA) is unleashing new ways to process vision data and will also enable new AISoC.

    Intel Corporation: Intel is a leader in the server and data center SoC market. Even though it is getting stiff competition from other vendors, Intel has been able to provide the industry with breakthrough AI chips. Even though the Nirvana series of AI chips did not work out as planned, it has big planes with Habana’s line of products. The manufacturing capability of Intel allows it to ensure that there is always a new way to design and manufacture AISoC. Intel Xeon’s line of products has also shown how the AI world how smaller SoCs are capable of running workloads on high bandwidth memory. With shrinking transistor size and Intel’s plan to move beyond 7nm, there will be elegant AISoC coming out.

    Infineon Technologies: Infineon Technologies is going big in the AI Chip domain. It has established an AI development center in Singapore and also has a series of MCU designed with AI in mind. Low-cost MCU running with AI capability is the perfect solution for portable smart devices like cameras, drones, and smart speakers. AISoC with inbuilt MCU is another avenue Infineon is capable of exploring.

    Marvell Technology Group: Marvell has launched a series of ASIC-based accelerators to cater to the AI data demand. The custom ASIC solutions used high speed interconnects and innovative packaging to optimize performance and cost. On top, Marvell has a strong collaboration with TSMC to provide 5/7/14 nm AI ASIC that allows it to pitch a wide range of portfolio to the growing AISoC market. 

    MediaTek: MediaTek’s Helio line of AI chips for edge computing on the go. It is also planning to use the solution for the 5G market. Apart from the hardware products, MediTek also provides hardware-oriented design solutions like NeuroPilot to make the most of its AISoC with AI Processing Unit (APU).

    Microsoft: Microsoft hardware division has provided many AISoC solutions to the market. Project Brainway is another such solution that allows the use of FPGA and ASIC to speed up the training. Microsoft also has plans to develop a tiny AI chip in collaboration with Sony. It may very well pave a new way for nanoelectronics well beyond what is available now.

    NVIDIA: GPUs have single-handedly accelerated the growth of AI research. NVIDIA has been one of the leaders that showcased how to train the data set faster using GPU architecture. Apart from catering to the data centers, NVIDIA also provides AI-enabled SoCs for smart cars. A few months back, NVIDIA also unveiled cost-efficient A100 architecture for the industry. With ARM’s acquisition, NVIDIA is on track to bring AI to the low power AISoC soon.

    NXP Semiconductors: NXP has several MCUs and MPUs optimized for AI applications and targeted for the automotive and smart industry. NXP’s i.MX series provides ML and DL optimized solutions. With growing semiconductor cost in automotive, NXP is stand to get the advantage with is a wide range of AI-enabled AISoC chip solutions.

    Qualcomm: As mobile AI is growing, Qualcomm is taking advantage of it by providing On-Device AI accelerators. Qualcomm has also taken steps towards a cloud AI Chip solution. It launched Cloud AI 100 chips to showcase its new architecture design for AI and data centers. Stronghold on mobile business with already out AI chips, Qualcomm can spring a surprise and enable new data centers that are not only AI-enabled but are also low-power and efficient AISoC.

    Samsung: Samsung has fingers in many pies. From the design of chips in-house to the capability of manufacturing chips for its products and the world. Like Qualcomm, Samsung has been pushing for an AI chip to enabled On-Device AI. It has also collaborated with Baidu to develop a server-class of AISoC. The advantage of owning a foundry allows Samsung to innovate end-to-end and will be vital to see if it goes in the data centers’ AISoC chip design and development.

    Tesla: Tesla already has the smartest AI-enabled cars out in the market. It has already designed in-house an AI chip to cater to Tesla’s growing need to provide more advanced and safe autonomous car driving solutions. Rumours says that Tesla would do away with cars and focus on an AISoC solution that can make any vehicle an autonomous one. Whether or not it will end up happening, Tesla’s AISoC will push the innovation around the self-driving car.

    Texas Instruments: Like NXP and Infineon, TI is also providing Edge AI chips that cater to the 5G market. TI’s manufacturing capability fueled with low power techniques is going to provide a way forward to the industry on how to innovate AISoC with low power consumption.

    The above summary shows how established companies are innovating and launching AISoC. The cost to establish a FAB-LESS semiconductor startup has gone down. The advanced EDA tools provide the ability to test ideas in the shortest possible time. RISC-V open ISA is also helping innovate without investing in royalty based ISA. 

    All this has lead to an increase in the number of FAB-LESS semiconductor startups that are coming up with new semiconductor chip designs and solutions to cater to the AISoC market. These startups have already got traction and some are even collaborating with established companies to test the solutions. 

    Below is the list of some of the top startups coming up with silicon level technology to drive AISoC design:

    AlphaICs: AlphaICs is focusing on Edge AI and has designed an AI Processor that finds application as both the mobile and the data center solution. AlphaICs call their AISoC as Real AI Processor (RAP)

    Alphawave: Alphawave provides Digital Signal Processor (DSP) solutions that are suited for high-speed performance and are low on power consumption. DSP provides audio/video processing and with Alphawave’s AppolloCORE IP semiconductor companies can build AISoC with an onboard accelerator. Alphawave was also the winner of TSMC’s Awards for Excellence in Accelerating Silicon Innovation

    Blaize: Blaize is another startup providing Edge AI solution that is built for AI workload. Blaize’s Graph Streaming Processor (GSP) is a power-efficient and adaptable core that caters to AI, ML and DL need on the fly.

    Cambricon Technologies: Cambricon used to provide processors to Huawei before it began its own in-house silicon design house HiSilicon. Since then, Cambricon has developed several general markets AI products catering to mobile and cloud. Their Cloud AI platform provides flexibility and adaptability. With more than 100 million smartphones and servers powered by Cambricon, it is going to be vital for the AISoC world.

    Cerebras Systems: Cerebras uses Wafer-Scale Engine technology to deliver a supersonic deep learning experience. It is benchmarked to be 1000 times faster than a GPU. Cerebras unique interconnects, memory, and package technology is pitched to break many records in computing shortly.

    EdgeQ: EdgeQ is taking a different approach to Edge and 5G by fusing both into a single AI-powered chip. This will massively off-load the tas from data centers to Edge Computing. With 5G rollout already in progress worldwide, the solution is at the right time for the right market.

    GrAI Matter Labs: GrAI Matter is targeting robotics, X-Reality, and the drone market by providing Edge AI Processor. The solution provided by GeAI Matter has ultra-low latency and is low power, two features critical for Edge processing.

    Graphcore: Graphcore has accelerator products that cater to machine learning and artificial intelligence by leverage the proprietory Intelligence Processing Unit (IPU) technology.

    Groq: Groq leverages Tensor Streaming Processor (TPU) to provide small programming cores that are packed in a tiny package with high-speed memory and performance fast operations.

    Hailo: Hailo is one more startup focusing on Edge AI. Hailo claims its Hailo-8 Edge AI processor can provide 26 tera-operations per second (TOPS) without comprising the area and power efficiency.

    Horizon Robotics: Journey and Sunrise processor architecture from Horizon Robotics is designed to provide an AI-enabled Brain Processing Unit (BPU). Journey BPU is designed for the automotive industry, while Sunrise is for the IoT market.

    Kneron: Kneron provides Edge AI solution and plans to take on Google and others with its AI-enabled chip. Kneron claims its KL720 AI SoC has the highest performance to power ratio in the market.

    Lightelligence: Lightelligence is taking a photonics approach to solving AI processing problems. It has already released an optical AI accelerator but yet to see mass production for the market needs.

    Lightmatter: In the same domain as Lightelligence, Lightmatter also plans to use electronics, photonics, and algorithms to provide processor and interconnect that is faster and more efficient than traditional AISoC.

    Luminous Computing: Still in stealth mode, Luminous also plans to leverage photonics to speed up A workload training.

    Mythic AI: Mythic uses Intelligence Processing Units (IPUs) to provide power-efficient, performance-oriented, and cost-efficient AISoC. Mythic Analog Matrix Processor is already available to order and will find use in Edge AI.

    NUVIA: NUVIA is a stealth mode startup focusing on ARM-powered CPUs to drive AI workload. More details about its architecture are yet to be known.

    SambaNova Systems: SamaNova is another startup that uses Reconfigurable Dataflow Unit (RDU) to enable new models without going into the algorithm complexity. SambaNova’s Cardinal SN10 is designed to eliminate constant data caching and excess data movement, something the majority of the SoC today suffers from.

    SiMa.ai: SiMa wants to make greener low-power AISoC for Edge AI. It is yet to share product details. SiMa plans to launch new silicon early next year.

    SimpleMachines: To accelerate AI/ML/DL application performance, SimpleMachines leverages Composable Computing. Simple Machines AISoC solution enables flexible and powerful real-time computation.

    Synthara AG: Synthara leverages RISC-V ISA to provide ultra-low power ASIC for Edge AI.

    Syntiant: Syntiant provides an ultra-low power AI processing solution for any battery-powered device, from earbuds to laptops. Syntiant Neural Decision Processors™ (NDP10x) is a tiny silicon that is always-on.

    Tenstorrent: Tenstorent Grayskull AISoC fast AI interference to enable accurate and faster prediction on the go. It is expected to into production soon.

    Wave Computing: Wave wants to accelerate AI computing with the help of MIPS architecture. The M-Class product from Wave Computing provides AISoC using MIPS architecture for IoT and smart devices.

    Both established companies and startups are showcasing the world’s new way to design chips and drive data processing. All this is making software development, training, testing, and data analytics faster. The AISoC from all these vendors is also providing avenues for low-cost AI-powered mobile and data centers.

    However, there are several challenges ahead.

    Picture By Chetan Arvind Patil

    THE CHALLENGES AHEAD FOR AI SEMICONDUCTOR CHIPS

    The majority of the challenges the AISoC face are still the same old problems faced by general-purpose CPU and GPU as the technology at the silicon level advanced. The new AISoC solution from both the established companies and startups are eventually going to hit with these challenges.

    Cost: Designing and establishing AISoC proof-of-concept using the software simulator demands resource and pushes the cost of development from FAB to OSAT. The cost of owning smartphones and running data centers is already high. On top of it, any new solution with AI-power will add cost to the customer. The technology node required to enable a high number of processing units to speed up the training and inference is eventually going to cost money. AISoC vendors need to balance the cost of manufacturing in order to breakeven the market. On top of all this, the amount of competition in developing new AISoC means time to market is vital than ever.

    Bottleneck: The reason to move away from general-purpose CPU and GPU was memory and interconnect bottleneck. There are few startups listed above that are trying to remove these bottlenecks. However, with the speed with which new AI-workload are getting generated, there is a high chance that bottlenecks will still exist. It will be vital to ensure that the new type of AISoC that both the established companies and startups are envisioning does not have any bottlenecks.

    Bandwidth: Bringing the data closer to the processing units (any type) is the key to processing AI data faster. However, for such a task high-speed memory with large bandwidth is required. The new AISoC are incorporating new processing units like RAP, GSP, TSP, BPU, AMP, RDU, NDP, and so on, but there is no clear strategy and details on how the data communication bandwidth is improved. May be such details are proprietary. 

    Programming: In the end, any AISoC cannot process the data efficiently if the workload is not optimized for the target architecture. While few AISoC is pitching their products as no need to change the data or framework before running it on their architecture, however, the reality is that every architecture ends up needing some or other form of optimization. All this adds to the time to develop data solutions. 

    Manufacturing: As the new AISoCs come out in the market, many of these will end up using advanced nodes beyond 7nm to provide high speed. Advanced packaging technology also is required to operate the AISoC within the thermal budget. Both the complex technology node and package technology will drive a high manufacturing cost. Apart from this, balancing yield and cost will be essential to ensure AISoC development is viable.

    Power Consumption: AISoC requires zillions of transistors that require faster cooling. The majority of the AISoC can do with liquid cooling but when such AISoC is connected together to form data centers then the cost to run data centers goes high. Hopefully, greener technologies will be able to run such data centers. However, the AISoC will get challenged to overcome the area, power, and thermal wall.

    No matter what, AISoC in coming years is going to be the semiconductor domain that will innovate and provide elegant semiconductor solutions that will challenge the end-to-end semiconductor design and manufacturing.

  • The Need For Semiconductor As A Service

    The Need For Semiconductor As A Service

    Photo by Laura Ockel on Unsplash

    THE SEMICONDUCTOR AS A SERVICE

    The software industry has adapted to the demand of business and consumers by changing the licensing and product delivery model over the last three decades. The post-1990 saw standalone one-time fee-based software with no incremental feature updates except security-related and termed as the pay and use model. Then post-2000, with the proliferation of the internet, the software license model moved to pay over month/year and also came with features and security updates. The software industry termed it as Software-As-A-Service Model. Post-2010, the software industry adapted to the changing business and applied the licensing model from software to platform, which came not only with features and security updates for the software itself but also the platform the software will run on. It has allowed software developers to provide more over the top services.

    In comparison to the software industry, the hardware industry (mainly the semiconductor industry) has not adopted the product delivery model. It has been constant and driven by build and ship, with no ability to provide new hardware features on the go. If there are security flaws in the hardware, then those are suppressed by an Over-The-Air (OTA) update. Consumer and business buying the piece of silicon get locked in with the product. It is also not easy to provide new features at the silicon level. On top, the majority of the products shipped by the semiconductor industry end up getting used differently based on the hardware company’s need. 

    The semiconductor products (from CPUs to NPUs to GPUs to ASICs to FPGAs to DSPs to Mixed/Analog/Digital devices) have a long design and manufacturing cycle. It also means a long-term vision of the future market needs and then aligning the investment in the design to the manufacturing process accordingly. As per the market demand, semiconductor products need to be more adaptable with in-built features that are more relevant a few years down the line and can be activated post-production.

    Semiconductor-As-A-Service Is Possible Today Than Ever Due To The Shrinking Transistor Size That Allows More Silicon Features To Be Built-In Today For The Future Needs.

    The approximate life of a smartphone is anywhere between three to five years. However, the majority of companies stop providing critical software updates that make the smartphones redundant. The launch of new smartphones with new silicon and software grabs consumer’s attention and they end up buying a new smartphone with the latest silicon features.

    Imagine, having adaptable silicon with features built-in that can be unlocked a few years later and thus making the hardware as new as the software? Either vendors or consumers can decide which silicon features should be activated and how it helps the device performance. Such a process will allow the semiconductor industry to deliver silicon services under Semiconductor-As-A-Service model.

    Semiconductor-As-A-Service – A product delivery business model for the semiconductor industry which allows silicon design and manufacturing with in-built silicon features that can be unlocked in the future as the market demand and software requirements align. For example – More graphics for new gaming applications. These silicon features can be enabled with the help of software updates and require a subscription or one-time payment license. The list of features can be endless, from more cache memory to DRAM memory to extra processing cores to additional GPU for gaming applications to secondary cellular (perhaps 6G) antenna. The shrinking transistor size and growth of heterogeneous integration as a More-Than-Moore (MTM) solution makes such features in silicon possible. Silicon area with extra features can reside inside the smartphone launched in 2020 as an inbuilt hidden feature with the option to enable in 2022 as long as consumers are willing to pay. Such service can also be bundled with software features wherein the smartphone manufacturers can tie the new feature like extra memory or storage.

    Picture By Chetan Arvind Patil

    THE PROCESS OF SEMICONDUCTOR AS A SERVICE

    Semiconductor-As-A-Service implementation can unlock a plethora of opportunities not only for the semiconductor industry but also for the software industry. However, implementing Semiconductor-As-A-Service requires a specific process to be followed from designing to manufacturing. It also requires the semiconductor industry to take risks by providing advanced technology node use today rather than a few years down. Using advanced technology is the key to fitting more silicon features that can be unlocked post-production as it allows more silicon in the smallest possible area as this helps in providing more features at the transistor level.

    Semiconductor-As-A-Service Process:

    Identify Future Software Needs – These software features should be those that become bottlenecks for consumers. It can be from understanding whether the consumers will need more memory than the product has been shipped with so that with the growing data-driven application enabling an extra memory at the silicon level can cater to the software demand. The same goes for CPUs and GPUs for processing power.

    Design Silicon With In-Built Hidden Features – Post identification of future software needs, packing the silicon with features that get unlocked in the future. The majority of these features will reside inside the System-On-A-Chip (SoC), as the active components are the ones that can provide more benefits of service-based features than passive components. Usage of advanced technology node is key to enabling such silicon level features.

    Ability To Enable The In-Built Hidden Silicon Features – Incorporating the in-built hidden silicon feature requires not only designing it with secure memory to store keys to activate features but also requires a secure manufacturing process. The secure way of design and manufacturing ensures that there are no security flaws that can be exploited by hackers.

    Innovative Manufacturing And Packaging – The critical piece of the Semiconductor-As-A-Service process is to ensure that the manufacturing flow and the packaging technology use advanced techniques to consider the effects when more silicon area is activated. Activating new features (more memory or processing capability) can have significant power and thermal effect.

    Product Cost: Planting more silicon with the expectation that it will get used in the future under a pay-as-use service is a business risk. It is vital to price such products so that the design and manufacturing costs invested gets recovered even when in-built hidden features do not get utilized.

    Above are the five key process steps that lay the foundation of Semiconductor-As-A-Service. It has the potential to make the silicon more adaptive. It will require massive research and development before the industry can use it as a real-world solution.

    Picture By Chetan Arvind Patil

    THE NEAR-TERM IMPACT OF SEMICONDUCTOR AS A SERVICE

    If Semiconductor-As-A-Service is implemented and widely used, then it has the potential to transform the computing industry.

    The ability to enable an extra layer of processing power on the go provides a new way to process data. With 3.5 billion 5G subscribers by 2026, the data consumption will skyrocket, and having silicon with in-built hidden features to cater to such high processing and memory demand will take computing to another level. Semiconductor-As-A-Service can also enable date centers and OEMs vendors with avenues to save cost and increase revenue by providing silicon level services.

    Semiconductor-As-A-Service Provides Avenues To Put Future Silicon Technology In Today’s Silicon Area

    FABs, FAB-LESS, IDMs, OSATs, and ATMPs will be able to use technology designed for future silicon today. It will help them understand its impact and usage before launching future silicon technology on a large scale. The semiconductor industry has already started embracing chiplets and heterogeneous computing. These two semiconductor and computing techniques can provide a perfect starting point where more silicon can be incorporated to use it in the future.

    IP based semiconductor business is going to benefit the most as it will allow designers to incorporate more features that can be locked and unlocked as per the need. FAB-LESS companies will make more business by providing vital features as-a-service.

    Semiconductor-As-A-Service also means every device out in the market is different than others as silicon features can be enabled and disabled to the consumer’s liking.

  • The Importance Of End-To-End Semiconductor Cluster Ecosystem

    The Importance Of End-To-End Semiconductor Cluster Ecosystem

    Photo by Laura Ockel on Unsplash

    THE END-TO-END SEMICONDUCTOR CLUSTER ECOSYSTEM

    The semiconductor industry is vital for high-tech advancement. From smartphones to satellites, a small piece of silicon forms the base for millions to trillions of data points. It is why worldwide, the semiconductor industry is a Key Enabling Technology (KET) provider. Semiconductor product development requires various resources to come together. With the growing demand for smart hardware, the need to develop these resources in-house is more critical than ever.

    In semiconductors, no single country wants to be 100% reliant. Countries are ramping up in-country semiconductor design and manufacturing efforts.

    The complexity of both the design and the manufacturing aspects of semiconductors makes it a tough business. It takes years and decades to come up with a turnkey ecosystem to drive in-country semiconductor design and manufacturing. The cutting-edge technology that is required to become self-reliant in semiconductor design and manufacturing demands a radically different approach than incentive-based schemes, which the majority of the governments provide.

    The End-To-End Semiconductor Cluster Ecosystem Requires In-Country Development And Growth Of Semiconductor To Drive Key Enabling Technology

    End-To-End Semiconductor Cluster Ecosystem: An end-to-end semiconductor design, manufacturing, and support ecosystem that enables seamless semiconductor product development. It requires different components of the semiconductor product development to be done in-country rather than globally. It drives in-country economic and talent development and is cost and time effective.

    The End-To-End Semiconductor Cluster Ecosystem is what countries should focus on building to pitch themselves as a one-stop destination for all semiconductor solutions. However, it is easier said than done. The list of different types of resources and solutions that are required to develop a semiconductor cluster ecosystem is long. Depending upon the market size and focus area, countries can have a different smaller focused end-to-end semiconductor cluster ecosystem that has all the components of semiconductor design to manufacturing to customer delivery.

    Picture By Chetan Arvind Patil

    THE COMPONENTS OF THE END-TO-END SEMICONDUCTOR CLUSTER ECOSYSTEM

    Creating a semiconductor cluster ecosystem is not easy. There are different components required to ensure that the environment supports the semiconductor business. Following are the major components of the semiconductor cluster ecosystem:

    RESEARCH AND DEVELOPMENT

    Research and Development (R&D) is key to both basic and applied science innovation. R&D requires the cooperation of government, academia, and industry. Given how complex semiconductor product development is (from technology node to packaging to power requirements), continuous and steady R&D spending is vital as it forms the base of the semiconductor cluster ecosystem.

    According to the Semiconductor Industry Association, in 2019, the U.S. semiconductor industry R&D spending was 16.40% of total sales. Europe spending was 15.30% of total sales, while Taiwan, Japan, China, Korea spending was 10.30%, 8.40%, 8.30%, 7.70%, respectively. It clearly shows the importance of R&D spending and how it helps drive the leadership in the semiconductor business.

    Countries wanting to implement the semiconductor cluster ecosystem need to increase R&D spending by collaborating with academia and industry, to drive advanced solutions for the market.

    DESIGN (FAB-LESS/EDA/IDM):

    Without the semiconductor design, there is no manufacturing. Countries around the globe are attracting businesses to design in-country. This requires setting up of FAB-LESS business which can drive the design of Analog, Digital, Processor, Memory, and Sensor-based products. To cater to the needs of FAB-LESS, EDA companies are required who can provide software-based tools to drive circuit to layout design, simulation, and validation. Apart from FAB-LESS, there are several IDMs (Intel, NXP, Marvell, etc.) which cater to the need of both the design and manufacturing aspect of the semiconductor.

    The development of an in-country design ecosystem requires a talent pool. This demands universities with excellent infrastructure that can provide deep technical training required to drive gain expertise in semiconductor engineering.

    MATERIAL:

    No FAB or OSAT in the world produces the materials required to bring the silicon to life. Different chemicals, silicon, photomasks, gases, substrates, compounds, etc., are required to develop the wafers and packaged materials. There is a big dependency on specific countries and companies that provide such materials.

    Semiconductor material development and procurements also mean a good understanding of the engineering aspect and as said it requires heavy R&D activities within the country where the materials eventually will get used, either by the FAB or the OSAT.

    EQUIPMENT:

    Semiconductor equipment is a billion-dollar market. Both FAB and OSAT require heavy machinery to process and assembly wafer silicon. ASML is the largest supplier in the world of lithography systems for the semiconductor industry apart from ASM, Applied Materials, and TEL. On the other hand, ADVANTEST, TEL, and Teradyne are the largest supplier of ATE-related equipment.

    Both FAB and OSAT equipment are vital to ensure the materials and design eventually get made in the form of a product. A country with a stronghold on the semiconductor equipment manufacturing market is key to anything semiconductors.

    FAB:

    Fabrication of semiconductor devices requires dedicated facilities with large clean rooms. The investment to create such a facility is big and is the primary reason why there is only a handful of semiconductor FAB around the world. Even out of the existing FABs, not all are equipped to handle the advanced technology node that the semiconductor industry has ventured into.

    TSMC, Intel, GLOBALFOUNDRIES, and Samsung Semiconductor are competing with each other to grab the opportunities presented by technology node 5nm and beyond. To make countries self-reliant in semiconductor, FAB play a vital role. It has pushed governments without any FAB facilities to provide incentives to set up new advanced FAB. However, setting up FAB also requires a supporting ecosystem, and this is why countries should focus on the cluster-based ecosystem that provides in-country end-to-end semiconductor solutions.

    OSAT:

    Outsourced Semiconductor Assembly And Test (OSAT) is as important as FAB. Packaging the products with the right technology enables long life. Testing every die on the wafer is vital to ensure there is no reliability or test escape. OSAT enables defect-free parts to the customer. They drive the back end of semiconductors, which in itself is a billion-dollar market.

    Historically, OSATs have been located in the Asia Pacific and have been dependent on America and Europe due to the R&D and design lead these two continents hold. For a semiconductor cluster ecosystem, all the major components need to be catered to, not only specific ones. This is why OSAT is trying to get into FAB and is also investing in in-house design.

    ATMP:

    Assembly, Testing, Marking, and Packing (ATMP) is different than OSAT. OSATs take the bare wafer silicon and convert it into a packaged product, which is then shipped to the ATMP houses. ATMP receive packaged semiconductor products from different OSATs and then they assemble it together on a printed circuit board (PCB). All the semiconductor devices are connected to form a working computer system and clear marking details are put on the PCB to ensure traceability of devices. As the last step, the PCB is covered with an aluminum or plastic body before being shipped to the customer in a fancy box.

    China is the leader in ATMP. India is another upcoming destination. Dell and Foxconn are the world’s largest ATMP houses. Having ATMP houses in-country provides economic development but at the same time negates the benefits when a country becomes 100% importer of semiconductor products. This is what has happened with India’s ATMP ecosystem.

    MISCELLANEOUS:

    Apart from all the major components, there are some crucial minor components that are also critical for the semiconductor cluster ecosystem. These include logistics, distribution, and enterprise-level software. Having delivery and development houses for these activities is also critical in ensuring an end-to-end semiconductor cluster ecosystem. Given these solutions are driven mostly by software in today’s day and age, the majority of countries have both development and R&D centers catering to the future of how to efficiently to logistics to distribution with the help of data and software.

    SUMMARY: End-to-end semiconductor cluster ecosystem requires all of the above components to be in close proximity. However, as of today, there is not a single full end-to-end semiconductor cluster ecosystem in the world. The majority of the semiconductor cluster ecosystem has one or max three of the above components. Given the race between countries to attract the world’s best semiconductor business and talent, the focus on the end-to-end semiconductor cluster ecosystem needs to increase by leveraging facilities within the same location or country. Having more FABs and then relying on other countries for OSATs and ATMPs is never going make a single country the destination for all semiconductor needs, and that is what the majority of the countries in the last two to three years are trying to achieve. Unfortunately, that is not possible till an end-to-end semiconductor cluster ecosystem is built in-country.

    Picture By Chetan Arvind Patil

    THE ACTIVE SEMICONDUCTOR CLUSTER ECOSYSTEM

    There are a handful of semiconductor cluster ecosystems located in different countries. However, these clusters do not cater to all the components discussed above. It will not be valid to call these centers a semiconductor cluster ecosystem, but it does show the importance of having one or more semiconductor components within vicinity.

    Following are a few active semi semiconductor cluster ecosystem but not end-to-end:

    Intel – Portland, Oregon, USA And Chandler, Arizona, USA: Intel has advanced FABs in Portland, Oregon, and Chandler, Arizona. There are two big universities in the proximity of these two FAB locations: Portland State University and Arizona State University. Cross-industry and academia collaboration at these two locations have to lead to the launch of several innovative semiconductor solutions. The exchange of talent for research activities has also helped. Intel’s presence in these two locations guided the formation of a semiconductor support environment that has helped its FAB execution. This is also the primary reason why TSMC has chosen Arizona as the destination of their next 5nm plant.

    ASE Global – Kaohsiung, Taiwan: ASE Global has multiple OSAT facilities in Taiwan. Kaohsiung plant stands out due to the proximity to other package technology solution providers like Amkor. The competition has helped with the development and availability of the semiconductor raw materials required to smoothly operate an OSAT facility.

    TSMC – Hsinchu, Taiwan: TSMC has several FABs around the globe with the majority of the FABs located in Hsinchu, and has helped TSMC develop an ecosystem that has allowed universities and OSAT nearby to thrive. Having OSAT and FAB in the same location also reduces the cost and time of product development.

    Newport Wafer Fab – Newport, United Kingdom: Newport Wafer Fab is the latest addition to the semiconductor ecosystem and promises to be the one-stop FAB needs for the UK region. It has tied up with Cardiff University to enable future compound semiconductor development. Showcasing why having universities nearby helps.

    Samsung – Gyeonggi, China: Samsung like TSMC has FABs in a different part of the world, with the majority located in Gyeonggi. China being home to both the OSAT and ATMP houses, has allowed Samsung to take advantage of the in-country ecosystem of semiconductors.

    TAKE AWAY: Above examples show the importance of having one or more semiconductor cluster ecosystem components in proximity. Imagine having all the semiconductor components in one location and that too within a single country. The benefits from employment, development, and growth will be immense. Whether or not such an ecosystem will end up getting developed, but for sure, countries are racing to attract the best talent and semiconductor businesses to drive in-country semiconductor growth.

    Picture By Chetan Arvind Patil

    THE WAY FORWARD FOR END-TO-END SEMICONDUCTOR CLUSTER ECOSYSTEM

    The semiconductor industry is going through massive critical changes. From mergers to acquisitions to new companies to new FABs, all this is shaking up the semiconductor business.

    Traditionally, semiconductor design and manufacturing has been all about specific regions/countries in the world having a stronghold on either the design or manufacturing or equipment. Post-2020, the story is going to change. Majority of the country has already started chasing giants of the semiconductor industry to set up their designs for manufacturing houses.

    Country With The End-To-End Semiconductor Cluster Ecosystem Will Lead In The Digital Technology World.

    Governments need to develop their country as an end-to-end semiconductor cluster ecosystem, with a solution for every component of the semiconductor development cycle. Having one facility and not the other is only going to make the new facilities in the new country dependent on the old facilities in other countries.

    The country that can create an end-to-end semiconductor cluster ecosystem is going to have an advantage over others and will lead the digital technology competition.

  • The BIG-5 Are Becoming Semiconductor Companies

    The BIG-5 Are Becoming Semiconductor Companies

    Photo by İsmail Enes Ayhan on Unsplash

    THE NEED TO PROCESS DATA

    Internet usage is growing. Every new user generates a new type of data. The technology companies are always eager to process and understand new consumer behavior. It requires continuous research and development of both the software and the hardware.

    Software development has advanced in the last two decades. It has kept pace with the need to understand and process data due to the development of software libraries and frameworks. The large amount of data that has generated post-2010 has helped the Deep Learning (DL), Machine Learning (ML), and Artificial Intelligence (AI) frameworks train networks, and that is now allowing new data to be processed faster and accurately.

    Hardware is vital in ensuring that the processing of data using training and prediction frameworks occurs in the shortest time possible. It requires a massive amount of computing. The majority of the technology companies now rely on massive data centers equipped with advanced computer architectures.

    BIG-5 – Facebook, Amazon, Apple, Microsoft, Google – FAAMG

    To fully utilize computer architectures, an in-depth architecture-level understanding is required. It is not always possible to do so, as the data centers still run general-purpose computer architectures that do not cater to different types of data the big technology companies have to process.

    The disconnect between the software, the hardware, and the data has promoted the need to move from General-Purpose SoC To Application-Specific SoC. Not all data companies are capable of setting up a dedicated team that can focus on in-house silicon development to come up with an Application-Specific SoC.

    To overcome the reliance on semiconductor companies, the BIG-5 (FAAMG) technology companies have started (or have already developed) developing in-house SoC with the hope of opening up the silicon to different data companies around the world.

    Picture By Chetan Arvind Patil

    THE PUSH FOR IN-HOUSE SILICON

    Two major factors drive the push to develop new computer architectures (silicon):

    • Memory
    • Parallel computing

    Memory:

    • Modern applications are becoming memory intensive and also demand faster computation. To process requests from memory-intensive applications in the shortest possible time, the data needs to reside closer to the processing unit.
    • The time to bring the data from SSD to DRAM to Cache adds cycles and delays processing of the data. To overcome such bottleneck, semiconductor companies have implemented the following three techniques:
      • Cache Prefetching:
        • Bring the data near the processing unit in advance to minimize cycle time
      • Increasing Level Of Cache:
        • Add Leve-1 (L1), Level-2 (L2 – Shared), and Level-3 (L3 – Shared) small (KB/MB) cache memory to improve memory prefetching speed
      • Enable High Bandwidth Memory:
        • An extra layer of large high-speed memory between Last Level Cache (LLC – Either L2 or L3) and DRAM to speed up prefetching
    • All the above three techniques improved the response time of processing units. However, as the application data started growing, the cache and memory trashing became a new hurdle.
    • Multiple processing units sharing the same level of memory started corrupting each other’s data to process the request faster. On top of all this, the lack of interconnect bandwidth added further bottlenecks.

    Parallel Computing:

    • Apart from being memory intensive, applications have become compute-intensive too. It prompted the need to have multiple processing units within the same SoC. Running multiple data requests on a single processing unit or two separate processing units provided a way to accomplish the task in the shorted possible time.
    • The processing units still relied on the low-level memories to bring the data to be processed quickly. It means new SoC designing techniques that can allow the sharing of cache and high bandwidth memories in elegant ways without compromising on the need to add latency.
    • Adding more processing units to a single SoC is not the solution. On top, the developers have to keep comping up with smart ways to distribute the data to multiple SoC to speed up the processing.
    • Distributed computing is what the majority of the technology companies have adopted to ensure the data is processed quickly. It means a massive number of servers with thousands of SoC and a large amount of memory. Over time this has increased the cost of operating data centers.

    Even though in the last decade, semiconductor companies have come up with unique computer architecture to cater to both memory and compute-intensive applications, it has not been enough to adopt the changing processing requirement of BIG-5.

    The need to handle memory and parallel computing demand by modern workloads and applications efficiently at the architecture level has pushed BIG-5 to go for in-house silicon.

    THE STATUS OF IN-HOUSE SILICON

    BIG-5 has been gearing towards the development of adaptive computer architecture for data and operating systems.

    Facebook:

    Facebook started working on in-house silicon a couple of years back. With a growing user base across multiple platforms (Instagram, WhatsApp, Messenger), Facebook ramped up silicon effort last year.

    They have a silicon team that is focusing on Application-Specific SoC development that not only caters to data centers but also portable devices like Oculus.

    Amazon:

    Amazon Web Services (AWS) is one of the leaders in cloud solutions. The desire to have customized SoC is vital to ensure the consumers and enterprises can make most of the wide range of computing services AWS provides.

    Apart from AWS, Amazon’s growing range of Echo products is also pushing it to drive in-house silicon development. Amazon is betting big on ARM architecture to drive its silicon needs.

    Apple:

    Apple was always into silicon development. This year with the Apple M1 launch, they are making big bets on in-house silicon development that caters well to their need.

    With Siri about to become the default search option on all the Apple devices, the need to have data-centric customized silicon will grow.

    Microsoft:

    Microsoft always had a keen interest in hardware. They already have a strong team of researchers focusing on hardware research. The Surface line of products has shown strong growth, and the SQ1 line of SoC establishes Microsoft’s goal of making Windows smoother to use on silicon.

    Recently, Microsoft also announced a plan to develop Secure Chip with the help of semiconductor giant Intel and AMD.

    Google:

    Like Microsoft, Google also has a dedicated team that has heavily contributed to silicon development via different computer architecture domains. They have already announced plans to develop in-house silicon for Pixel and Chromebook devices.

    A few years ago, Google showcased the world Tensor Processing Units (TPUs) to speed up the training of data set using the TensorFlow framework. Google’s latest data shows they have been successful in doing so.

    Picture By Chetan Arvind Patil

    THE POSSIBLE FUTURE SCENARIOS

    BIG-5 is betting big on in-house silicon development. This requires not only years of planning and investments but also a dedicated semiconductor development team and flow chain. Going forward there are two possible scenarios that BIG-5 might take:

    Scenario 1:

    BIG-5 will keep collaborating with semiconductor companies (Intel, ARM, AMD, and Qualcomm) to design silicon for their products and data centers with strict control over features and the manufacturing process. It will enable BIG-5 to enter the in-house FAB-LESS business model.

    Scenario 2:

    BIG-5 will slowly move away from semiconductor companies and spin-off an in-house team with a full turnkey silicon development chain. It will be more like an IDM business model and might require the acquisition of existing semiconductor manufacturing units.

    The probability of the second scenario occurring soon is unlikely. In a decade or so, BIG-5 may go big on the semiconductor business and try to keep themselves as in-house FAB-LESS silicon developers (while owning a piece of IDMs/FABs), which will ultimately play in the hands of FAB/Pure-Play Foundries like TSMC and GLOBALFOUNDRIES.

    Whichever scenario ends up occurring, there will be exciting developments in computer architectures that will drive the semiconductor industry to new levels.

  • The Challenges And Way Forward For Computer Architecture In Semiconductor Industry

    The Challenges And Way Forward For Computer Architecture In Semiconductor Industry

    Photo by Luan Gjokaj on Unsplash

    OVERVIEW

    Computers are designed to provide real-time feedback to all user requests. To enable such real-time feedback, Central Processing Unit (CPU) is vital. CPU is also referred to as processing units or simply processors. These incredibly small semiconductor units are the brain of the computer and are capable of performing Millions/Billions of Instructions Per Second (MIPS/GIPS). High MIPS/GIPS, means faster data processing.

    A lot of processing goes on inside these processing units. With the advancement of the technology nodes, more processing units are being glued together to form System-On-A-Chip (SoC). These SoCs have different individual units like GPU, DRAM, Neural Engine, Cache, HBM, ASIC accelerators, apart from the CPU itself.

    It is incredibly difficult to design an SoC that has the best of two important worlds of computer architecture: Power and Performance.

    Both in academia and the industry, Computer Architects (responsible for design and development of next-gen CPU/SoC) play a key role and are often presented with the challenge of understanding how to provide faster performance at the lowest power consumption possible. It is a difficult problem to solve.

    The battery technology has not advanced at the speed at which SoC processing capability has. Shrinking technology node offers opportunities to computer architects to put more processing power, but at the same time, it also invites issues related to the thermal and power budget.

    All this has lead to semiconductor companies focusing on design challenges around the power and performance of the SoC.

    CHALLENGES

    Semiconductor industry has been focusing on two major SoC design challenges:

    • Challenge 1: Efficient and low latency SoC design for portable devices
    • Challenge 2: High throughput and performance oriented SoC for data center
    Picture By Chetan Arvind Patil

    Challenge 1:

    • Portable:
      • Portable devices suffer from the constraint on the battery capacity. The battery capacity has been increasing mainly due to the shrinking board inside these devices due to the shirking transistor size.
      • This has allowed the OEMs to put more lithium-ion. However, to balance the form factor and portability, batteries cannot be scaled out forever. It is a challenge for OEMs to understand how to manage portability by balancing the battery size apart from making the computer system efficient with low latency.
    • Efficiency And Low Latency
      • To tackle efficiency and low latency, innovative designs are coming out in the market with the ability to adapt the clock and voltage domain depending on the application being executed by the user. It is no more about how many cores are in the SoC, but more about how an application-specific core can provide a much better user experience than ever.
      • This has presented researchers with an interesting problem of improving the performance per watt (PPW). To improve PPW, researchers around the globe are taking different approaches around DVFS schemes, apart from improving transistor level techniques.
      • Frequency and voltage level scaling also has a direct impact on the response time. Processing units like CPU are designed to provide low latency so that all the request coming in, can be catered to in real-time.
      • Improving efficiency without compromising on the latency is still a big challenge for the computer architects.

    Challenge 2:

    • Data Center:
      • On the opposite pole, data centers are designed to be compute-intensive. The SoC required to cater data center has exactly the opposite need compared to portable devices. As companies become data aggregators, the analysis requires dedicated hardware that provides streamlined computation of the data on the go.
      • This is prompting companies like Google, Facebook, and Amazon to come up with their silicon that understands the data being generated and how to swiftly analyze it on the go.
    • Performance And High Throughput:
      • Designing custom SoC requires a fresh look and is drastically different than the block based approach. Improving throughput requires high speed interconnect to remove bottlenecks in data processing, else the performance will be affected.
      • In order to improve throughput, the data needs to reside near the computation block. This demands a new way to predict data to be used in order to bring in the cache or add a memory hirerachy with the help of MCDRAM.

    The challenges are many and researchers around the globe are already working to provide elegant computer architectures both from academia and the industry.

    WAY FORWARD

    As the need of the application running on the computer systems is changing, so is the approach to designing SoC. Various examples from different companies show how the development of computer architecture is changing and will eventually help others come up with new computer architectures.

    These new architecture designs are taking the traditional approach of computer architecture and providing a different way to tackle both memory and compute bottlenecks.

    Cerebras came up with Wafer-Scale Engine (WSE), which is developed on the concept of fabricating full wafer as a single SoC. The performance data of WSE show a promising future of how computer architecture becomes more wafer-level designing than die level. WSE also takes different approach on interconnects by utilizing wafer scribe lines to transfer data which provide more bandwidth.

    Fungible’s Data Processing Unit (DPU) architecture is another way forward that shows how SoC will be increasingly get designed for scale-out systems to handle massive data.

    Picture By Chetan Arvind Patil

    Google’s TPU and Amazon’s Inferentia shows how custom ASIC based SoC will become de-facto. Companies that generate a lot of data will try to run their center on in-house developed SoC.

    Apple’s M1 launch showed how ARM will start eating the x86 market for energy-efficient portable devices. In few years, the integration will become more intuitive and might attract other x86 portable devices OEMs who have failed to take Windows on ARM to its true potential.

    NVIDIA’s bid to acquire ARM shows that the future GPU will be designed with a blend of fusion technology that will combine ARM/CPU with GPU more than ever. This will allow data centers to improve on latency apart from focusing on throughput.

    In the end, all these are promising development for the computer architecture community. Provides numerous opportunities to research and develop new ways to enable lower latency and higher throughput while balancing power consumption.

  • The Semiconductor Industry Shake Up

    The Semiconductor Industry Shake Up

    Photo by Jason Leung on Unsplash

    THE SEMICONDUCTOR INDUSTRY STATUS

    In 2020, the semiconductor industry has seen both negative and positive trends.

    The first half of 2020 showed mostly the negative trend driven by the COVID-19 restrictions, as it lead to slower semiconductor production and increased inventory due to decreasing sales. The second half of 2020 has been more positive. The sales have gone up and production lines are 100% occupied, to cater the newly launched devices and products by vendors across the globe.

    Apart from the steady increase in design, development, and production, merger/acquisition have gone up too. There have been some unexpected takeovers which are bound to have a strong impact in the long run.

    The semiconductor industry from a product point of view can be divided into:

    • CPU
    • GPU
    • SoC/MPSoC/RFSoC
    • ASIC/FPGA/ASSP/ACAP
    • Digital/Analog/Mixed
    • Memory
    Picture By Chetan Arvind Patil

    The mergers and acquisitions that have occurred in 2020 have affected each of the above product domains.

    All these acquisitions from the design have shaken up the semiconductor design industry. However, at the same time, it is turning out to be a boon for semiconductor manufacturing, as many IDMs plan on becoming FAB-LITE to focus more on the design aspect and increase share the mobile, AI, and data center market.

    This raises the question of how the future of semiconductor design and manufacturing is going to be.

    THE SEMICONDUCTOR INDUSTRY SHAKE UP AND FUTURE

    Taking a look from the semiconductor design point of view, it is getting clear that companies are more focused on a specific product domain and want to dominate the market. To achieve this, companies are either creating new asset via acquisition or selling old asset that do not align with the goal.

    Intel last year sold its smartphone modem business to Apple and this year Intel also decided to sell NAND memory business. This shows that Intel wants to focus on its strength of personal and data center computing. For sure, NVIDIA’s acquisition of ARM is concerning for Intel, given how much strength an established IP from ARM will give NVIDIA and also allow it to extend its business from GPUs to CPUs, and that too smartphone business which is not Intel’s primary domain.

    On top, with AMD’s solid performance and acquisitions, the fight for smart computing is going to heat up. AMD (since 2009) and NVIDIA are FAB-LESS semiconductor companies. This allows AMD and NVIDIA to focus more on the design aspect and let external manufacturers take care of the manufacturing. This is a big advantage as semiconductor manufacturing is hard and takes a long time to perfect.

    Picture By Chetan Arvind Patil

    All these points towards a major shake-up that will occur in near future. The business mode; will change and semiconductor companies will go either:

    • FAB-LESS
    • FAB/Pure-Play Foundry

    Competing in both the arena as a single entity is going to be challenging. Spinning off or selling part of the semiconductor manufacturing might be a more viable solution. Such shake-up will eventually end up creating more business for the semiconductor manufacturing companies and they will have to predict today and start planning on increasing the capacity (or acquisition) to keep the business running.

    It will be vital for countries like India to take advantage of such market business change by coming out with policies that heavily incentivize semiconductor manufacturing.

  • Indian Automotive Industry Can Drive Semiconductor Manufacturing In India

    Indian Automotive Industry Can Drive Semiconductor Manufacturing In India

    Photo by Amin Khorsand on Unsplash

    India’s automotive industry is the world’s largest two-wheeler, three-wheeler, and tractor manufacturer. It is also the world’s second-largest bus manufacturer, third-largest heavy truck manufacturer, and fourth-largest car manufacturer.

    With the advancement in automotive solutions that will enable sensor-based safe assisted driving, the demand for smart Electronic Component Units (ECUs) will grow for every car, tractors, trucks, buses, and motorcycles that will come out of India’s production unit. Smart semiconductor products have started to change the automotive industry for good.

    Picture By Chetan Arvind Patil

    With the growing demand for automotive vehicles in India and the ability to export it in different regions, isn’t it time to ensure that the semiconductor products that are drivers of smart capabilities are also manufactured in India?

    The fairly old and well established automotive industry can surely enable semiconductor manufacturing growth given the increasing demand for smart solutions from infotainment to radar to collision avoidance.

    It is being anticipated that in 2030, 80% of the cars driving capability will be based on smart ECUs. Rest 20% will depend on the rules and regulation around level 5 is autonomy.

    AUTOMOTIVE INDUSTRY IN INDIA

    India has set an ambitious goal to move to alternate-fuels for automotive. First step is 100% electric cars only by 2030. This also means increased demand for semiconductor products that make up the smart ECUs to enable efficient driving.

    With each new car having more feature than the last one and all of these features relying on semiconductor solutions, raises the question of how automotive industry can drive semiconductor manufacturing in India.

    A very crucial and vital role will be played by the Original Equipment Manufacturers (OEMs). India is already home to all of the top automobile manufacturers that are evenly spread across different clusters and states.

    India Will Be The World’s Third-Largest Automobile Manufacturer With A Market Size of $300 Billion By 2026.

    Picture By Chetan Arvind Patil

    Designing of automotive ECUs is actively done in many of the R&D offices by the semiconductor companies in India. However, the majority of these smart solutions are manufactured overseas and then imported by automotive customers back in India.

    If all the activities from design to manufacturing of semiconductor products for the auto industry is done in house in India, then the benefits in terms of employment and business are huge.

    Any advanced car that is manufactured in 2020, has around 100 million lines of software code that has to meet automotive regulation. The established software industry in India can then provide over the top (OTT) solutions that will quadruple the amount of safety features on top of the smart mobility solution provided by the semiconductor solutions.

    LEADERS IN SEMICONDUCTOR AUTOMOTIVE SOLUTIONS

    Currently, the leaders in semiconductor manufacturing for the auto industry are all based in the Americas, EU, or East Asia. India certainly is home to all these semiconductor giants but only from designing and R&D point of view, which in itself is a great advantage.

    The automobile industry’s growing demand for smart semiconductor solutions in India should be an attractive opportunity. Establishing semiconductor manufacturing units in India not only allows semiconductor design companies to have access to the automobile industry but can also enable the growth of other sectors that they are catering to apart from the auto industry in India.

    Having an end-to-end design to manufacturing solutions in India will also help companies innovate faster and enable cost savings without compromising on security and quality.

    Another important factor is India’s open market that already has a talent pool to drive innovation. Many companies already heavily invested in R&D activities that can then quickly be tested using the automotive infrastructure.

    SEMICONDUCTOR AUTOMOTIVE OPPORTUNITIES

    As per Deloitte, the cost contribution of automotive electronics in 2007 was 20% and in 2017 it increases to 40%, and by 2030 it is expected to be 50%. With the way smart mobility is changing the auto industry, it is fair to say that the trend is only going to go up. Alternate fuel will demand more smart semiconductor solutions and many of these are already in use.

    From infotainment to lane assistance, all require ECUs that are not manufactured from start to end in India. This has lead to an increase in the cost of semiconductor components.

    Picture By Chetan Arvind Patil

    With the increasing use of electronics in automobiles along with exploding automobile industry, India needs to re-think the policies such that they not only cater to the automotive industry but also enable semiconductor manufacturing.

    Having policies that provide more financial and profits based incentives is going to attract foreign direct investment in semiconductor manufacturing much faster than a scheme that only caters to one domain that needs to be started from scratch in India.

    AUTOMOTIVE AND SEMICONDUCTOR MANUFACTURING COLLABORATION

    India cannot afford to be 100% importer of these electronic semiconductor solutions if it has to be the leader in the automobile industry in the smart mobility world.

    The automotive industry in most cases doesn’t require the lowest semiconductor technology node possible. That can be one take away where semiconductor fabrication can be set up for higher technology nodes, which are cost-effective and affordable given India’s history with semiconductor manufacturing. Apart from automobiles, higher technology node can also cater to other segments like smart devices, smartphones, and smart infrastructures.

    Picture By Chetan Arvind Patil

    There are already many greenfield electronics manufacturing clusters that are in the region where automotive companies already have manufacturing units. Collaboration between the automotive and semiconductor industry in India needs to be explored to drive semiconductor automotive manufacturing.

    Automotive Electronics Account For 8% of Electronic In India

    The sensor market for automotive itself is going to be worth $5 billion by 2022. This is just one of the many segments that automotive in semiconductor solutions for. Adding AI to semiconductor solutions for smart mobility is going to increase the market demand tenfold. Having all such solutions in the house is critical for long term financial and innovation growth.

    Picture By Chetan Arvind Patil

    Designing of advanced automotive semiconductor chips already happens in India. The only hurdle is manufacturing and testing at large scale. The government of India is already looking to expand the local manufacturing of automobile equipment and it is about time to enable the same for semiconductor manufacturing from automobile use.

    The success of automobile semiconductor manufacturing in India will surely enable the growth of end-to-end semiconductor manufacturing for different domains.

    PSA

    NXP is a leader in advance autonomous semiconductor solutions from design to manufacturing and below videos show a glimpse of how they do it.

  • One Breakthrough Technology That Is Must For Work From Anywhere Workplace

    One Breakthrough Technology That Is Must For Work From Anywhere Workplace

    Photo by Andrea Caramello on Unsplash

    Work From Anywhere Workplace is becoming the standard mode of working. Many of the top companies have started providing Work From Anywhere Workplace as an option to the employees. According to the survey by Buffer, 98% of the respondents will prefer remote working if given an option.

    Many software and hardware tools are required to ensure that Work From Anywhere Workplace experience is as good as Work From Office. These resources and tools are only valid when the nature of the job enables employees to work remotely. Unfortunately, not all jobs fall into this category.

    Even though there are so many resources available for Work From Anywhere Workplace, there are still gaps needed to be filled.

    THE TOOLS

    To work efficiently from anywhere, one needs a set of correct tools. These tools increase productivity.

    Any person working in the technology-enabled domain has to have the basic tools like hardware that establishes connectivity to the external remote work. This hardware tool runs the software. Depending on the need and nature of the work, this can be a smartphone, a tablet, a desktop, a laptop, or an ultra-book.

    To make use of hardware, seamless connectivity is required. Always-on high-speed internet networks are so crucial that without it none of the tools running on top of the hardware can be utilized fully. High-speed connectivity is defacto but still suffers from issues of downtime and intermittent slowness.

    Connectivity also comes in the form of ensuring enough ports are available from the power outlets to multiple USB and HDMI outlets via a docking systems.

    Picture By Chetan Arvind Patil

    Communication is a key aspect of Work From Anywhere Workplace. It requires a reliable cell network for tasks that demand urgent inputs from the team. Many use VoIP tools nowadays, but the fact that calling and asking for solutions establishes resolves issues quickly.

    Audio and Video are key tools to Work From Anywhere Workplace. Getting up from the cubicle and walking to the colleague for help is not possible in remote work. The only way to efficiently hold project presentations and important work calls are by using audio and video compatible tools.

    All these tools discussed above are good to work efficiently from anywhere. Workplace need not be just home but can be any place like an outdoor place, co-working area, or even a coffee shop.

    If all the tools that are required for remote working are available then what is the missing link?

    THE NEED

    A quiet place to work is everyone’s dream setup. No interference mainly from the surrounding noise is key to working efficiently. To eliminate any type of noise, many have started using noise cancellation headphones. These work perfectly and do the job of eliminating noise but are often costly to afford and not everyone is willing to pay the high premium cost. On top, not everyone likes to keep headphones on all the time.

    Another solution to a focused workplace remotely is to have a dedicated room that can allow focused working. However, finding a quiet place in a home environment with family members is not an easy task. If the employee is working from a public places then too it is difficult to find a quiet corner.

    Picture By Chetan Arvind Patil

    Apart from noise, there is a privacy concern too. What you hear can be routed through headphones/earphones/speakers, but then when one speaks over the work call, then it is not possible to cancel the outgoing noise. In Work From Anywhere Workplace environment, one can be working next to anyone, thus inviting privacy concerns when sensitive work information is discussed.

    If we take a holistic look, then there are different types of sounds that a person working in an remote working environment has to deal with. These ambient sounds can be from nearby houses or roads, or from people in close proximity. All this leads to audio privacy concerns.

    This audio privacy concern is the missing link from calling a Work From Anywhere Workplace a perfect workplace environment. The question is how to ensure audio privacy with the help of Audio Privacy Technology.

    THE SOLUTION

    Audio Privacy Technology solution involves eliminating noise/sound without the need for a dedicated headphones/earphones. Even if one uses headphones/earphones, then these devices are only capable of eliminating ambient noise but do not ensure that the output audio from the employee is only heard by the employer and the team, and no one else.

    A perfect Audio Privacy Technology will ensure that without using any device and by only having the workstation, one is able to create different focused Work From Anywhere Workplace Zone.

    These focused sound zones work in harmony with the nearby environment to ensure that none of the ambient sounds sneak into Work From Anywhere Workplace Zone. It also ensures that the sound generated from the Work From Anywhere Workplace Zone does not disperse.

    These two features working in synchronization ensure that there is bi-directional audio privacy.

    Picture By Chetan Arvind Patil

    Audio Privacy Technology is audio aware. It provides breakthrough solutions that can not only be used for Work From Anywhere Workplace but can also be applied in different zones like in cars, public areas, airports, flights, and many more places where audio privacy is non-existent.

    Another impact of Audio Privacy Technology is the elimination of headphones/earphones. With the decreasing number of ports in laptops to smartphones, wireless devices are the only solution. With wireless, the solution is not an issue but the degrading battery power when working remotely for longer hours is.

    Thus, having a chip-based feature that is embedded inside the smart devices which is capable of creating audio zones is the need of the hour. This will also make headphones/earphones obsolete. On top of it, not everyone likes to be wired all the time, and having bi-directional audio privacy without the need for headphones/earphones can make people more productive.

    THE BREAKTHROUGH

    There are already Audio Privacy Technology solutions in the market. The key is to make it accessible at large scale. Like the elimination of audio jack lead to the adoption of wireless headphones/earphones. Similarly, one-day electronic chips embedded in the smart devices will remove the need of having headphones/earphones. One less device to carry as embedded chip with Audio Privacy Technology will ensure more privacy and noise cancellation than any other solution currently in use.

    Noveto Systems already have the 

    SeatCentric by Bose already has Audio Privacy Technology. It will be interesting to see how soon they can release products that can be used by employees Working From Anywhere Workplace.

    Video By Bose

    Hyundai also has the Audio Privacy Technology ready, and “>certainly, it looks promising.

    Samsung owned HARMAN also showcased a similar solution back in 2015 CES.

    It will be interesting to see how the future of Work From Anywhere Workplace evolves with Audio Privacy Technology. This breakthrough technology will not only help audiophiles but can literally make headphones and earphones obsolete while ensuring that anyone can work remotely at any given location by soundproofing the work zone.

    Audio Privacy Technology can also be applied in Work From Office environment where one doesn’t like to listen to colleagues speaking loudly, or the noise from table fan used by someone next to the work cube.

    There will be concerns about not being able to hear emergency information from nearby surroundings, but then the same applies when someone is wired in with noise cancellation headphones. Surely, there will be options and features to enable emergency information to sneak in.

    Only time will tell how the solution will evolve, but certainly Audio Privacy Technology breakthrough is must for Work From Anywhere Workplace.

  • The Data-Driven Approach Towards Semiconductor Manufacturing

    The Data-Driven Approach Towards Semiconductor Manufacturing

    Photo by Vishnu Mohanan on Unsplash

    THE GROWTH OF DATA IN THE SEMICONDUCTOR MANUFACTURING

    The growth of digitization has led to the generation of massive amounts of data on day to day basis. The data explosion has pushed several industries to adapt to data-driven analysis and decision-making processes. The same is true for the semiconductor industry.

    Data was always an integral part of semiconductor design and manufacturing. The need for data has increased further due to the cost and risk involved in designing products that are becoming smaller and smarter. The cost to fabricate semiconductor products is high and that is why during the design phases innovative simulations/modeling approaches are used to validate product design against specification before sending the final design files to the semiconductor fabrication houses. The risk of not spending time to collect and review semiconductor data both during the design and the manufacturing stage can cost a lot of money and time if the product starts failing in the field. 

    Data-Driven Approach Is Raising The Bar Of Semiconductor Manufacturing

    As the technology-node and package technology have advanced, so needs to take a data-driven approach to semiconductor manufacturing. The goal of the data-driven approach is to ensure that the products being are defect-free. The sophisticated tools and equipment used by the semiconductor manufacturing process have only gotten more advanced. The de-facto feature of these system-driven tools is to capture data and present it in a form that makes the decision-making process easier and faster.

    The data-driven approach is required to meet the following two important requirements of semiconductor manufacturing:

    Quality: Qualifying every semiconductor products is vital. It has to meet different industrial standards. Automotive semiconductor products have to go through one of the most stringent qualifying processes to ensure the product can work seamlessly everywhere. It is vital to ensure the manufacturing process established meets the high standards of quality requirement, which is possible only if the data captured to drive the decision-making process.

    Waste: Every data point tells a story. In semiconductor manufacturing, data can be about the design or the manufacturing tools, or handling issues, and many more. Every semiconductor data type counts towards ensuring the wafers and assembled parts are not getting scrapped. Scrapping raises yield concerns and indirectly impacts the cost.

    Without investing time and money in the data-driven approach to manufacturing semiconductor products, the semiconductor companies run into the risk of overlooking the quality metric. Driven by the need to ensure the products using the semiconductor solutions meet user satisfaction, the customers/companies using the semiconductor products are increasingly using Defective Parts Per Billion (DPPB) as a new metric than Defective Parts Per Million (DPPM).

    The need to capture, process and review semiconductor manufacturing data is becoming vital when there is a possibility that the semiconductor device might end up powering critical infrastructures likes satellites to aerospace to automotive (and many other solutions).

    All this is why the semiconductor industry is becoming more data-driven than ever before. The manufacturing part of the semiconductor process is not complete without letting the data speak.

    Picture By Chetan Arvind Patil

    THE PROCESS TO CAPTURE DATA IN THE SEMICONDUCTOR MANUFACTURING

    The semiconductor manufacturing process is a well-established flow. While every product has different requirements driven by the specification it is supposed to work within, some standard flows/stages are common across numerous semiconductor products.

    These processes use tools/solutions/recipes to capture data, which eventually drives a data-driven decision-making approach in semiconductor manufacturing.

    Following are the major process that leads to capturing of semiconductor manufacturing data:

    Fabrication: The first step towards semiconductor manufacturing is to fabricate the product on the wafer. Every layer/mask/stage of the semiconductor device fabrication is data-driven. If the data does not align at the etch or the lithography stage (for example), the wafer/lot cannot move forward. Several modules are placed in the kerf region of the wafer to data of the devices (that eventually make up the full die) to ensure is no deviation in the semiconductor wafer. All this data is unique to the technology-node. At the end of the wafer fabrication, data in the form of visual images ensure no particles or scratches are present on the wafer/die.

    Wafer Test: The defect-free wafer coming from the semiconductor fabrication facility gets sent to the test facility. The wafer test stage ensures all the die on the wafer gets electrically tested with the test program to validate whether the die meets the specification or not. The good and the bad die are marked accordingly. All this is possible due to ATE tools and probing hardware. The data is analyzed to understand any excursion, which eventually gets debugged to ensure no test escapes. 

    Final Test: The final test can be performed on the wafer (Wafer-Level Chip Scale Package (WLCSP)) or on the assembled parts (non-WLCSP). The final test uses similar testing criteria and tools like the wafer test). The major difference is the packaging testing versus dies testing. The data points are specific to the package testing. This is often the last electrical data point the semiconductor manufacturing process generates before the parts are shipped to the end customer or assembly.

    Assembly Inspection: Post assembly, the visual inspection of the assembled parts is carried out. The tools used for visual inspection capture all sides of the assembled parts. The data in the form of images show if chipping/defects occurred during the assembly process. If so, then the root cause needs to be determined. The data in the form of images often requires a highly automated system to alert any defects.

    System Testing: A final system (like a smartphone) not only uses one specific semiconductor product but several others. All the different semiconductor products are assembled onto the Printed Circuit Board (PCB) and should work in harmony. If anything fails, then the data relevant to the failing semiconductor product is capture and sent to the company that fabricated it. System testing also drives the semiconductor manufacturing process from the customer side.

    Miscellaneous: The last data point in the semiconductor manufacturing are from the equipment that enables the different process. These need periodic maintenance to avoid breakdown. All the equipment connected via a Manufacturing Execution System (MES) is monitored to capture every activity the equipment/tools perform. This data is vital for keeping the semiconductor manufacturing facility up 24x7x365.

    The above data points are vital in ensuring the semiconductor manufacturing process is fabricating/testing the high-quality products that meet the stringent quality requirements. To ensure, the high-quality demand data-driven process is critical in semiconductor manufacturing.

    Picture By Chetan Arvind Patil

    THE ADVANCED DATA APPROACH IN THE SEMICONDUCTOR MANUFACTURING

    Even though data exploration in semiconductor manufacturing is not new, the need to take a different data analysis approach is growing. The main reason is due to the high cost required to generate the semiconductor data. In the smallest data available, the semiconductor engineers need to predict that the products will work in the market as per the specification.

    To lower the time taken to move the wafers to the next stage, the semiconductor manufacturing process is relying on different advanced approaches to enable data-driven semiconductor manufacturing:

    Algorithm: Due to the proliferation of fast systems that can crunch data on the go has led to the development of algorithmic solutions that can detect scratches, clusters, excursions, particles, etc., faster than ever. The algorithms deployed can capture the patterns to raise an alert if there are issues with the semiconductor manufacturing process. These algorithms take the help of numerical data or visual data. Visual data is more vital as it shows gate-level details using an advanced microscope. There are already solutions, but there is a need to minimize the time taken for decision making as spending more time to decide on the correctness of the product means holding the lot/wafer from releasing, which can be costly.

    Analytic: Data lying without insights is not useful. Given that every die/chip that gets shipped out of the semiconductor manufacturing flow is visualize/tested, it is important to develop an analytical approach that uses computational analysis of statistics to drive data exploration. It helps the semiconductor data engineers to visualize data using diverse statistical methods to find issues/outliers. Analytical solutions are already available in the market/industry, but often requires skills to look beyond what is being presented on the dashboard.

    Tool: Tools are required to view the detailed history of every die that gets manufactured. The standard process to capture and store is already defined. The end-to-end semiconductor manufacturing data analysis still requires different tools to visualize and critically analyze the data. One single data tool may not be enough. It is good to test and use data analytical tools (or programming methodologies) as per the need to drive the decision-making process.

    Decision: Eventually, the goal of data analysis in semiconductor manufacturing is to either hold or release the product. It requires the perfect combination of computer and human. Computers can enable data collection and storage. However, the data wrangling is still on humans. In semiconductor manufacturing, every new data requires a new inspection approach as every new product is different from the last one. All this means every changing data-driven approach to decision-making (hold or release).

    The speed at which the semiconductor industry is driving towards the shirking technology-node year-on-year, the process to scrutinize the data will also change, and the semiconductor data inspection process will keep getting directed towards new ways to enable the smarter data-driven approach towards the semiconductor manufacturing.

  • Semiconductor Fabrication

    Semiconductor Fabrication

    Photo by Mathew Schwartz on Unsplash.

    Samsung’s technical blog has series of articles on semiconductor fabrication. It covers majors steps from tape out to packaging. All nine parts of the series are not linked together, so I thought of creating a list that may help those interested in learning about semiconductor manufacturing.

    All the images below are from respective part of the series linked in the title.

    Part 1: Creating the Wafer

    Picture By Samsung

    Part 2: The Oxidation Process

    Picture By Samsung

    Part 3: The Integrated Circuit

    Picture By Samsung

    Part 4: Drawing Structures in Nano-Scale

    Picture By Samsung

    Part 5: Etching A Circuit Pattern

    Picture By Samsung

    Part 6: The Addition of Electrical Properties

    Picture By Samsung

    Part 7: The Metal Interconnect

    Picture By Samsung

    Part 8: Electrical Die Sorting (EDS)

    Picture By Samsung

    Part 9: Packaging and Package Testing

    Picture By Samsung