Leather-jacketed, bespectacled and grey-haired, Jensen Huang has the appearance of an intellectual biker. “Accelerate everything,” the Nvidia boss told attendees at this week’s Computex show in Taipei. He was, of course, not talking about two-wheeled modes of transportation but urging companies to continue their splurge on Nvidia’s graphics processing units (GPUs), the chips that have serendipitously turned out to be good for artificial intelligence (AI). Customers seem to be fighting over constrained supplies, like bog-roll shoppers in a pandemic. Gaining 14 percentage points in the last year, Nvidia’s gross margin is an obese 78%. Systemic risk is made of such things.
But it’s another, more apparently mundane product that is the cause of some excitement in telecom. As well as building GPUs, Nvidia is also a customer of Arm, the UK-headquartered chips designer whose blueprints feature in most of the world’s phones. Various chipmakers are now using these Arm blueprints to produce central processing units (CPUs), general-purpose silicon that forms the brain of a laptop or server. The Grace part of Nvidia’s Grace Hopper superchip is an Arm-based CPU. In the future, it could power 5G networks.
This overarching CPU market today is dominated by Intel. Estimates of its market share vary, but in the subsector for server CPUs it claimed 71% of all sales in 2022, according to Counterpoint Research. Like AMD, the second-biggest vendor by far, Intel bases its chips on a so-called complex instruction set computing (CISC) architecture dubbed x86. Reduced instruction set computing (RISC), the main alternative, is the province of Arm (whose name literally stands for Advanced RISC Machines).
For years, CISC and RISC have been like the armies of WWI, barely advancing beyond their established lines. Intel’s x86 muscle is a valuable powerlifter in the data center but judged too resource hungry for a mobile device. Arm has long seemed the opposite – supremely energy efficient but too puny to cope with any workload needs outside a smartphone chassis. Yet this perception is changing in Arm’s favor.
Exploding silos
This matters in the world of telecom because operators are increasingly interested in using general-purpose processors and IT platforms to host network software. With such virtualization, or even cloudification, they could theoretically explode the silos that come with customized products and have commonality across all workloads. But Intel’s CPU dominance is an obvious barrier. Numerous telco sources still call it a monopoly.
In the virtual radio access network (RAN), the label seems apt. While virtual RAN still accounts for only a sliver of the total RAN market, just about every rollout so far is based on Intel’s Xeon-branded family of CPU products. Judging by telco feedback, the lack of options is partly to blame for the slow progress. Upset by government curbs on the use of Huawei, a Chinese vendor, operators are nervier than most about concentrations of market power.
Despite this, competitors to Intel have focused mainly on Layer 1, a slice of the technology that includes the most resource-hungry network software. In a virtual RAN, Layer 1 processing happens chiefly in a server box called the distributed unit (DU). Intel has stumped up its own accelerators – hardware dedicated to Layer 1’s specific needs. Yet so have chipmakers including AMD, Marvell Technology, Qualcomm and Nvidia. In most cases, these accelerators live on smart network interface cards that can be plugged into a server. But the Intel CPU still does everything else.
Changing that means cultivating CPU alternatives to Intel. AMD, given its established presence in server CPUs, is perhaps the most obvious candidate. Because it also hails from the x86 world, it could necessitate less redesign of the network software originally intended to run on Intel.
Yet several companies have latched onto Arm’s designs for virtual RAN CPUs. Amazon, most prominently, has built a Graviton-branded chip for its public cloud data centers. In principle, this could also host RAN workloads, although it does not yet appear to have been employed this way. Ampere Computing, an Arm licensee backed by Oracle, is developing servers that could be deployed in a more conventional RAN environment.
For Nvidia, another member of this group, the big pitch to operators is about the Hopper GPU as an accelerator for Layer 1. In typical style, Nvidia even has a set of Layer 1 software, branded Aerial. So far, however, relatively few telcos have looked eager to bite. Specs for Nvidia’s Hopper-based accelerator cards last year put maximum power at a range of between 230 and 350 watts. That compared unfavorably with a 40-watt Layer 1 card then being marketed by Qualcomm. Unfortunately, the economics of GPUs look questionable unless they support both AI and Layer 1 workloads. And while that is what Nvidia intends, the opportunity for AI outside public cloud facilities is unclear.
Grace in favor
By contrast, experimentation with the Grace CPU has been far more positive. Take customized accelerators out of the equation and a previous drawback of Arm noted by Ericsson was its lack of support for vector processing. Critical to Layer 1, this can effectively tackle a whole array of data in one go rather than working sequentially through individual data points. Intel caters to it with an instruction set known as AVX512. Thanks to a recent v9 architectural update, including a library called SVE2, Arm has been adapting to the same needs. And Nvidia’s implementation of vector processing in Grace CPUs appears to have brought success in recent third-party trials, the results of which were obtained by Light Reading.
Those trials used a Grace CPU with 72 Arm “cores” (the building blocks of a chip) and ranked Nvidia ahead of other Arm licensees on several important metrics. The points of comparison were Intel CPUs mixed with separate accelerators for the forward error correction (FEC), an especially gluttonous bit of Layer 1 software.
With vector processing, Grace was able to handle the FEC and still have cores available. When Grace was combined with an accelerator card from Qualcomm, rather than Nvidia’s Hopper GPU, this “free” core count rocketed, leaving Intel far behind. While results also put Intel slightly ahead on CPU performance, they scored Grace – even with its higher overall number of cores – better on power efficiency.
There are total cost of ownership trade-offs to consider in all this, but the main black mark for Intel is still likely to be its dominant position in virtual RAN, small as that market may be. In newer products, Intel has stopped putting Layer 1 accelerators on separate cards and started to host them on the same die as the CPU. It has been scathing about cards as an extra cost and complication. But the move could backfire. Some operators are still using its cards, for one thing. And an “integrated” product, as the chipmaker calls it, makes replacing Intel in any area even harder. Given that open and virtual RAN is supposed to be all about “disaggregation,” it is an odd marketing tactic.
None of this means Nvidia or Arm generally is an ideal choice. The Arm ecosystem for CPUs is far less developed, just as the AI one is for anyone except Nvidia. This makes it hard for some telcos to regard any Arm-based product as a common, off-the-shelf server, housed in a data center and good for any workload. The cloud universe remains an x86 constellation.
But at the same time, the servers built around Intel’s latest “integrated” products arguably feature so much customization that they are unsuitable for anything except the RAN. Given a mix of regulatory, technical and cost concerns, most RAN compute equipment is still found at the base of antennas. Where centralization has occurred, it has not usually resulted in the sharing of equipment between RAN and other workloads. And the public cloud has had a negligible influence on the RAN. In this world, finding alternatives to Intel is perhaps even more important.
This post was originally published on the 3rd party mentioned in the title ofthis site
