The “AI PC” race is ignoring the AI leader: NVIDIA

Walking around the exhibit halls and the hotel conference areas in Las Vegas this week during the annual CES (Consumer Electronics Show) event there are two things that stand out. First, that this feels like a return to the years before 2020 with the size of the crowds and amount of new products. Second, you simply cannot turn a corner or sit at a blackjack table without someone talking about AI.

Because this event focuses mostly on consumer-facing products, all of the key players in the AI PC race are here in force, including Intel, AMD, Qualcomm, and Microsoft. Perhaps Intel made the most noise at the event by hosting a press conference to highlight its latest consumer chip targeting laptops and AI workloads, the Intel Core Ultra. Intel launched this part back in December, but executive vice president and GM of the client group at Intel, Michelle Johnston Holthaus used CES to highlight key system partners, talk more about the company’s leadership position in software, and showcase some new demos of the AI ecosystem in action.

There is no denying that Intel has the footprint to drive impact for the upcoming “super cycle” of PC upgrades that some analysts are calling for, with its sheer scale of units (Holthaus said they have already shipped millions of the processors to customers) and significant investment in channel and marketing activations. Intel is pushing the narrative now that it has such a significant lead in AI PC software, including AI model support, developer engagement, and applications that run on the integrated NPU (neural processing unit), that only Intel can bring AI to consumers “ready, out of the box.”

AMD released a pre-recorded “special address” this week to talk about its advancements in the AI PC space, including more details on its Ryzen 8040-series of chips that also include an integrated NPU for accelerating AI processing. AMD went as far as showing new comparative benchmarks for its chips coming out in the next 60 days or so with Intel’s new Core Ultra product family. As you would expect, the comparisons are favorable, both in the AI results shown and the integrated graphics performance, attempting to dissuade the industry from thinking Intel’s latest chip is leaving AMD in the dust.

But since the “AI PC” mantra started sometime in the middle of last year, there has been one name curiously missing from the debate: NVIDIA. The company is hoping to change that mindset by putting some serious muscle of its own behind marketing the running of AI on the PC. After all, NVIDIA has history on its side: it was the hardware vendor that essentially paved the road that the AI markets are all driving down today when it introduced CUDA, programmable GPU chips, and created a software ecosystem rivalling anyone in the industry.

With NVIDIA reaching its $1 trillion valuation on the back of the growth of the AI market in the data center, selling GPUs that cost tens of thousands of dollars to companies training the biggest and most important AI models, there is reason to question what NVIDIA gains by spending effort in this AI PC race. The most basic benefit is selling more GPUs into the consumer space, in particular for machines that don’t already have a separate GPU as part of the standard build. Gaming laptops and gaming PCs always have a graphics card in them, but more mainstream devices tend to exclude them for cost considerations. If NVIDIA can make the case that any true AI PC has a GeForce GPU in it, that translates into more sales across the spectrum of price points.

Other benefits include maintaining NVIDIA GPU chips as the bedrock of which the next great AI application is built on. NVIDIA achieved this in the first wave of AI, ensuring that it and its software stack were the default choices for developers of tools running locally or in the cloud. It’s also key that NVIDIA can convince the industry and investors that the advent of the NPU, pushed by Intel, AMD, and Qualcomm, isn’t going to lead to a fundamental shift in the AI compute market.

What makes the NVIDIA GPU argument for the AI PC so interesting is the raw performance that it brings to the table. While the integrated NPU on the Intel Core Ultra platform offers roughly 10 TOPS (tera-operations per second, a standard way of talking about AI performance), a high-end GeForce GPU can supply more than 800 TOPS. Obviously an 80x improvement in AI computing resources means that the capability to build innovative and revolutionary AI application is a lot higher! Even mainstream discrete GPUs from NVIDIA will offer multiple times more compute capability than the NPUs that are coming out on the traditional “AI PC” chips we often hear about.

And these kinds of GPUs are not just on desktop machines but are available in the notebook space as well. This means that the laptop “AI PC” doesn’t HAVE to be one powered by just an Intel Core Ultra, but can also include a high-performance GPU for the most intense AI applications.

Graphics chips have been the basis for AI application development from the beginning, and even the generative AI push that has skyrocketed the popularity of AI in the consumer space, runs best on NVIDIA hardware today. Local Stable Diffusion tools that can create images from text prompts, all default to utilizing NVIDIA GPU hardware, but require careful tweaking and inclusion of specialized software modules to run effectively on Intel or AMD NPUs.

NVIDIA had a couple of demos at its showcase that impressed and drove the point home of how it sees the world of the AI on the PC. First was a tool it worked on with a company called Convai to change how game developers create and how gamers interact with non-player characters in a game or virtual world. Essentially, the implementation allows a game dev to use a large language model like ChatGPT, adding in some bits and flavor about a character’s background, characteristics, likes and dislikes, to generate a virtual character with a life-like personality. A gamer can then talk into a microphone, having that converted to text via another AI model, sent to the game character like a modern AI-based chatbot, getting a response that is converted into speech and animation in the game. 

I watched multiple people interact with this demo, challenging an AI character with different scenarios and questions, and the solution worked amazingly well, and amazingly quickly, enabling a real-time conversation with an AI character that is game and context aware. And this AI computing happens partially on the local gaming machine and its GPU, and partially in the cloud on a collection of NVIDIA GPUs; truly a best case scenario for NVIDIA.

Another demo used the power of the GPU in a desktop system to build a personalized ChatGPT-like assistant, using an open-source language model like Llama2 from Meta and then simply pointing it to a folder full of personal documents, papers, articles, and more. This additional data “fine tunes” the AI model and allowed the user to have a conversation or ask questions of the chatbot based on that data that included personal emails and previous writings. It was only a tech demo and not ready for any kind of consume release, but this is one of the promises that an AI PC addresses and, in this instance, it’s all running on an NVIDIA GPU.

There are tradeoffs of course. Most of the time, the NVIDIA GeForce GPUs in a laptop or a desktop system are going to use a lot more power than the integrated NPU on a chip like the Intel Core Ultra. That means it can impact battery life and the heat a system generates when doing AI work. For sustained AI processing, like converting your background in a video call with coworkers, that isn’t a good solution as it means power consumption, heat, and fan noise. But for AI work where you need an output quickly, the power of discrete GPUs is going to help you get that work done faster. Need to quickly create 10 different images of an office environment with busy programmers using generative AI? Then a discrete GPU is going to get that done and get you on to your next work task the fastest.

I have no doubts that AI is going to transform how we interact with and use our PCs for the better, and SOONER than many believe possible. There is going to be a spectrum of solutions to enable this, from the low power integrated NPU on Intel, AMD, and Qualcomm’s newest laptop chips, to the high-performance GPUs from NVIDIA and AMD, to cloud and edge connected computing. All of these options will be blended to provide the best consumer experiences, and in some instances, you may not even know where the compute is happening for any given action. But it is clear that having discussions about the “AI PC” revolution on our doorstep without NVIDIA is a pretty big miss.