Is Nvidia well positioned to sustain its meteoric growth in AI?

Not a surprise to any, it turns out that Nvidia made a lot of money last quarter. The company reported $18.1B in revenue, an increase of 206% compared to just one year ago. Earnings per share were up 6 times or 12 times, depending on if you view at the GAAP or non-GAAP numbers. Nvidia has been raising eyebrows with investors and analysts in the markets for some time, and yesterday’s results continue to do so.

The driver of this growth is its data center segment, responsible for the GPU (graphics processing units) and chips that have been powering high performance computing and the AI boom. As recently as a year ago, Nvidia had a year-on-year revenue decrease in this segment (Q3 to Q4 FY23). But since that reporting, revenue in this segment has jumped 4 times through four quarters and more than 3 times in just the last six months, now bringing in $14.5B of that $18B in revenue.

For comparison – the company’s gaming division, which as recently as FY22 was a higher revenue share than data center – had a revenue increase of 55% year-on-year in yesterday’s results. But it represents just $2.8B of its gross revenue.

The stunningly fast growth for Nvidia to a trillion-dollar company (over $1.2T now) thanks to this AI boom and need for processing chips to handle it is starting to raise questions. Can this growth continue? Are there other risks to this meteoric rise? The market seemed to react tepidly after hours last night with the stock initially dropping more than 4% but then settling in down just under 1%.  

Can the growth in the data center business continue? Yes.

This is the most important question that investors and analysts must consider. With data center segment revenue going from $4.2B to $14.5B in just two quarters for Nvidia, can this kind of growth continue for the segment as a whole, let alone Nvidia specifically? This question came up during the earnings call last night and CEO Jensen Huang was bullish, as expected, saying that he sees growth in this space through 2025 at least.

There seems to be a common theory that we are nearing a saturation point in the compute needed for AI or that we might be near the end of the AI training (the compute-intensive process of building an AI model) cycle. I think this is incredibly shortsighted thinking. For the foreseeable future, AI training will never be “done” and the need for more expansive, more accurate, and more customized models will continue to expand. ChatGPT and current generative AI applications are really just the start of this revolution, and tools like Microsoft Copilot are just starting to unveil the potential for AI usages. 

Huang described what he calls the creation of “AI factories” that enable enterprises, governments, and infrastructure developers to develop their own AIs, tailored to and specific to different needs. These will provide some of the safety and security needed for inclusion of proprietary and personal data. This vision paints a future where the need for additional AI processing continues to ramp, not one that stabilizes or decreases.

One piece of this puzzle comes from the recently enacted restrictions on shipping GPUs to China. Nvidia reported yesterday that its sales for Q4 and into calendar year 2024 will be impacted, as China and the other restricted regions represented 20-25% of its total sales. But offsetting that is growth in other parts of the world, Nvidia says, and it makes sense considering the company has been “sold out” of chips for some time. If a customer in China can’t buy those H200 AI chips, I’m sure a suitor is out there in Europe or the US.

Nvidia is apparently working on custom designs of its GPUs for the China market that will uphold the performance restrictions from the US government, but it will take a few months for those to start to filter out and represent notable revenue.

Does Nvidia lose its dominant position as the AI world moves from training to inference? No.

AI training, or the use of high performance super computers to build complex AI models like Llama and GPT, has been the primary driver of the market for AI computing over the last 5-10 years. As researchers learned how to best create these models and the size of the inputs has increased to make them more useful, the need for hundreds of thousands of GPUs came along with it.

AI inference is the application of those AI models. Once GPT has been trained, then a company like OpenAI can create a user-facing application like ChatGPT that uses the model to “infer” answers based on it and input from the user. The question has been asked if Nvidia’s growth in AI could be impacted by the move from training-focus to inference-focused markets and as AI gets integrated into applications.

Perhaps the best examples of AI inference at the corporate or consumer level today are ChatGPT and Adobe’s Firefly. Both are generative AI solutions, creating new content based on inputs from the user; text and analysis from ChatGPT and text-to-image for Firefly. And both are utilizing massive numbers of GPUs for AI processing in the cloud.

I expect that GPU usage to continue going forward. Nvidia’s AI chips are very performant in inference workloads, not just training, and the company has a huge advantage since basically all AI developers are writing and testing code on Nvidia GPUs and its CUDA software development stack. Any competitor in this arena not only has to compete at the hardware level, but with a software layer that can offer efficient development and reliability – no easy task.

One potential area to watch is on-device AI processing. As users start demanding more AI applications on their laptops, PCs, and smartphones, companies like Intel, Qualcomm, and AMD are ramping up performance of their own consumer chips for AI. Qualcomm recently showed off its Snapdragon X-series processors, AMD is touting its Ryzen AI technology, and Intel is set to launch its Core Ultra processors next month, all of which include some form of dedicated AI processing on-chip. If this model of AI processing can catch on, then it might diminish the need for Nvidia’s AI chips in the data center.

Are there competitive AI chips to worry about? Maybe.

If anything should worry Nvidia and its investors, it is competition in the AI chip space. A $1.2T market cap breeds competitors in many different forms.

For its part, during the investor call yesterday Nvidia did talk about how it plans to increase its product launch cadence, going from a 24-month release cycle to a 12-month one. That means the company will be bringing new chips to market faster, with more performance and more features with each launch. Clearly the company understands it cannot sit idly and let competitors sneak up.

AMD represents the biggest competitive threat in the short term. It’s GPUs have been the second choice for many years, both in the PC market and in the data center. They are based on similar designs and architecture, though they are not as closely aligned as Intel and AMD PC processors are; there is still considerable work that has to be done on the software side to migrate from a CUDA development stack. AMD’s recent MI300 family of AI chips looks to be ramping well, with a strong announcement of support from Microsoft for Azure cloud implementations and CEO Lisa Su is confident in it adding $1B in revenue very quickly.

The custom AI accelerator market was recently made more interesting with the announcement of the Microsoft Maia 100 chip, but also includes chips built by Meta, Amazon AWS, and startups like Groq. These options have the potential to offer compelling advantages over Nvidia chips like higher energy efficiency and better performance thanks to the ability to customize the silicon for specific workloads and algorithms. It can also offer a cost advantage in the long run as it removes the need to help fill out that 75% margin that Nvidia reported yesterday.

The challenge presented by Intel is an interesting one. While its GPU development work has struggled to gain market share in the years after it hired (and then lost) Raja Koduri, it is focused on its Gaudi branded family of chips that are dedicated for AI processing, acquired with the purchase of Habana Labs back in 2019. These chips are proving to be competitive in several areas of the AI segment, but displacing Nvidia in the data center continues to be a struggle.

Intel could turn out to be a partner for Nvidia in the AI race if it can get its foundry services ramped up, providing an alternative to TSMC for the manufacturing of these massive chips. This would offer pricing negotiation advantages and provide additional capacity that has limited Nvidia so far.