The tech world is buzzing with the rise of a new cool tech king, a company that is not selling any gizmos but instead pushing the boundaries of hardcore technology.
Nvidia has ascended the become the coolest kid on the block, crafting semiconductors that can conquer the most gargantuan GenAI tasks without breaking a sweat.
Nvidia’s latest offering is the Blackwell graphics processing unit (GPU), christened B200. The cutting-edge processor promises to deliver a staggering 30 times more inference throughput and four times the training performance over its predecessor, the H100 GPU.
In anticipation of insatiable demand for computational power, Nvidia also created a three-die superchip that couples two Blackwell GPUs with its Grace CPU.
The result? A massively powerful AI processor dubbed the GB200 Grace Blackwell that promises to redefine the boundaries of what is possible in GenAI.
The technological tour de force was unveiled by Nvidia’s chief executive Jensen Huang at the company’s annual developer conference, GTC, on March 19 in the United States.
The event, which has become the AI Woodstock, a landmark gathering for those pushing the frontiers of AI, drew a staggering 11,000 developers to the SAP Centre in San Jose.
Huang took to the stage in his trademark black leather jacket and spoke for two hours in a tech-heavy keynote packed with groundbreaking announcements.
Beyond the hardware, there was a suite of new AI software and tools, solidifying Nvidia’s pole position in AI infrastructure.
Nvidia has emerged as a dominant force in the AI and GenAI boom, thanks to its early start producing GPUs for computer gaming.
As it found its footing in AI in the mid-2010s, it developed its tech stack comprising powerful GPUs and ecosystem of software, tools and partnerships.
Blackwell, which contains more than 200 billion transistors, enables organisations to build and run real-time generative AI on trillion-parameter large language models at up to 25 times less cost and energy consumption than its predecessor.
Huang said that the core of the processor was “pushing the limits of physics of how big a chip could be”. Hence to get more power, Nvidia combined the power of two Blackwells, offering speeds of 10TB/s.
The superchip GB200 Grace Blackwell can process a trillion-parameter model with constant uptime for superscale GenAI training and inference workloads.
Used in data centres, the GB200 will be packed into a liquid cooled NVL72 rack, capable of holding 72 Blackwell GPUs and 36 Grace CPUs. A single rack can train models with up to 27 trillion parameters, letting companies take on the biggest generative AI challenges.
Nvidia poured US$10 billion into developing Blackwell, which will cost between US$30,000 and US$40,000 each. The chip will be available later this year.
Industry analysts report that the impact of Blackwell’s performance could lead to breakthroughs in everything from data processing and engineering simulations to electronic design automation and computer aided drug design.
Already, demand is high, with some analysts reporting that Blackwell is sold out into 2025. Major companies like Amazon Web Services, Dell Technologies, Google, Meta, Microsoft, OpenAI, and Tesla are among the customers using Blackwell.
Rivals, however, want in on the AI action. AMD announced its most advanced MI300 chips late last year as a challenger to Nvidia’s H100. Intel has fallen behind in this AI acceleration race but it is offering its service as a chip manufacturing facility.
Intel’s AI Foundry aims to cater to the growing demand for GPUs by offering its manufacturing expertise and technology optimised for AI workloads. Of course, it hopes that Nvidia will use its facility.
Meanwhile, each of Nvidia’s big customers such as Google, Amazon and Microsoft have their own chip development as they seek to reduce their reliance on Nvidia.
There are other benefits. Building their own processors could be cheaper. Microsoft and Alphabet can tailor their in-house chips to address specific needs of their AI workloads, potentially leading to better performance and efficiency.
Sam Altman, founder of OpenAI, and an early supporter of Nvidia’s GPU, is seeking US$7 billion for a new semiconductor company for the similar reasons.
Nvidia, meanwhile, is spreading its business wings. It is partnering governments and companies across the world to power innovations and to provide GPU resources to drive research in areas like climate change and 6G telco technology.
Singapore telco Singtel will launch a GPU-as-a-service in Southeast Asia in the third quarter of this year. The on-demand service will be deployed through Nvidia H100 GPU-powered clusters that are operated in Singtel’s existing upgraded data centres in Singapore.
This service will be expanded to run in its three upcoming sustainable AI data centres across Singapore, Thailand and Indonesia when they begin operations.
Malaysian company YTL Power International will launch its first AI cloud using the new Blackwell chips. The YTL AI Supercomputer is expected to surpass more than 300 exaflops of AI computing, making it one of the fastest supercomputers in the world.
Besides its new GPUs, Nvidia is also extending its market lead with other tools. For example, it has come up with production-ready APIs (application programming interfaces) designed to streamline the deployment of AI models.
Residing in the cloud, these are pre-configured tools to help software developers minimise setup complexity and significantly reduce deployment times.
Meanwhile, Earth-2 is a new digital twin cloud platform to help meteorologists and weather experts create more detailed and nuanced simulations, potentially leading to earlier and more accurate weather reports.
Organisations across the world are interested in this tool. The Taiwan Central Weather Administration aims to use it for better detection of typhoon landfall.
Then there is Nvidia’s Apple collaboration. Nvidia’s Omniverse, a real-time 3D graphics collaboration platform will enable Apple Vision Pro users to with a more immersive and realistic 3D experience by offloading heavy processing tasks to the cloud.
At GTC, the demonstration showed a fully interactive model of a car streamed into Apple Vision Pro VR headsets.
By offloading the heavy processing tasks to the cloud, the Vision Pro itself doesn’t need to be as powerful, potentially leading to a more comfortable design or lower production cost.
