Energy efficiency: The decisive metric for the future of AI

Electricity has established itself as the strictest limit for the development of infrastructures aimed at artificial intelligence. In a scenario where the demand for processing grows rapidly, the number of tokens that an AI factory can generate within a fixed energy budget determines its real profitability. Therefore, the performance per watt has become the fundamental metric, being an indicator that cannot be manipulated and that reflects results obtained under real operating conditions.

As demand for agent AI increases, strategic decisions made by organizations today will define who will have the ability to scale in an energy-constrained world. Efficiency is not just a technical goal, but the pillar that supports the economic viability of any large-scale operation in the technology sector.

The evolution of architectures and the scale of domains

Currently, the vast majority of cutting-edge models use the Mixture-of-Experts (MoE) architecture. To process these structures efficiently, the size of the GPU domain — which represents the number of units connected by a very high-speed network — has become a determining factor. While the previous generation set the standard with eight GPUs, the scale of today's needs has surpassed that capacity, requiring much more integrated systems.

A platform NVIDIA Blackwell NVL72 exemplifies this transition by utilizing domains of 72 GPUs. This change allows for optimizations that dramatically increase efficiency, demonstrating that performance on MoE models improves significantly as the domain increases. This technological level serves as the basis on which the future platform NVIDIA Vera Rubin will be built, focusing on further increasing rack-level efficiency.

Optimization through co-design

Achieving higher levels of efficiency requires a rigorous engineering effort that integrates hardware and software from conception. This co-design process ensures that crucial components like the NVLink Switch are developed specifically to handle heavy data traffic rather than being adapted from general-purpose networking technologies. The integration allows the system to perform computation directly on the switch, reducing the workload on the GPUs.

Furthermore, software plays a vital role in maximizing the performance per watt over time. Modern libraries allow you to apply advanced techniques, such as NVFP4 quantization and intelligent routing, which multiply the delivery capacity of each processing unit. It was even observed that continuous software optimizations can significantly increase the efficiency of specific models in short periods of time.

Intelligent management and reliability

In large data centers, power losses caused by cooling systems and distribution inefficiencies can waste a considerable portion of electricity. Platforms like NVIDIA DSX MaxLPS aim to close this gap by adjusting power consumption between racks and GPUs in real time. This management allows operators to install up to 40% more processors within the same energy limit available in the environment.

Rack-scale reliability is a challenge that only production experience can overcome. Systems of this magnitude face failures that do not occur in single-node implementations, requiring technical rigor and operating time under real traffic. Companies and reference laboratories use this AI infrastructure to ensure that theoretical efficiency gains translate into solid profit margins and daily operational stability.

FAQ

• Why is performance per watt the most important metric? It defines profitability by maximizing token production within a fixed energy budget.
• What is the advantage of larger GPU domains? They enable greater efficiency in MoE models, reducing the cost per token generated and optimizing scale.
• Can software increase the efficiency of already installed hardware? Yes, continuous optimizations in libraries allow you to extract more performance without the need to change physical components.
• How does refrigeration impact efficiency? Inefficient systems waste energy; the use of intelligent load management allows for greater processing density.

Also read: Startups use AI to create military software prototypes.

Source and methodology

This article was prepared based on information published by blogs.nvidia.com, on July 14, 2026. See the publicação original: Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency. HTechBD reorganized and contextualized the data for the Brazilian public, without reproducing the source text.

Image: panumas nikhomkhai no Pexels.