Why You Care
Ever wonder why your AI models sometimes run slower than expected, even with GPUs? What if a significant portion of that computing power is just… wasted? A new startup, Niv-AI, is tackling this very problem, promising to unlock previously squandered performance from your expensive AI hardware. This creation could mean more efficient AI operations and lower costs for anyone relying on large-scale AI processing.
What Actually Happened
Niv-AI, a Tel Aviv-based startup, recently exited stealth mode, according to the announcement. Their mission is to improve the power performance of Graphics Processing Units (GPUs) in data centers. These chips are the backbone of artificial intelligence, but their intense and fluctuating power demands create significant challenges for data center operators. Currently, data centers often have to “throttle down” their GPU usage by as much as 30% to manage power, as mentioned in the release. This throttling happens because GPUs switch rapidly between tasks, causing millisecond-scale power surges. These surges make it hard for data centers to maintain a stable power supply from the grid. To cope, they either pay for expensive temporary energy storage or reduce GPU output. Both options decrease the return on investment for costly AI chips.
Why This Matters to You
This new system directly addresses a major bottleneck in AI creation and deployment. If you’re running AI workloads, whether for research or commercial applications, power efficiency directly impacts your bottom line and computational capacity. Imagine being able to utilize nearly all of your purchased GPU power, instead of just a fraction. The company reports this could significantly improve operational efficiency.
Here’s how Niv-AI plans to make a difference:
- Rack-level sensors: These sensors detect power usage at a millisecond level on GPUs.
- Data collection: The goal is to understand specific power profiles of different deep learning tasks.
- Mitigation techniques: They aim to develop solutions that allow data centers to use more of their existing capacity.
- AI copilot: An AI model will predict and synchronize power loads across the data center.
Think of it as a smart energy manager for your AI infrastructure. Instead of blindly over-provisioning power or under-utilizing hardware, your systems could dynamically adjust. This leads to substantial savings and improved performance. As Jensen Huang, CEO of Nvidia, stated during a keynote speech, “There is so much power squandered in these AI factories.” He further emphasized, “Every unused watt is revenue lost.” How much revenue might your organization be losing due to inefficient power management?
The Surprising Finding
What’s truly surprising is the sheer scale of the problem. Despite the nature of AI hardware, data centers are routinely forced to underutilize their GPUs significantly. The company proclaimed during an annual presentation that “Every unused watt is revenue lost.” This highlights a fundamental inefficiency. It challenges the common assumption that simply adding more GPUs automatically translates to proportional gains in AI performance. Instead, the power grid’s limitations and the chips’ dynamic demands create a hidden ceiling on actual usable capacity. Lior Handelsman, a partner at Grove Ventures and a Niv board member, noted, “We just can’t continue building data centers the way we build them now.” This suggests that current data center designs are not sustainable for the future of AI.
What Happens Next
Niv-AI is currently deploying rack-level sensors to gather crucial data. The team revealed they are working with design partners to understand power profiles for deep learning tasks. They expect to have an operational system in a handful of U.S. data centers within the next six to eight months. This means we could see real-world impact by late 2026 or early 2027. For example, imagine a large language model training facility that can now train models 20% faster without needing to purchase additional hardware. This system could set a new standard for data center design and operation. The industry implications are significant, pushing towards more sustainable and cost-effective AI infrastructure. Your future AI projects might benefit from these efficiencies, allowing for more ambitious undertakings without spiraling energy costs.