New AI Architecture Demonstration Achieves 50% Speedup Using Nvidia GPUs
What happened
At the Open Compute Project Global Summit, a live inference pipeline demonstrated Attention-FFN Disaggregation using Nvidia H100 GPUs and Corsair XPUs. The pipeline routed attention layers to the GPUs and expert feed-forward layers to the XPUs. This configuration reduced time-per-output-token by 50.6 percent compared to a standalone H100 on the GPT-OSS-20B model.
From techtimes.com
Why it matters
This result confirms that Attention-FFN Disaggregation is moving from academic design to production architecture. The technique routes different computational tasks to specialized hardware, which could enhance the efficiency of large language model serving and computation.
From techtimes.com
Who's involved
- NvidiaProvided the Nvidia H100 GPUs used in the live demonstration.
- GoogleWas involved in the research and application of the new architecture.
- IntelWas involved as a researcher confirming the new concept.
- Alphabet Inc.Is the parent company of Google and was involved in the research.
Who could feel it
Possible knock-on effectsThese are possibilities Brind reasoned out, not predictions, and not advice. Most are not stated in any report.
Possible ripples
Who this could reach next, traced step by step from the reporting. Possibilities worth considering, not forecasts.- Nvidia's new architecture increases pressure on AMD's AI hardwareNvidia's new architecture increases pressure on AMD's AI hardwareLikely
- New AI architecture may boost demand for Astera Labs' connectivityNew AI architecture may boost demand for Astera Labs' connectivityPossible
- New AI inference architecture could boost CoreWeave's offeringsCoreWeave may benefit from new AI inference speedupPossible
- New AI architecture may boost TSMC's advanced chip demandNew AI architecture may boost TSMC's advanced chip demandPossible
- New AI architecture could drive demand for Micron's HBM memoryNew disaggregated AI architecture may boost demand for high-bandwidth memory.Possible
Keep exploring
The entities involved
-
Intel
American multinational technology company
-
Google
American multinational technology company, a subsidiary of Alphabet Inc.
-
Nvidia
American multinational technology company
- The rise of perpetual futures is attracting established financial players, allowing small companies to achieve giant market size, while centralized exchanges face challenges from decentralized markets.
- Kraken and Coinbase filed stock perpetual products, making Nvidia and Tesla stocks eligible for new perps.
Related events
- Anthropic filed an S-1 registration statement with the SEC, noting its reliance on Nvidia GPUs for AI model development.
- The event covers Nvidia/OpenAI data center negotiations in Ohio, alongside state-backed computing projects in China.
- Paycom and Paychex operate in the finance and HR software segment, while Nvidia's chips require specialized infrastructure.
- Cisco and Nvidia are collaborating on an AI factory, with Cisco supplying the necessary networking stack for Nvidia's silicon.
- Nvidia and Intel are competing in the established CPU market.