Nvidia Stock Tumbles. IBM and Supermicro AI Deal Delivers 56x Speed Boost

Nvidia NVDA shares fell 6% in morning trading Thursday despite a new collaboration with Super Micro Computer SMCI and IBM IBM put the spotlight on efforts to improve the speed of AI inference systems.

The companies have developed a storage system that allows AI applications to reuse previously processed information instead of repeatedly calculating it. The technology combines Supermicro's flash storage servers, IBM software and Nvidia's Dynamo inference framework and Spectrum-X networking, according to a Thursday press release.

Testing involving a 120-billion-parameter AI model showed the system delivered up to 56 times faster Time-to-First-Token performance with prompts containing 130,000 tokens. This metric measures how long an AI system takes to generate its first response after receiving a prompt.

The technology uses persistent key-value cache storage to retain information generated during earlier processing. This could help AI systems handle longer conversations and complex workloads without relying solely on GPU and CPU memory. The companies also tested performance under heavy network traffic and simultaneous user requests, with the system designed to share cached data across multiple GPU clusters.