Charles Fan, CEO and Co-founder of MemVerge, emphasized the critical importance of overcoming the bottleneck of HBM capacity. "Scaling LLM performance cost-effectively means keeping the GPUs fed with data," stated Fan. "Our demo at GTC demonstrates that pools of tiered memory not only drive performance higher but also maximize the utilization of precious GPU resources."
The demonstration, conducted by engineers from MemVerge and Micron featured a FlexGen high-throughput generation engine and OPT-66B large language model running on a Supermicro Petascale Server equipped with an AMD Genoa CPU, Nvidia A10 GPU, Micron DDR5-4800 DIMMs, CZ120 CXL memory modules, and MemVerge Memory Machine™ X intelligent tiering software.
The results of the demonstration were impressive. The FlexGen benchmark, utilizing tiered memory, completed tasks in less than half the time compared to conventional NVMe storage methods. Simultaneously, GPU utilization soared from 51.8% to 91.8%, thanks to the transparent management of data tiering across DIMMs and CXL modules facilitated by MemVerge Memory Machine X software.
This collaboration between MemVerge, Micron, and Supermicro marks a significant milestone in advancing the capabilities of AI workloads, enabling organizations to achieve unprecedented levels of performance, efficiency, and time-to-insight. By harnessing the power of CXL memory and intelligent tiering, businesses can unlock new opportunities for innovation and accelerate their journey towards AI-driven success.
"Through our collaboration with MemVerge, Micron is able to demonstrate the substantial benefits of CXL memory modules to improve effective GPU throughput for AI applications resulting in faster time to insights for customers. Micron's innovations across the memory portfolio provide compute with the necessary memory capacity and bandwidth to scale AI use cases from cloud to the edge," said Raj Narasimhan, senior vice president and general manager of Micron's Compute and Networking Business Unit.
For more information about MemVerge, Micron, and Supermicro's collaborative efforts and the transformative potential of CXL memory for AI workloads, please visit www.memverge.com/gtc2024
About MemVerge
MemVerge is a pioneering developer of Big Memory Computing software. In the cloud or on-premises, data-intensive workloads run faster, cost less, and recover automatically with the company's award-winning Memory Machine™ products. Memory Machine X is poised to revolutionize how CXL® memory will be used in the future, while Memory Machine Cloud stands out with its ability to continuously rightsize cloud cost and performance. Learn more about MemVerge and its Memory Machine software at www.memverge.com.
Media Contacts:
Frank Berry
MemVerge
Email Contact
949.300.8917
View original content to download multimedia:
https://www.prnewswire.com/news-releases/memverge-and-micron-boost-nvidia-gpu-utilization-with-cxl-memory-302090960.html
SOURCE MemVerge
| Contact: |
| Company Name: MemVerge
|
