MR. Memory Report

Glossary

Training vs inference

Updated July 04, 2026

training-inference-chip.jpeg

Training is when an AI model learns, chewing through massive datasets in giant data center runs. Inference is the model actually answering you afterward, over and over, every day. Both are memory hogs. Training leans hard on HBM, the ultra fast stacked memory bolted next to AI chips. That HBM market is already $54.6B, up 58% year over year. Inference then scales that demand across billions of daily queries. More AI use means more memory, structurally.

More from the glossary

What is a hyperscaler? Jevon's Paradox
← The full explainer Daily briefs →