I’m looking at people running Deepseek’s 671B R1 locally using NVME drives at 2 tokens a second. So why not skip the FLASH and give me a 1TB drive using a NVME controller and only DRAM behind it? The ultimate transistor count is lower on the silicon side. It would be slower than typical system memory but the same as any NVME with a DRAM cache. The controller architecture for safe page writes in Flash and the internal boost circuitry for pulsing each page is around the same complexity as the memory management unit and constant refresh of DRAM and it’s power stability requirements. Heck, DDR5 and up is putting power regulation on the system memory already IIRC. Anyone know why this is not a thing or where to get this thing?
You can get motherboards with enough slots if you’re willing to pay enterprise prices for them. I have a system with 1TB of RAM at work that I use as a fast data cache. I just mount tmpfs on it, write hundreds of gigs of data into it (overwriting it all every few hours), and it works great. Cost was somewhere in the $10~15K (US) range a few years ago, IIRC. Steep for an individual, sure, but not crazy for an organization.
Last I saw you had to use super premium ultra dense memory modules to get 1tb into a motherboard. Maybe that’s less so now. But the hope would be to use commodity ram and CPUs etc. 10k for a 1tb system is pretty good. Last I looked it was a lot more.
Pretty sure you can do much better than that now (plus or minus tariff insanity) – quick check on Amazon, NewEgg, etc. suggests ballpark of $5K for 1TB RAM (Registered DDR4) + probably compatible motherboard + CPU.