filister@lemmy.world to Technology@lemmy.worldEnglish · 2 days agoDeepSeek R1 Is Reportedly Running Inference On Huawei's Ascend 910C AI Chips, Showing China's Growing AI Capabilitieswccftech.comexternal-linkmessage-square13fedilinkarrow-up10arrow-down10
arrow-up10arrow-down1external-linkDeepSeek R1 Is Reportedly Running Inference On Huawei's Ascend 910C AI Chips, Showing China's Growing AI Capabilitieswccftech.comfilister@lemmy.world to Technology@lemmy.worldEnglish · 2 days agomessage-square13fedilink
minus-squareCorkyskog@sh.itjust.workslinkfedilinkEnglisharrow-up0·2 days agoWait, so after you train, you don’t need all those fancy Nvidia chips? They should make one place where there is an overabundance of geo thermal energy and train all models there…
minus-squareSavvyBeardedFish@reddthat.comlinkfedilinkEnglisharrow-up0·2 days agoYes, so R&D and finalizing the model weight is done on NVIDIA GPUs (I guess you need an excessive amount of VRAM). Inference is probably gonna be offloaded to consumers in the end where the NPU is taking care of the inference cost (See Apple, Qualcomm etc)
minus-squareRichard@lemmy.worldlinkfedilinkEnglisharrow-up0·2 days agoYes. You still need similar ones if you want to run the models really fast, but not nearly the same amount or cost. That’s how people run LLMs on their laptops. You don’t even need a GPU, a multi-core CPU is sufficient, just not very fast at it.
Wait, so after you train, you don’t need all those fancy Nvidia chips?
They should make one place where there is an overabundance of geo thermal energy and train all models there…
Yes, so R&D and finalizing the model weight is done on NVIDIA GPUs (I guess you need an excessive amount of VRAM).
Inference is probably gonna be offloaded to consumers in the end where the NPU is taking care of the inference cost (See Apple, Qualcomm etc)
Yes. You still need similar ones if you want to run the models really fast, but not nearly the same amount or cost. That’s how people run LLMs on their laptops. You don’t even need a GPU, a multi-core CPU is sufficient, just not very fast at it.