AMD's newest AI chip, the Instinct MI300X, could give big tech buyers leverage over Nvidia, if it works as advertised and is price-competitive.
      • Aniki 🌱🌿@lemm.eeEnglish
        0·
        11 months ago

        I haven’t super looked into it but I’m not interested in playing the GPU game against the gamers so if AMD can do a Tesla equivalent with gobs of RAM and no display hardware I’d be all about it.

        Right now it’s looking like I’m going to build a server with a pair of K80s off ebay for a hundred bucks which will give me 48GB of RAM to run models in.

        • dublet@lemmy.worldEnglish
          0·
          11 months ago

          Some of the LLMs it ships with are very reasonably sized and still be impressive. I can run them on a laptop with 32GB of RAM.

          • Aniki 🌱🌿@lemm.eeEnglish
            0·
            11 months ago

            This is very interesting! Thanks for the link. I’ll dig into this when I manage to have some time.