• tooclose104@lemmy.ca
            link
            fedilink
            arrow-up
            0
            ·
            5 months ago

            Apparently phone too! Like 3 cards down was another post linking to instructions on how to run it locally on a phone in a container app or termux. Really interesting. I may try it out in a vm on my server.

        • boomzilla@programming.dev
          link
          fedilink
          arrow-up
          0
          ·
          edit-2
          5 months ago

          I watched one video and read 2 pages of text. So take this with a mountain of salt. From that I gathered that deepseek R1 is the model you interact with when you use the app. The complexity of a model is expressed as the number of parameters (though I don’t know yet what those are) which dictate its hardware requirements. R1 contains 670 bn Parameter and requires very very beefy server hardware. A video said it would be 10th of GPUs. And it seems you want much of VRAM on you GPU(s) because that’s what AI crave. I’ve also read 1BN parameters require about 2GB of VRAM.

          Got a 6 core intel, 1060 6 GB VRAM,16 GB RAM and Endeavour OS as a home server.

          I just installed Ollama in about 1/2 an hour, using docker on above machine with no previous experience on neural nets or LLMs apart from chatting with ChatGPT. The installation contains the Open WebUI which seems better than the default you got at ChatGPT. I downloaded the qwen2.5:3bn model (see https://ollama.com/search) which contains 3 bn parameters. I was blown away by the result. It speaks multiple languages (including displaying e.g. hiragana), knows how much fingers a human has, can calculate, can write valid rust-code and explain it and it is much faster than what i get from free ChatGPT.

          The WebUI offers a nice feedback form for every answer where you can give hints to the AI via text, 10 score rating thumbs up/down. I don’t know how it incooperates that feedback, though. The WebUI seems to support speech-to-text and vice versa. I’m eager to see if this docker setup even offers programming APIs.

          I’ll probably won’t use the proprietary stuff anytime soon.

        • Phoenicianpirate@lemm.ee
          link
          fedilink
          English
          arrow-up
          0
          ·
          5 months ago

          Thank you very much. I did ask chatGPT was technical questions about some… subjects… but having something that is private AND can give me all the information I want/need is a godsend.

          Goodbye, chatGPT! I barely used you, but that is a good thing.

        • Alsephina@lemmy.ml
          link
          fedilink
          English
          arrow-up
          0
          ·
          5 months ago

          Unfortunately it’s trained on the same US propaganda filled english data as any other LLM and spits those same talking points. The censors are easy to bypass too.

        • Phoenicianpirate@lemm.ee
          link
          fedilink
          English
          arrow-up
          0
          ·
          5 months ago

          Yeah but China isn’t my main concern right now. I got plenty of questions to ask and knowledge to seek and I would rather not be broadcasting that stuff to a bunch of busybody jackasses.

          • Mongostein@lemmy.ca
            link
            fedilink
            arrow-up
            0
            ·
            5 months ago

            I agree. I don’t know enough about all the different models, but surely there’s a model that’s not going to tell you “<whoever’s> government is so awesome” when asking about rainfall or some shit.

    • CeeBee_Eh@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      I asked it about Tiananmen Square, it told me it can’t answer that because it can only respond with “harmless” responses.

    • Doomsider@lemmy.world
      cake
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      With that attitude I am not sure if you belong in a Chinese prison camp or an American one. Also, I am not sure which one would be worse.

      • RandomVideos@programming.dev
        cake
        link
        fedilink
        arrow-up
        0
        ·
        5 months ago

        They should conquer a country like Switzerland and split it in 2

        At the border, they should build a prison so they could put them in both an American and a Chinese prison

    • UnderpantsWeevil@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      5 months ago

      Not really a question of national intentions. This is just a piece of technology open-sourced by a private tech company working overseas. If a Chinese company releases a better mousetrap, there’s no reason to evaluate it based on the politics of the host nation.

      Throwing a wrench in the American proposal to build out $500B in tech centers is just collateral damage created by a bad American software schema. If the Americans had invested more time in software engineers and less in raw data-center horsepower, they might have come up with this on their own years earlier.

  • wrekone@lemmy.dbzer0.com
    link
    fedilink
    arrow-up
    0
    ·
    5 months ago

    Wait. You mean every major tech company going all-in on “AI” was a bad idea. I, for one, am shocked at this revelation.

  • NutWrench@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    5 months ago

    The “1 trillion” never existed in the first place. It was all hype by a bunch of Tech-Bros, huffing each other’s farts.

    • Zink@programming.dev
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      I don’t have one to cancel, but I might celebrate today by formatting the old windows SSD in my system and using it for some fast download cache space or something.

  • PlutoniumAcid@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    5 months ago

    So if the Chinese version is so efficient, and is open source, then couldn’t openAI and anthropic run the same on their huge hardware and get enormous capacity out of it?

    • Jhex@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      Not necessarily… if I gave you my “faster car” for you to run on your private 7 lane highway, you can definitely squeeze every last bit of the speed the car gives, but no more.

      DeepSeek works as intended on 1% of the hardware the others allegedly “require” (allegedly, remember this is all a super hype bubble)… if you run it on super powerful machines, it will perform nicer but only to a certain extend… it will not suddenly develop more/better qualities just because the hardware it runs on is better

      • merari42@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        5 months ago

        Didn’t deepseek solve some of the data wall problems by creating good chain of thought data with an intermediate RL model. That approach should work with the tried and tested scaling laws just using much more compute.

      • PlutoniumAcid@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        5 months ago

        This makes sense, but it would still allow a hundred times more people to use the model without running into limits, no?

    • AdrianTheFrog@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      5 months ago

      OpenAI could use less hardware to get similar performance if they used the Chinese version, but they already have enough hardware to run their model.

      Theoretically the best move for them would be to train their own, larger model using the same technique (as to still fully utilize their hardware) but this is easier said than done.

    • Yggnar@lemmy.world
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      It’s not multimodal so I’d have to imagine it wouldn’t be worth pursuing in that regard.

    • shawn1122@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      5 months ago

      Deepthink R1(the reasoning model) was only released on January 20. Still took a while though.

    • NoSpotOfGround@lemmy.world
      cake
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      Text below, for those trying to avoid Twitter:

      Most people probably don’t realize how bad news China’s Deepseek is for OpenAI.

      They’ve come up with a model that matches and even exceeds OpenAI’s latest model o1 on various benchmarks, and they’re charging just 3% of the price.

      It’s essentially as if someone had released a mobile on par with the iPhone but was selling it for $30 instead of $1000. It’s this dramatic.

      What’s more, they’re releasing it open-source so you even have the option - which OpenAI doesn’t offer - of not using their API at all and running the model for “free” yourself.

      If you’re an OpenAI customer today you’re obviously going to start asking yourself some questions, like “wait, why exactly should I be paying 30X more?”. This is pretty transformational stuff, it fundamentally challenges the economics of the market.

      It also potentially enables plenty of AI applications that were just completely unaffordable before. Say for instance that you want to build a service that helps people summarize books (random example). In AI parlance the average book is roughly 120,000 tokens (since a “token” is about 3/4 of a word and the average book is roughly 90,000 words). At OpenAI’s prices, processing a single book would cost almost $2 since they change $15 per 1 million token. Deepseek’s API however would cost only $0.07, which means your service can process about 30 books for $2 vs just 1 book with OpenAI: suddenly your book summarizing service is economically viable.

      Or say you want to build a service that analyzes codebases for security vulnerabilities. A typical enterprise codebase might be 1 million lines of code, or roughly 4 million tokens. That would cost $60 with OpenAI versus just $2.20 with DeepSeek. At OpenAI’s prices, doing daily security scans would cost $21,900 per year per codebase; with DeepSeek it’s $803.

      So basically it looks like the game has changed. All thanks to a Chinese company that just demonstrated how U.S. tech restrictions can backfire spectacularly - by forcing them to build more efficient solutions that they’re now sharing with the world at 3% of OpenAI’s prices. As the saying goes, sometimes pressure creates diamonds. Image Image Last edited 4:23 PM · Jan 21, 2025 · 932.3K Views

  • DiaDeLosMuertos@aussie.zone
    link
    fedilink
    English
    arrow-up
    0
    ·
    5 months ago

    I am extremely ignorant of all this AI thing. So please can somebody “Explain Like I’m 5” why can this new thing can wipe off over a trillion dollars in US stock ? I would appreciate it a lot if you can help.

    • Cere@aussie.zone
      link
      fedilink
      arrow-up
      0
      ·
      edit-2
      5 months ago

      Basically US company’s involved in AI have been grossly over valued for the last few years due to having a sudo monopoly over AI tech (companies like open ai who make chat gpt and nvidia who make graphics cards used to run ai models)

      Deep seek (Chinese company) just released a free, open source version of chat gpt that cost a fraction of the price to train (setup) which has caused the US stock valuations to drop as investors are realising the US isn’t the only global player, and isn’t nearly as far ahead as previously thought.

      Nvidia is losing value as it was previously believed that top of the line graphics cards were required for ai, but turns out they are not. Nvidia have geared their company strongly towards providing for ai in recent times.

    • Yozul@beehaw.org
      link
      fedilink
      arrow-up
      0
      ·
      edit-2
      5 months ago

      Let’s say I make a thing. Let’s say somebody offers to buy it from me for $10. I sell it to them, and then let’s say somebody else makes a better thing, and now no one will pay more than $2 for my thing. If my thing is a publicly traded corporation, then that just “wiped off” $8 from the stock market. The person I sold it to “lost” $8. Corporations that make AI and the hardware to run it just “lost” a lot of value.

    • Cynicus Rex@lemmy.ml
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      5 months ago

      "You see, dear grandchildren, your grandfather used to have an apple orchard. The fruits were so sweet and nutritious that every town citizen wanted a taste because they thought it was the only possible orchard in the world. Therefore the citizens gave a lot of money to your grandfather because the citizens thought the orchard would give them more apples in return, more than the worth of the money they gave. Little did they know the world was vastly larger than our ever more arid US wasteland. Suddenly an oriental orchard was discovered which was surprisingly cheaper to plant, maintain, and produced more apples. This meant a significant potential loss of money for the inhabitants of the town called Idiocracy. Therefore, many people asked their money back by selling their imaginary not-yet-grown apples to people who think the orchard will still be worth more in the future.

      This is called investing, children, it can make a lot of money, but it destroys the soul and our habitat at the same time, which goes unnoticed by all these people with advanced degrees. So think again when you hear someone speak with fancy words and untamed confidence. Many a times their reasoning falls below the threshold of dog poop. But that’s a story for another time. Sweet dreams."

  • Ech@lemm.ee
    link
    fedilink
    English
    arrow-up
    0
    ·
    5 months ago

    Hilarious that this happens the week of the 5090 release, too. Wonder if it’ll affect things there.

    • drspod@lemmy.ml
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      Apparently they have barely produced any so they will all be sold out anyway.

      • chiliedogg@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        5 months ago

        And without the fake frame bullshit they’re using to pad their numbers, its capabilities scale linearly with the 4090. The 6090 just has more cores, Ram, and power.

        If the 4000-series had had cards with the memory and core count of the 5090, they’d be just as good as the 50-series.

        • lordnikon@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          edit-2
          5 months ago

          By that point you will have to buy the Mico fission reactor addon to power the 6090. It’s like Nvidia looked at the power triangle of power / price and preformence and instead of picking two they just picked one and to hell with the rest.

  • Treczoks@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    5 months ago

    Looks like it is not any smarter than the other junk on the market. The confusion that people consider AI as “intelligence” may be rooted in their own deficits in that area.

    And now people exchange one American Junk-spitting Spyware for a Chinese junk-spitting spyware. Hurray! Progress!

    • kshade@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      5 months ago

      Looks like it is not any smarter than the other junk on the market. The confusion that people consider AI as “intelligence” may be rooted in their own deficits in that area.

      Yep, because they believed that OpenAI’s (two lies in a name) models would magically digivolve into something that goes well beyond what it was designed to be. Trust us, you just have to feed it more data!

      And now people exchange one American Junk-spitting Spyware for a Chinese junk-spitting spyware. Hurray! Progress!

      That’s the neat bit, really. With that model being free to download and run locally it’s actually potentially disruptive to OpenAI’s business model. They don’t need to do anything malicious to hurt the US’ economy.

    • wulrus@programming.dev
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      With understanding LLM, I started to understand some people and their “reasoning” better. That’s how they work.

    • tetris11@lemmy.ml
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      It is progress in a sense. The west really put the spotlight on their shiny new expensive toy and banned the export of toy-maker parts to rival countries. One of those countries made a cheap toy out of jank unwanted parts for much less money and it’s of equal or better par than the west’s.

      As for why we’re having an arms race based on AI, I genuinely dont know. It feels like a race to the bottom, with the fallout being the death of the internet (for better or worse)

    • UnderpantsWeevil@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      5 months ago

      And now people exchange one American Junk-spitting Spyware for a Chinese junk-spitting spyware.

      LLMs aren’t spyware, they’re graphs that organize large bodies of data for quick and user-friendly retrieval. The Wikipedia schema accomplishes a similar, abet more primitive, role. There’s nothing wrong with the fundamentals of the technology, just the applications that Westoids doggedly insist it be used for.

      If you no longer need to boil down half a Great Lake to create the next iteration of Shrimp Jesus, that’s good whether or not you think Meta should be dedicating millions of hours of compute to this mind-eroding activity.

      • daltotron@lemmy.ml
        link
        fedilink
        arrow-up
        0
        ·
        5 months ago

        I think maybe it’s naive to think that if the cost goes down, shrimp jesus won’t just be in higher demand. Shrimp jesus has no market cap, bullshit has no market cap. If you make it more efficient to flood cyberspace with bullshit, cyberspace will just be flooded with more bullshit. Those great lakes will still boil, don’t worry.

      • WoodScientist@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        0
        ·
        5 months ago

        There’s nothing wrong with the fundamentals of the technology, just the applications that Westoids doggedly insist it be used for.

        Westoids? Are you the type of guy I feel like I need to take a shower after talking to?

    • Naia@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      0
      ·
      5 months ago

      I’m tired of this uninformed take.

      LLMs are not a magical box you can ask anything of and get answers. If you are lucky and blindly asking questions it can give some accurate general data, but just like how human brains work you aren’t going to be able to accurately recreate random trivia verbatim from a neural net.

      What LLMs are useful for, and how they should be used, is a non-deterministic parsing context tool. When people talk about feeding it more data they think of how these things are trained. But you also need to give it grounding context outside of what the prompt is. give it a PDF manual, website link, documentation, whatever and it will use that as context for what you ask it. You can even set it to link to reference.

      You still have to know enough to be able to validate the information it is giving you, but that’s the case with any tool. You need to know how to use it.

      As for the spyware part, that only matters if you are using the hosted instances they provide. Even for OpenAI stuff you can run the models locally with opensource software and maintain control over all the data you feed it. As far as I have found, none of the models you run with Ollama or other local AI software have been caught pushing data to a remote server, at least using open source software.

    • RandomVideos@programming.dev
      cake
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      artificial intelligence

      AI has been used in game development for a while and i havent seen anyone complain about the name before it became synonymous with image/text generation

      • kshade@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        5 months ago

        It was a misnomer there too, but at least people didn’t think a bot playing C&C would be able to save the world by evolving into a real, greater than human intelligence.

    • gerryflap@feddit.nl
      link
      fedilink
      arrow-up
      0
      ·
      5 months ago

      The difference is that you can actually download this model and run it on your own hardware (if you have sufficient hardware). In that case it won’t be sending any data to China. These models are still useful tools. As long as you’re not interested in particular parts of Chinese history of course ;p