• REDACTED@infosec.pub
        link
        fedilink
        English
        arrow-up
        0
        ·
        9 days ago

        How do you think language in our brains work? Just like many things in tech (especially cameras), things are often inspired by how it works in nature.

  • CeeBee_Eh@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 days ago

    This happened to me the other day with Jippity. It outright lied to me:

    “You’re absolutely right. Although I don’t have access to the earlier parts of the conversation”.

    So it says that I was right in a particular statement, but didn’t actually know what I said. So I said to it, you just lied. It kept saying variations of:

    “I didn’t lie intentionally”

    “I understand why it seems that way”

    “I wasn’t misleading you”

    etc

    It flat out lied and tried to gaslight me into thinking I was in the wrong for taking that way.

    • greygore@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      10 days ago

      It didn’t lie to you or gaslight you because those are things that a person with agency does. Someone who lies to you makes a decision to deceive you for whatever reason they have. Someone who gaslights you makes a decision to behave like the truth as you know it is wrong in order to discombobulate you and make you question your reality.

      The only thing close to a decision that LLMs make is: what text can I generate that statistically looks similar to all the other text that I’ve been given. The only reason they answer questions is because in the training data they’ve been provided, questions are usually followed by answers.

      It’s not apologizing you to, it knows from its training data that sometimes accusations are followed by language that we interpret as an apology, and sometimes by language that we interpret as pushing back. It regurgitates these apologies without understanding anything, which is why they seem incredibly insincere - it has no ability to be sincere because it doesn’t have any thoughts.

      There is no thinking. There are no decisions. The more we anthropomorphize these statistical text generators, ascribing thoughts and feelings and decision making to them, the less we collectively understand what they are, and the more we fall into the trap of these AI marketers about how close we are to truly thinking machines.

      • CeeBee_Eh@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        edit-2
        10 days ago

        The only thing close to a decision that LLMs make is

        That’s not true. An “if statement” is literally a decision tree.

        The only reason they answer questions is because in the training data they’ve been provided

        This is technically true for something like GPT-1. But it hasn’t been true for the models trained in the last few years.

        it knows from its training data that sometimes accusations are followed by language that we interpret as an apology, and sometimes by language that we interpret as pushing back. It regurgitates these apologies without understanding anything, which is why they seem incredibly insincere

        It has a large amount of system prompts that alter default behaviour in certain situations. Such as not giving the answer on how to make a bomb. I’m fairly certain there are catches in place to not be overly apologetic to minimize any reputation harm and to reduce potential “liability” issues.

        And in that scenario, yes I’m being gaslite because a human told it to.

        There is no thinking

        Partially agree. There’s no “thinking” in sentient or sapient sense. But there is thinking in the academic/literal definition sense.

        There are no decisions

        Absolutely false. The entire neural network is billions upon billions of decision trees.

        The more we anthropomorphize these statistical text generators, ascribing thoughts and feelings and decision making to them, the less we collectively understand what they are

        I promise you I know very well what LLMs and other AI systems are. They aren’t alive, they do not have human or sapient level of intelligence, and they don’t feel. I’ve actually worked in the AI field for a decade. I’ve trained countless models. I’m quite familiar with them.

        But “gaslighting” is a perfectly fine description of what I explained. The initial conditions were the same and the end result (me knowing the truth and getting irritated about it) were also the same.

        • greygore@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          10 days ago

          The only thing close to a decision that LLMs make is

          That’s not true. An “if statement” is literally a decision tree.

          If you want to engage in a semantically argument, then sure, an “if statement” is a form of decision. This is a worthless distinction that has nothing to do with my original point and I believe you’re aware of that so I’m not sure what this adds to the actual meat of the argument?

          The only reason they answer questions is because in the training data they’ve been provided

          This is technically true for something like GPT-1. But it hasn’t been true for the models trained in the last few years.

          Okay, what was added to models trained in the last few years that makes this untrue? To the best of my knowledge, the only advancements have involved:

          • Pre-training, which involves some additional steps to add to or modify the initial training data
          • Fine-tuning, which is additional training on top of an existing model for specific applications.
          • Reasoning, which to the best of my knowledge involves breaking the token output down into stages to give the final output more depth.
          • “More”. More training data, more parameters, more GPUs, more power, etc.

          I’m hardly an expert in the field, so I could have missed plenty, so what is it that makes it “understand” that a question needs to be answered that doesn’t ultimately go back to the original training data? If I feed it training data that never involves questions, then how will it “know” to answer that question?

          it knows from its training data that sometimes accusations are followed by language that we interpret as an apology, and sometimes by language that we interpret as pushing back. It regurgitates these apologies without understanding anything, which is why they seem incredibly insincere

          It has a large amount of system prompts that alter default behaviour in certain situations. Such as not giving the answer on how to make a bomb. I’m fairly certain there are catches in place to not be overly apologetic to minimize any reputation harm and to reduce potential “liability” issues.

          System prompts are literally just additional input that is “upstream” of the actual user input, and I fail to see how that changes what I said about it not understanding what an apology is, or how it can be sincere when the LLM is just spitting out words based on their statistical relation to one another?

          An LLM doesn’t even understand the concept of right or wrong, much less why lying is bad or when it needs to apologize. It can “apologize” in the sense that it has many examples of apologies that it can synthesize into output when you request one, but beyond that it’s just outputting text. It doesn’t have any understanding of that text.

          And in that scenario, yes I’m being gaslite because a human told it to.

          Again, all that’s doing is adding additional words that can be used in generating output. It’s still just generating text output based on text input. That’s it. It has to know it’s lying or being deceitful in order to gaslight you. Does the text resemble something that can be used to gaslight you? Sure. And if I copy and pasted that from ChatGPT that’s what I’d be doing, but an LLM doesn’t have any real understanding of what it’s outputting so saying that there’s any intent to do anything other than generate text based on other text is just nonsense.

          There is no thinking

          Partially agree. There’s no “thinking” in sentient or sapient sense. But there is thinking in the academic/literal definition sense.

          Care to expand on that? Every definition of thinking that I find involves some kind of consideration or reflection, which I would argue that the LLM is not doing, because it’s literally generating output based on a complex system of weighted parameters.

          If you want to take the simplest definition of “well, it’s considering what to output and therefore that’s thought”, then I could argue my smart phone is “thinking” because when I tap on a part of the screen it makes decisions about how to respond. But I don’t think anyone would consider that real “thought”.

          There are no decisions

          Absolutely false. The entire neural network is billions upon billions of decision trees.

          And a logic gate “decides” what to output. And my lightbulb “decides” whether or not to light up based on the state of the switch. And my alarm “decides” to go off based on what time I set it for last night.

          My entire point was to stop anthropomorphizing LLMs by describing what they do as “thought”, and that they don’t make “decisions” in the same way humans do. If you want to use definitions that are overly broad just to say I’m wrong, fine, that’s your prerogative, but it has nothing to do with the idea I was trying to communicate.

          The more we anthropomorphize these statistical text generators, ascribing thoughts and feelings and decision making to them, the less we collectively understand what they are

          I promise you I know very well what LLMs and other AI systems are. They aren’t alive, they do not have human or sapient level of intelligence, and they don’t feel. I’ve actually worked in the AI field for a decade. I’ve trained countless models. I’m quite familiar with them.

          Cool.

          But “gaslighting” is a perfectly fine description of what I explained. The initial conditions were the same and the end result (me knowing the truth and getting irritated about it) were also the same.

          Sure, if you wanna ascribe human terminology to what marketing companies are calling “artificial intelligence” and further reinforcing misconceptions about how LLMs work, then yeah, you can do that. If you care about people understanding that these algorithms aren’t actually thinking in the same way that humans do, and therefore believing many falsehoods about their capabilities, like I do, then you’d use different terminology.

          It’s clear that you don’t care about that and will continue to anthropomorphize these models, so… I guess I’m done here.

    • Whitebrow@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 days ago

      Not even a good use case either, especially when it spews such bullshit like “there’s no recorded instance of trump ever having used the word enigma” and “there’s 1 r in strawberry”.

      LLMs are a copy paste machine, not a rationalization engine of any sort (at least as far as all the slop that we get shoved in our face, I don’t include the specialized protein folding and reconstructive models that were purpose built for very niche applications)

      • The Quuuuuill@slrpnk.net
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        they’re solid starting point for shopping now that wirecutter, slant, and others are enshittified. i hate it and it makes me feel dirty to use, and you can’t just do whatever the llm says. but asking it for a list of options to then explore is currently the best way i’ve found to jump into things like outdoor basketball shoe options

  • kureta@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    10 days ago

    People should understand that words not “unaware” or “overconfident” are not even applicable to these pieces of software. We might build intelligent machines in the future but if you know how these large language models work, it is obvious that it doesn’t even make sense to talk about the awareness, intelligence, or confidence of such systems.

    • turmacar@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      10 days ago

      I find it so incredibly frustrating that we’ve gotten to the point where the “marketing guys” are not only in charge, but are believed without question, that what they say is true until proven otherwise.

      “AI” becoming the colloquial term for LLMs and them being treated as a flawed intelligence instead of interesting generative constructs is purely in service of people selling them as such. And it’s maddening. Because they’re worthless for that purpose.

  • Lodespawn@aussie.zone
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 days ago

    Why is a researcher with a PhD in social sciences researching the accuracy confidence of predictive text, how has this person gotten to where they are without being able to understand that LLM don’t think? Surely they came up when he started even co soldering this brainfart of a research project?

      • Lodespawn@aussie.zone
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        I guess, but it’s like proving your phones predictive text has confidence in its suggestions regardless of accuracy. Confidence is not an attribute of a math function, they are attributing intelligence to a predictive model.

        • FanciestPants@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 days ago

          I work in risk management, but don’t really have a strong understanding of LLM mechanics. “Confidence” is something that i quantify in my work, but it has different terms that are associated with it. In modeling outcomes, I may say that we have 60% confidence in achieving our budget objectives, while others would express the same result by saying our chances of achieving our budget objective are 60%. Again, I’m not sure if this is what the LLM is doing, but if it is producing a modeled prediction with a CDF of possible outcomes, then representing its result with 100% confindence means that the LLM didn’t model any other possible outcomes other than the answer it is providing, which does seem troubling.

          • Lodespawn@aussie.zone
            link
            fedilink
            English
            arrow-up
            0
            ·
            11 days ago

            Nah so their definition is the classical “how confident are you that you got the answer right”. If you read the article they asked a bunch of people and 4 LLMs a bunch of random questions, then asked the respondent whether they/it had confidence their answer was correct, and then checked the answer. The LLMs initially lined up with people (over confident) but then when they iterated, shared results and asked further questions the LLMs confidence increased while people’s tends to decrease to mitigate the over confidence.

            But the study still assumes intelligence enough to review past results and adjust accordingly, but disregards the fact that an AI isnt intelligence, it’s a word prediction model based on a data set of written text tending to infinity. It’s not assessing validity of results, it’s predicting what the answer is based on all previous inputs. The whole study is irrelevant.

            • jj4211@lemmy.world
              link
              fedilink
              English
              arrow-up
              0
              ·
              11 days ago

              Well, not irrelevant. Lots of our world is trying to treat the LLM output as human-like output, so if human’s are going to treat LLM output the same way they treat human generated content, then we have to characterize, for the people, how their expectations are broken in that context.

              So as weird as it may seem to treat a stastical content extrapolation engine in the context of social science, there’s a great deal of the reality and investment that wants to treat it as “person equivalent” output and so it must be studied in that context, if for no other reason to demonstrate to people that it should be considered “weird”.

  • rc__buggy@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 days ago

    However, when the participants and LLMs were asked retroactively how well they thought they did, only the humans appeared able to adjust expectations

    This is what everyone with a fucking clue has been saying for the past 5, 6? years these stupid fucking chatbots have been around.

  • Modern_medicine_isnt@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 days ago

    It’s easy, just ask the AI “are you sure”? Until it stops changing it’s answer.

    But seriously, LLMs are just advanced autocomplete.

    • jj4211@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 days ago

      I kid you not, early on (mid 2023) some guy mentioned using ChatGPT for his work and not even checking the output (he was in some sort of non-techie field that was still in the wheelhouse of text generation). I expresssed that LLMs can include some glaring mistakes and he said he fixed it by always including in his prompt “Do not hallucinate content and verify all data is actually correct.”.

      • Passerby6497@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        Ah, well then, if he tells the bot to not hallucinate and validate output there’s no reason to not trust the output. After all, you told the bot not to, and we all know that self regulation works without issue all of the time.

        • jj4211@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 days ago

          It gave me flashbacks when the Replit guy complained that the LLM deleted his data despite being told in all caps not to multiple times.

          People really really don’t understand how these things work…

    • Lfrith@lemmy.ca
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 days ago

      They can even get math wrong. Which surprised me. Had to tell it the answer is wrong for them to recalculate and then get the correct answer. It was simple percentages of a list of numbers I had asked.

      • saimen@feddit.org
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        I once gave some kind of math problem (how to break down a certain amount of money into bills) and the llm wrote a python script for it, ran it and thus gave me the correct answer. Kind of clever really.

      • GissaMittJobb@lemmy.ml
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        Language models are unsuitable for math problems broadly speaking. We already have good technology solutions for that category of problems. Luckily, you can combine the two - prompt the model to write a program that solves your math problem, then execute it. You’re likely to see a lot more success using this approach.

        • jj4211@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 days ago

          Also, generally the best interfaces for LLM will combine non-LLM facilities transparently. The LLM might be able to translate the prose to the format the math engine desires and then an intermediate layer recognizes a tag to submit an excerpt to a math engine and substitute the chunk with output from the math engine.

          Even for servicing a request to generate an image, the text generation model runs independent of the image generation, and the intermediate layer combines them. Which can cause fun disconnects like the guy asking for a full glass of wine. The text generation half is completely oblivious to the image generation half. So it responds playing the role of a graphic artist dutifully doing the work without ever ‘seeing’ the image, but it assumes the image is good because that’s consistent with training output, but then the user corrects it and it goes about admitting that the picture (that it never ‘looked’ at) was wrong and retrying the image generator with the additional context, to produce a similarly botched picture.

      • jj4211@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        Fun thing, when it gets the answer right, tell it is was wrong and then see it apologize and “correct” itself to give the wrong answer.

  • Perspectivist@feddit.uk
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 days ago

    Large language models aren’t designed to be knowledge machines - they’re designed to generate natural-sounding language, nothing more. The fact that they ever get things right is just a byproduct of their training data containing a lot of correct information. These systems aren’t generally intelligent, and people need to stop treating them as if they are. Complaining that an LLM gives out wrong information isn’t a failure of the model itself - it’s a mismatch of expectations.

    • shalafi@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 days ago

      Neither are our brains.

      “Brains are survival engines, not truth detectors. If self-deception promotes fitness, the brain lies. Stops noticing—irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. You experience a simulation built from assumptions. Shortcuts. Lies. Whole species is agnosiac by default.”

      ― Peter Watts, Blindsight (fiction)

      Starting to think we’re really not much smarter. “But LLMs tell us what we want to hear!” Been on FaceBook lately, or lemmy?

      If nothing else, LLMs have woke me to how stupid humans are vs. the machines.

      • jj4211@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        It’s not that they may be deceived, it’s that they have no concept of what truth or fiction, mistake or success even are.

        Our brains know the concepts and may fall to deceipt without recognizing it, but we at least recognize that the concept exists.

        An AI generates content that is a blend of material from the training material consistent with extending the given prompt. It only seems to introduce a concept of lying or mistakes when the human injects that into the human half of the prompt material. It will also do so in a way that the human can just as easily instruct it to correct a genuine mistake as well as the human instruct it to correct something that is already correct (unless the training data includes a lot of reaffirmation of the material in the face of such doubts).

        An LLM can consume more input than a human can gather in multiple lifetimes and still bo wonky in generating content, because it needs enough to credibly blend content to extend every conceivable input. It’s why so many people used to judging human content get derailed by judging AI content. An AI generates a fantastic answer to an interview question that only solid humans get right, only to falter ‘on the job’ because the utterly generic interview question looks like millions of samples in the input, but the actual job was niche.

      • Perspectivist@feddit.uk
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        There are plenty of similarities in the output of both the human brain and LLMs, but overall they’re very different. Unlike LLMs, the human brain is generally intelligent - it can adapt to a huge variety of cognitive tasks. LLMs, on the other hand, can only do one thing: generate language. It’s tempting to anthropomorphize systems like ChatGPT because of how competent they seem, but there’s no actual thinking going on. It’s just generating language based on patterns and probabilities.

      • aesthelete@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 days ago

        Every thread about LLMs has to have some guy like yourself saying how LLMs are like humans and smarter than humans for some reason.