• huginn@feddit.it
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    Unless you want to call your predictive text on your keyboard a mind you really can’t call an LLM a mind. It is nothing more than a linear progression from that. Mathematically proven to not show any form of emergent behavior.

    • MxM111@kbin.social
      link
      fedilink
      arrow-up
      0
      ·
      11 months ago

      I do not think that it is “linear” progression. ANN by definition is nonlinear. Neither I think anything is “mathematically proven”. If I am wrong, please provide a link.

        • MxM111@kbin.social
          link
          fedilink
          arrow-up
          0
          ·
          11 months ago

          Thank you. This paper though does not state that there are no emergent abilities. It only states that one can introduce a metric with respect to which the emergent ability behaves smoothly and not threshold-like. While interesting, it only suggests that things like intelligence are smooth functions, but so what? Some other metrics show exponential or threshold dependence and whether the metric is right depends only how one will use it. And there is no law that emerging properties have to be threshold like. Quite the opposite - nearly all examples in physics that I know, the emergence appears gradually.

    • Kogasa@programming.dev
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      No such thing has been “mathematically proven.” The emergent behavior of ML models is their notable characteristic. The whole point is that their ability to do anything is emergent behavior.

      • huginn@feddit.it
        link
        fedilink
        English
        arrow-up
        0
        ·
        edit-2
        11 months ago

        Here’s a white paper explicitly proving:

        1. No emergent properties (illusory due to bad measures)
        2. Predictable linear progress with model size

        https://arxiv.org/abs/2304.15004

        The field changes fast, I understand it is hard to keep up

        • Kogasa@programming.dev
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 months ago

          Sure, if you define “emergent abilities” just so. It’s obvious from context that this is not what I described.