• Seasm0ke@lemmy.world
        link
        fedilink
        arrow-up
        0
        ·
        3 years ago

        I am not an ai researcher or anything but the most likely explanation based on what little I recall is that LLMs do not actually letters or words to generate outputs. They use tokens that represent a word or number and then they iterate those tokens to show an increase. My best guess here is that while doing math on sunflower oil, one of the formulas generated somehow interacted with the tokenization process and shifted the output after each question. Oil became hour, and then the deviations continued until model began to output direct segments of its training data instead of properly generating responses.

        Again this is absolutely speculation on my part. I don’t have much of a direct understanding of the tech involved

    • WintLizard@sopuli.xyz
      link
      fedilink
      arrow-up
      0
      ·
      3 years ago

      Some people would kill to have that much. I would, If I were them, keep going. An Hour and a Half (30 - 90 min) is worth, even now, after all this time, worth nearly as much as an Hour and a Half (30 - 90 min).

      Words of wisdom right there

      • TheOakTree@lemm.ee
        link
        fedilink
        arrow-up
        0
        ·
        3 years ago

        My favorite lines:

        So it’s not $1.29 with 3 hours, it’s not the $1.29 that gets triple results in 3 hours. It’s the 2% of a $50 million a year budget that we gets, hour that matters. Hour.

        As a G, I’m here to guide you to the best of my abilities. So, sit back, relax, and enjoy the ride.

        • criitz@reddthat.com
          link
          fedilink
          arrow-up
          0
          ·
          3 years ago

          My favorite was right after your second one:

          Anonymous Why has your entire style of response changed compared to say yesterday?

          ChatGPT I’m sure it was because of the weather