• MadMadBunny@lemmy.ca
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    But, will it work, huh? HUH?

    I can also type a bunch of random sentences of words. Doesn’t make it more understandable.

    • ryannathans@aussie.zone
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      Some models are getting so good they can patch user reported software defects following test driven development with minimal or no changes required in review. Specifically Claude Sonnet and Gemini

      So the claims are at least legit in some cases

      • 6nk06@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        Oh good. They can show us how it’s done by patching open-source projects for example. Right? That way we will see that they are not full of shit.

        Where are the patches? They have trained on millions of open-source projects after all. It should be easy. Show us.

          • 6nk06@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            0
            ·
            1 year ago

            I’m not the one trying to prove anything, and I think it’s all bullshit. I’m waiting for your proof though. Even with a free open-source black box.

                • ryannathans@aussie.zone
                  link
                  fedilink
                  English
                  arrow-up
                  0
                  ·
                  1 year ago

                  At work, the software is not open source.

                  I would use it for contributions to open source projects but I do not pay for any AI subscriptions, and I can’t use my employee account for copilot enterprise for non-work projects.

                  Every week for the last year or so I have been testing various copilot models against customer reported software defects and it’s seriously at a point now where with a single prompt Gemini pro 2.5 is solving the entire defect with unit tests. Some need no changes in review and are good to go.

                  As an open source maintainer of a large project I have noticed a huge uptick in PRs which has created a larger review workload, I’m almost certain these are due to LLMs. Quality of a typical PR has not decreased since LLMs have become available and I am thus far very glad

                  If I were to speculate I’d guess the huge increase in context windows has made the tools viable, models like GPT5 are garbage on any sizable code bases

        • JustinTheGM@ttrpg.network
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          That’s an interesting point, and leads to a reasonable argument that if an AI is trained on a given open source codebase, developers should have free access to use that AI to improve said codebase. I wonder whether future license models might include such clauses.

      • WaitThisIsntReddit@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        A couple agent iterations will compile. Definitely won’t do what you wanted though, and if it does it will be the dumbest way possible.

        • TORFdot0@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          Yeah you can definitely bully AI into giving you some thing that will run if you yell at it long enough. I don’t have that kind of patience

          Edit: typically I see it just silently dump errors to /dev/null if you complain about it not working lol

          • Darkenfolk@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            0
            ·
            1 year ago

            And people say that AI isn’t humanlike. That’s peak human behavior right there, having to bother someone out of procrastination mode.

            The edit makes it even better, swiping things under the rug? Hell yeah!

    • Thorry@feddit.org
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      Also just because the code works, doesn’t mean it’s good code.

      I’ve had to review code the other day which was clearly created by an LLM. Two classes needed to talk to each other in a bit of a complex way. So I would expect one class to create some kind of request data object, submit it to the other class, which then returns some kind of response data object.

      What the LLM actually did was pretty shocking, it used reflection to get access from one class to the private properties with the data required inside the other class. It then just straight up stole the data and did the work itself (wrongly as well I might add). I just about fell of my chair when I saw this.

      So I asked the dev, he said he didn’t fully understand what the LLM did, he wasn’t familiar with reflection. But since it seemed to work in the few tests he did and the unit tests the LLM generated passed, he thought it would be fine.

      Also the unit tests were wrong, I explained to the dev that usually with humans it’s a bad idea to have the person who wrote the code also (exclusively) write the unit tests. Whenever possible have somebody else write the unit tests, so they don’t have the same assumptions and blind spots. With LLMs this is doubly true, it will just straight up lie in the unit tests. If they aren’t complete nonsense to begin with.

      I swear to the gods, LLMs don’t save time or money, they just give the illusion they do. Some task of a few hours will take 20 min and everyone claps. But then another task takes twice as long and we just don’t look at that. And the quality suffers a lot, without anyone really noticing.

      • airgapped@piefed.social
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        Great description of a problem I noticed with most LLM generated code of any decent complexity. It will look fantastic at first but you will be truly up shit creek by the time you realise it didn’t generate a paddle.

        • Kissaki@feddit.org
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          They did say why they’re doing it

          Whenever possible have somebody else write the unit tests, so they don’t have the same assumptions and blind spots.

          Did that not make sense to you?

          I usually wouldn’t do that, because it’s a bigger investment. But it certainly makes logical sense to me and is something teams can weigh and decide on.

      • Kissaki@feddit.org
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        So I asked the dev, he said he didn’t fully understand what the LLM did, he wasn’t familiar with reflection.

        Big baffling facepalm moment.

        If they would at least prefix the changeset description with that it’d be easier to interpret and assess.

      • floofloof@lemmy.ca
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        Who hasn’t encountered that one jerk who builds only new code to impress management, and never maintains or fixes existing code? I think of them as proof-of-concept posers. They make things that look flashy, impress the execs, and barely work for a single use care, then dump all the bugs, maintenance and actual architecture on the other devs. LLMs are going to be a gift to these people and a pain for everyone who actually knows how to engineer things well. They’ll encourage this kind of shallow flashiness and make the maintenance problems worse, but the execs will be convinced that only the LLM posers are productive and everyone else is sitting idle.

      • criss_cross@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 year ago

        They’ve been great for me at optimizing bite sized annoying tasks. They’re really bad at doing anything beyond that. Like astronomically bad.

  • thejml@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    Copilot keeps finishing my code for me in near real time… it completely disrupts my train of thought and my productivity dropped tremendously. I finally disabled it.

    I LIKE writing code, stop trying to take the stuff away that I WANT to do and instead take away the stuff I HATE doing.

    • criss_cross@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      I wish I could get it to stop finishing comments for me. It’s like some jackass is trying to complete my sentence for me but gets it completely wrong every time and it breaks my train of thought.

    • Serinus@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      Yeah, I just wrote a blog post comment about how I enjoy using Copilot. But that’s when I explicitly ask it a question or give it a task. The auto complete is wrong more often than it’s right.

      Probably doesn’t help that if it was tedious, boilerplate code I would have already explicitly asked it.

      • bluGill@fedia.io
        link
        fedilink
        arrow-up
        0
        ·
        1 year ago

        I like auto complete because I’m a terrible speller I’d write “int countOfCommplixThang”, but auto complete guesses “int countComplexThing” Sometimes it even comes up with a better name than I would

        • Serinus@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          That old autocomplete is great. It’s specifically the AI autocomplete that’s less useful.

    • lauha@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      What I don’t want AI to do:

      • write code for me
      • write fixes for me

      What I want it to do:

      • find bugs and tell me about them (but still don’t fix them)
  • kreskin@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    yes but all the code will be wrong and you will spend your entire day chasing stupid mistakes and hallucinations in the code. I’d rather just write the code myself thanks.

  • 0xDREADBEEF@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    My problem is that the dev and stage environments are giving me 502 gateway errors when hitting only certain api endpoints from the app gateway. My real problem is devops aren’t answering my support tickets and telling me which terraform var file I gotta muck with and tell me what to fix on it. I’m sure you’ll be fixed soon though right copilot?

    • jj4211@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      No, just complete. Whatever the dude does may have morning to do with what you needed it to do, but it will be “done”

    • Evotech@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      Depends. If it’s a script that will like, cut your video file every 10 seconds with ffmpeg or something simple. Yeah it will one-shot it.

  • Treczoks@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    What they forget to mention is that you then spend the rest of the week to fix the bugs it introduced and to explain why your code deleted the production database…

  • melsaskca@lemmy.ca
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    I would rather paint a portrait by myself, spending the time to do it, rather than asking some computer prompt to spit me out a picture. Same logic applies with coding for me.