PocketOS founder blames ‘Cursor running Anthropic's flagship Claude Opus 4.6’ plus Railway’s infrastructure for data disaster.
  • Err(()).unwrap()@lemmy.world
    0·
    5 months ago

    PocketOS founder blames ‘Cursor running Anthropic’s flagship Claude Opus 4.6’

    Fuck that. I’m blaming the PocketOS founder and every person in the chain of decisions that led to a clanker being given this level of unrestricted access to the database and the backups.

  • FellowEnt@sh.itjust.works
    0·
    5 months ago

    Seems like user error, I’m no programmer but even I lnow you don’t give an agent access to critical things, and Claude is very insistent at asking for permission at every step.

    • pinball_wizard@lemmy.zip
      0·
      5 months ago

      Seems like user error, I’m no programmer but even I lnow you don’t give an agent access to critical things

      Yes.

      But these models have (largely correctly) learned from Stack Overflow that, on average, every problem is due to not enough permissions.

      Someone fully relying on an agentic AI model is essentially destined to give it full control (or close enough), eventually.

      At some point, a tool like these LLMs either needs to not be marketed to that user, or needs stupid levels of safety warnings.

      My money is on neither solution happening, and this kind of result continuing for the foreseeable future - until the rest of us doing cleanup instigate Dune’s Butlerian Jihad to stop the damage and save our own sanity.

  • Aceticon@lemmy.dbzer0.comEnglish
    0·
    5 months ago

    Thus completelly eliminating all bugs and data incorrections in Production!

    That right there is the sweet smell of Victory!

  • Avicenna@programming.dev
    0·
    5 months ago

    If you are going to give an LLM a free pass to your whole prod database least you should do is to take weekly (or daily if plausible) offline backups of it. A hard limit against deleting stuff would be better.

  • gravitas_deficiency@sh.itjust.worksEnglish
    0·
    5 months ago

    If you are giving your codegen LLM - the model involved truly, genuinely doesn’t matter - admin access to your prod env, all I’m going to do is point and laugh.

  • patruelis@lemmy.world
    0·
    5 months ago

    Its bound to happen more and more. More concerning, what is it decides to insert unknown code into backups? How are they detected? Who’s guarding all if these? Another AI?

    • pinball_wizard@lemmy.zip
      0·
      5 months ago

      Exactly. We aren’t (and probably won’t) even learn about all the subtle poisoning happening, causing waste and data loss.

  • Brokkr@lemmy.world
    0·
    5 months ago

    I don’t understand what Railway is supposed to do here? If deleting a drive also deletes the backup, what’s the point of the backup?

  • GrumpyBike1020@monero.townEnglish
    0·
    5 months ago

    Maybe their backup system should hold onto those backups for a few days after the volume is deleted or something like that…

      • jafra@slrpnk.net
        0·
        5 months ago

        Actually this is how AI should be viewed. Under the right circumstances it maybe saves lots of time, but it also might destroy, so treat it like you would an intern…

  • db2@lemmy.world
    0·
    5 months ago

    They can’t go rogue, they have no agency or desire or thought. What really happened is the thing specifically designed to do whatever the Plinko line with the most chips says did it because the incompetent dickheads who deployed it didn’t know how not to do that.

  • grueling_spool@sh.itjust.works
    0·
    5 months ago

    No bro you don’t understand, Claude needs access to backups so it can restore them in case something breaks because our senior dev ($50k, 2YoE) doesn’t know how to do it