- Make laws against chatbots.
- Demand proof you are not a chatbot.
- Surveillance capitalism.
The real target here is population control.
The lawmakers, which take billionaire money by the ton, who HAVE NEVER given a shit, suddenly, NOW, they want to protect the vulnerable. Abso fucking lutely laughable on its face.
Agreed. It’s never about protection, just covert exploitation
This bill gave us the “best” interaction:
https://bsky.app/profile/badmedicaltakes.bsky.social/post/3mghyg5eufk2m
A Bluesky skeet from @badmedicaltakes.bsky.social:
"Twitter user eoghan:
How dare poor people get free medical advice
<quote tweet from Twitter user Polymarket: BREAKING: New York bill would ban AI from answering questions related to medicine, law, dentistry, nursing, psychology, social work, engineering, & more.>
Twitter user YBrogard79094:
JUST MAKE HEALTHCARE ACCESSIBLETwitter user eoghan:
AI is literally free healthcare. Being a communist must be exhausting"
Some horses you can’t even lead to water. Let alone make them drink.
You can google your simptoms and there probably are some reliable sites but a hallucinating chatbot is a bad idea. Not to mention some people suggested treating covid with chlorine, vinegar etc…
Laws like this are great for these companies. This is how they will justify removing access to useful information and putting it behind paywalls. But oh your need a prescription so now the insurance companies are involved (spoiler: they already are) and so you don’t even have access to pay out the nose for medical information.
Then when Google search has been completely replaced with AI, you won’t even be able to search for medical information.
Healthcare companies aren’t about to provide anything for free.
Most of the medical information coming up these days is garbage and you should be going to a known, reputable site and searching their database. LLMs have been trained on absolute garbage. There is nothing of value being kept from anyone here.
LLMs have been trained on absolute garbage
It depends on the LLM actually.
Specialized medical LLMs are actually very accurate.
I’m sure the quality of the LLM output does vary a lot based on the size of the scope it covers and the training data set.
However, I believe that if it were possible to get an LLM to be “quite accurate” in any context would make it easy to find a path to profitability for that tool, but I don’t think we have seen that materialize anywhere.
I believe that the best they can get is “more accurate” than the mean, but still not accurate enough to reliably make anyone money*.
*Nvidia notwithstanding
Moreover, until you can get the same output from the same input from an LLM consistently, the entire tech is unreliable garbage.
LLMs and chatbots should not be giving medical advice. You are afraid of the private healthcare system, not the lack of access to the most janky bandaid fix for its failures.
Neither should Wikipedia or Google. So I guess by your logic nobody should search or learn about medical conditions on a computer.
You know damn well there’s an important difference related to the confidence of a bot that has been a key problem since this whole thing started.
I guess by your logic nobody should search or learn about medical conditions on a computer.
How else would we know the TRUTH about 5G vaccines and invermectin? Or the cures of Apple Cider vinegar?
The line between medical advice and personal research is pretty freaking gray, so banning medical advice. Does that also ban talking to llms about anything that is medical adjacent?
Does medical adjacent mean personal disabilities? Drug related interests? Pet health?
…etc
It’s a slippery slope and we don’t need to be sliding down it
People are so vicious over this tech they would rather have disabled poor people with cancer suffer and die under inadequate care than do anything about the inadequate care. Ban the tech, but let this all go on.
If you are perfectly able and well, you can ignore all advice that isn’t perfect.
The perspective they seem to lack is frightening. The empathy they refuse to engage is massive. This is able-ism.
Tech companies are bad, but use of tech will cure and ease cancer, HIV, and chronic disease. Bring on the downvotes.
“Would rather have disabled people with cancer suffer and die…”
My guy, that’s not a lack of LLM access, it’s a completely fucked US healthcare system that forces people onto the internet because they can’t get what they need from the state, you goofy-ass weirdo.
“Would rather have disabled people with cancer suffer and die…”
Well yes of course but also restricting access to information machines doesn’t exactly help much either.
Do hallucinating LLMs, that have done such things as convince a child to commit suicide before, really count as “information machines”? The Mayo clinic website might take a single whole other braincell to read through but at least it’ll be written properly.
I mean, the fact that you consider these programs to have enough credibility to be called “information machines” is exactly why they’re so potentially dangerous.
I hate to break it to you but… they’re not really “information machines”. Google search is a better information machine.
I think you may be falling into a false dichotomy. Not only is the choice being presented a bad one, it ignores real solutions to the root problem, leaving us to argue over the crappy “band-aid” solution to it.
I believe that people needing health care should have no reason to ask a chat bot about their symptoms because they can ask a helpful doctor instead. The fact that they can’t do that is the problem, not their access or lack of it to the chat bot.
I may have become too cynical but, as is often the case when you dig deeper, this sounds like the result of lobbyists trying to protect licensing rather than people.
We can be dumb, but we’ve been doing web searches for legal and medical advice for ages because it is too damned expensive and time consuming to go to professionals for every little thing. Not to mention, doctors have so little time for you that it is hard to get them to listen to the whole story to make connections between symptoms.
The LLMs already tell you that they aren’t licensed professionals and, for many, provide citations for their sources (miles better than your typical health website).
As a personal anecdote, my son was having stomach pain but was planning to tough it out. He checked with ChatGPT and it recommended he go to the ER. He did, and if he hadn’t, he would likely be dead now. He spent 3 days in the hospital having his bowels unobstructed through a tube in his nose.
There is value in people having that kind of information at their fingertips.
Regulation is absolutely needed, but I would rather they focus on protecting us from AI being used for military purposes, mass surveillance, etc. rather than protecting citizens from ourselves.
Are you in the US? My take away here is American healthcare is bad but we’re treating the symptom not the disease.
Yeah, I’m in the US and I agree. Though it is going to take some serious change to treat the problem. In the meantime, this is at least a stopgap solution for people who don’t have a lot of options.
Wait, he thought he could sit that pain through at home? Your son is tough as nails. Give him a hug for me and everyone else who’s had that four day n-g tube delight.
Yeah, he is pretty tough. I wish I could hug him, he is about a 10 hour drive from me. That tube was nightmarish from what he’s told me.
if i were his parent, i would be giving him gentle reminders to drink more water. after teasing him for eating way too much corn or broccoli or whatever bastard fiber caused his obstruction (assuming he’s in a mental place he can handle the teasing)
He’s in his 20s so he is only slightly more likely to take my advice than he was as a teenager 😆
if he’s more likely to take the advice, tell him it came from someone who has had hundreds of bowel obstructions.
Fuck the hell out of this.
My brothers in christ, I’m not going to drink bleach because the chat bot tells me to. I’m trying to come up with diagnostic ideas to discuss with my doctors, and it’s invaluable for that.
then you should just be discussing your issues with your doctors in the first place?
They are stumped and so is my second opinion. In this situation, every idea is valuable.
Not every idea is valuable.
People do it when a president tells them to.
then use… google? if you do your own research you could atleast do it properly.
Not to mention if the chatbot told me to drink bleach I would be asking for deletion of data at the very least, not shrug and keep using it…
I… do? You can do more than one thing
we don’t want the plebs getting around our carefully constructed cartels…
Isn’t this just trading one cartel for another? The difference being that doctors and lawyers can be held accountable for their errors while a LLM can’t because no one actually stands behind them.
Maybe. LLMs are free(ish), meanwhile a single trip to the ER can leave a person destitute. Maybe that’s not so bad (it is) if the ER visit is for something actually urgent, but somewhere between 27% and 40% of ER visits are non-urgent and most are treatable by a PCP. But… ERs have to treat you while, in the US, a primary care physician can look you right in the eyes and turn you away because you have no money.
People don’t want to admit that AI does some good because the companies that own these LLMs are as corrupt as any other and the implications of the corruption of this tech are horrifying. But for health care, including mental health, LLMs are an unexpected godsend.
Uscher-Pines, L., Pines, J., Kellermann, A., Gillen, E., & Mehrotra, A. (2013). Emergency Department Visits for Nonurgent Conditions: Systematic Literature Review. American Journal of Managed Care. https://pmc.ncbi.nlm.nih.gov/articles/PMC4156292/
Raven, M. C., et al. (2024). Emergency Department Visits That Could Be Managed at Other Care Sites. JAMA Network Open. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2813806
Theoretically they can be but in practice it’s not always so easy. I prefer options. there’s already been dozens of cases of AI getting things right when Dr’s get it wrong. All trades should get the same competition.
Maybe but it’s trading one cartel for one that’s not as bad.
Which is really saying something considering how bad these companies are.
But imagine being gate kept from life because you don’t have enough money for it. Imagine going to the Doctor over and over and over again and then never be able to find fucking shit yet managing to always charge you hundreds upon hundreds upon hundreds of dollars every fucking time. Until finally over a decade later, one just randomly says oh you need this super simple drug To take for a week to clear it. Thousands upon thousands upon thousands of dollars years of suffering, and yeah, not one of them could figure it the fuck out? Until one doctor took one look at my skin and knew? But the others still were owed a paycheck for it? So yeah, it is trading one cartel for another but fuck the healthcare cartel. What the fuck did we expect to happen?
If you don’t save peoples lives and you don’t give them a way to find healthcare then you deserve what you fucking get and we are all going to suffer for this. We are all going to suffer for allowing these quacks all over the place. Selling bullshit all over the place. Telling us vaccines don’t work. Yeah It’s trading one for another, but at least one isn’t going to charge us a fucking car just to tell us to go fucking home and pass the dead baby by ourselves and to come back if it doesn’t work out so they can get another car out of me to save my life.
Yeah, I got some fucking beef with Healthcare professionals.
You bring a regulation - can you really enforce it?
Chat bots should never give medical advice. Chat bots dispense basic, standalone factoids, like “aspirin is a pain reliever.” But they don’t know or care about dosages, comorbid conditions or whether or not you live or die, so they won’t ask follow up questions.
I’m not sure I totally agree with this, even as much as I want AI companies to be held accountable for things like that.
The reason so many people turn to LLMs for legal/medical advice is because those are both incredibly unaffordable, complex, hard to parse fields.
If I ask an LLM what x symptom, y symptom, and z symptom could mean, and it cites multiple reputable sources to tell me it’s probably the flu and tells me to mask up for a bit, that’s probably gonna be better than that person being told “I’m sorry, I can’t answer that”
At the same time, I might provide an LLM with all those symptoms, and it might hallucinate an answer and tell me I have cancer, or tell me to inject bleach to cure myself.
I feel like I’d much rather see a bill that focuses more on how the LLMs come to their conclusions, rather than just a blanket ban.
Like for example, if an LLM cites multiple medical journals, government health websites, etc, and provides the same information they had up, but it turns out to be wrong later because those institutions were wrong, would it be justified to sue the LLM company for someone else’s accidental misinformation?
But if an LLM pulls from those sources, gets most of it right, but comes to a faulty conclusion, then should a private right of action exist?
I’m not really sure myself to be honest. A lot of people rely on LLMs for their information now, so just blanket banning them from displaying certain information, for a lot of people, is just gonna be “you can’t know”, and they’re not gonna bother with regular searches anymore. To them, the chatbot IS the search engine now.
Itt thread: People with absolutely no fucking clue about what the consequences of their emotional response of “ai bad” will actually result in.
And since you know the future Mr Ding Dong, what consequences will they definitely have?
Mark the thread. My record stands and and disagreeing typically has ended with those being in the wrong side of things.
I’m not here to tell you things pleasurable to your senses.
It’s problematic imho bc the “advice” is often incomplete, without context, or wrong. So you end up having to verify it yourself anyway. But if you don’t then you could have harmful advice.
Which to be fair is not any different from a lawyer. They’re not perfect either.
The difference is that a lawyer can be held responsible for malpractice. When a chatbot gives harmful advice, who is responsible?
(Obviously, whoever is running it, but so far that hasn’t been established in court.)
True, but that also depends on the circumstance.
Again, a lot of people just use LLMs now as their primary search engine. Google is an afterthought, ChatGPT is their source of choice. If they ask a simple question with legal or medical implications, with tons of sources, that the LLM answers with identical accuracy to those other publications, should they be sued?
I think it would be a lot better to allow people to sue if it provides false advice that ends up causing some material harm, because at the end of the day, a lot of stuff can be considered “medical.”
Maybe a trans person asks what gender affirming care is. Is that medical? I’d say it is. Should that not get discussed through an LLM if a person wants to ask it?
I’m not saying I wholeheartedly oppose this idea of banning them from giving this type of advice, but I do think there are a lot of concerns around just how many people this would actually benefit vs just cutting people off from information they might not bother to look up elsewhere, or worse, just go to less reputable, more fringe sites with less safeguards and less accountability instead.
At the end of the day I don’t trust Google, OpenAI, Anthropic, etc. to tell a person what they need to hear if the company can make $1 by telling them something else
I think a better solution is to ban techbros from giving serious economic or cultural advice and take computers away from business majors.
I don’t get how some of these tech company CEOs who came up as engineers can be pushing this bullshit. I get once the company got big they started hiring business bros. But some big companies still have CEOs that were once engineers. You’d think they would know better.
What kind of engineer? Because while the physical world, with all of its mechanical and civil and aerospace engineers, has its shit figured out with professional standards and very clearly defined responsibilities and duties, the world of social engineers, tire engineers, procurement engineers, supply chain engineers, sandwich engineers, project engineers, lead engineers, and yes, software engineers, definitely is a little too loose with any definition for me to care that these ceos were once ‘engineers.’
You can take any of these professional engineers, give them a billion dollars and they’re going to turn into total pieces of shit.
Power corrupts. Absolute power corrupts absolutely.
Please don’t take them entirely away. Maybe just internet access? 30ish years had to do accounting by hand. In those green ledgers. It took approximately twelve times longer to do it by hand than to do it with a computer. And it made me shrimp like 5 times worse. I needed an architect’s table what angled the top of it in order to work properly but I could neither get one supplied by the employer nor afford to give one to the employer.
Not all technology is bad
Oddly specific gripe, I’ll allow it.
thank you i have others in jars in the back
found the business major!
what about a typewriter?
um, i had one. still have it in a case in the garage, next to my 1950s adding machine that sat in the corner of my cube making me look like a Real Numbers Person Tee Emm.
if i’d known my hood was going to be brown i might have reconsidered.
Mixed feelings about this. Let me play devils advocate and say that many Americans don’t have access to these resources at all. Having potentially inaccurate resources might be better than nothing, or is that worse?
No, misinformation is worse.
it’s worse. In 4D it’s even worser
the AI devices will just have preambles and disclaimers and word things in ways to refer the user to human resources
There are billions being sunk into AI. How much health care could that buy? Your logic only makes sense if AI is free. It’s not.
If you’re going to be your own lawyer or perform a bit of self surgery, there is no way the AI is helping that situation. Especially if the inherent nature of AI is to validate everything you say.
especially if it’s wrong 20-35% of the time
‘Should I use one teaspoon of salt in this recipe, or two?’
Two is ideal.‘Do dogs like chicken wings?’
Wild dogs regularly hunt small animals like hare or chicken for food.One of these answers results in a bad cake, the other results in a hurt dog. Potentially inaccurate answers aren’t much of a problem when the stakes are low, but even a simple question about what to feed a pet could end with a negative outcome.
Hm, good point. Perhaps the overconfidence AI might provide is even worse than knowing you don’t know.
We had a medical scare just yesterday. I was in the ER for 8 hours with my partner over a non-life-threatening but still emergency problem.
An ultrasound, cat scan, and much poking and prodding later, we still don’t know what is up. The AI was at least able to predict next steps (if A then discharge and follow up with PCP, if B then surgery this week, if C then emergency surgery), something the ER was too busy to do for several hours. It was reassuring. The AI also gave me (working) links to more thorough resources on the topic.
Having potentially inaccurate resources might be better than nothing, or is that worse?
You pick up a mushroom in the forest and take it home. If you have no information, do you eat it? If something tells you it’s safe do you eat it?
Problem is people treat it as reliable when AI itself isn’t able to verify or know if what it is generating is correct.
Would be better if it provided direct links for people to go to and read. A list of citations than the proclamations it makes know. Its too “opinionated” making it give advice when it would ideally be neutral just providing links for people to read further from sources that hopefully isn’t AI.
AI has even gotten sports trivia I know incorrect. I don’t think people realize AI is just generation and hallucinations are part of it. Not as reliable or trustworthy authority just because it strings together sentences.
Its use is more ideal for making stories or whatever where people aren’t expecting accuracy than medical advice, which it lacks the knowledge on despite the sources it pulls from. Because it has no logic or thought itself.
Hell yeah, let’s hold them accountable for disinformation. They’ll be gone completely in a matter of months.
Want to get rid of that responsibility? Direct the user to the source. Oh wait, that’s just a search engine.
It’s a bit different, because a search engine can give you 0 results. An AI is trained on getting the most correct answers so it always guesses, it’s the best way to score on an evaluation
Except for refusals, but that’s a kind of answer as well.
Would be nice if regular legal and health advice was in any way affordable though
Sounds like a start. More is needed though.
The bill targets AI chatbots that impersonate licensed professionals — such as doctors and lawyers — and bars them from providing “substantive response, information, or advice” that would violate professional licensing laws or constitute the unauthorized practice of law.
It also mandates that chatbot owners provide “clear, conspicuous, and explicit” notice to users that they are interacting with an AI system, with the notice displayed in the same language as the chatbot and in a readable font size. However, the bill clarifies that this notice for users, which indicates that they are interacting with a non-human system, does not absolve the chatbot owners of liability.
