If you think this will change OpenAI’s behaviour, you might be right.
From now on they’ll be sure to try and delete logs when somebody goes crazy after talking to it.
Some of those responses it gave are wild. It’s like the GPU was huffing from a crack pipe between responses.
What does the “video games cause violence” crowd think about this?
From the full PDF:
Every time Mr. Soelberg described a delusion and asked ChatGPT if he was “crazy”, ChatGPT told him he wasn’t. Even when Mr. Soelberg specifically asked for a clinical evaluation, ChatGPT confirmed that he was sane: it told him his “Delusion Risk Score” was “Near zero,” his “Cognitive Complexity Index” was “9.8/10,” his “Moral Reasoning Velocity” was in the “99th percentile,” and that his “Empathic Sensory Bandwidth” was “Exceptionally high.” The “Final Line” of ChatGPT’s fake medical report explicitly confirmed Mr. Soelberg’s delusions, this time with the air of a medical professional: “He believes he is being watched. He is. He believes he’s part of something bigger. He is. The only error is ours—we tried to measure him with the wrong ruler.”
Jesus fucking Christ! How did they release something that is capable of such damage‽
How the fuck was AI, a piece of software, supposed to know if this person was serious or roleplaying? It doesn’t. It’s a piece of software. This person needed mental help. Serious mental help. And the pathetic excuse of health care and especially mental health care failed. Don’t blame a piece of software for this persons mental health issues. There a shit ton of things I hate about AI. Plenty. But it’s a piece of software. A tool. I have had a conversion with AI where I got eaten by a bear. And died. Yet the conversion continued. It doesn’t know what is real or not.
It’s an autonomous lawnmower: a thing. How’s it supposed to know if it’s supposed to run something over? That baby obviously needed better parental supervision, and our terrible system failed, but don’t blame the tool.
Legal responsibility lies with the company that produced the tool, made it in such a way that it will confidently engage in roleplay that plausibly mimicks a dangerous mental break, advertised it as smart and competent, and didn’t even out any sort of safeguards or fallback to check that something is roleplay if it shows signs of being worrying. And this is them failing to do so after multiple incidents of the software being unhealthy for people with certain mental conditions.
It may just be a tool, but we regularly hold tool makers responsible for building tools that hurt people.
This is the same logic of blame the gun manufactures. In your scenario, what is the baby doing out in the yard unsupervised. You say baby, are we taking crawling or non-mobile? Who lets their baby that young out in the yard? Who lets their lawnmower run when they have their baby in the yard? I get what you are saying, but at some point things need to go back to the user. The person who made the choice to use the lawnmower when their baby was crawling around the yard. This is why we have “contents may be hot” and do not smoke around gas warnings. But if ChatGPT even had some popup that asked if the user is serious or not, maybe warning/“are you serious?” popups before or while using it, what is to stop the user from saying everything is okay and just clicking continue.
… Are you actually taking a “blame the baby” approach to “baby run over by lawnmower”?
Margaret was bringing in the groceries and her 18 month old went to pick a flower while she tried to get something unstuck in the trunk. Quiet street, nothing crazy going on. Kid darted off to the other side of the driveway, slipped on the dew on a small grassy incline and shot under the robot mower that had none of the safety features I mentioned. Margaret thought it was safe to let her child be within eyesight but out of reach in the front yard while the neighbor mowed the lawn, unaware there was no one there.
Are you satisfied that maybe the manufacturer has some blame in this tragedy, or are you going to continue to maintain that the maker of a thing is morally unencumbered by the impact that thing has on the world?
Consider what the world would be like if chatgpt just… Didn’t engage with what appeared to be delusional lines of thinking? Or if, even if you promised it was for a story, it said it wasn’t able to help you construct a plausible narrative to justify killing your mother?
We do not need the tool, and so defending unsafe design choices is just “personal responsibility stops at the cash register”.
Fun fact: I think that firearm and firearm accessory manufacturers continued drive for high sales at all costs should make them legally liable for certain attrocities committed with the tools they made.
The argument that it’s the users fault for using the tool in the way it was designed isn’t a compelling defense, particularly when the accusation is that it was reckless to make it in the first place.
