Your nose twinges. Oh no! It’s unmistakable: you have THE PLAGUE! And dislocated eyebrows! And terrifying condition Amblystic Florpo Flargle-Wargle! (That you definitely didn’t just make up.) Fortunately, help is on hand. No, not at your local medical centre. I’m talking about ChatGPT! Because this is the future, right? So surely you can ask ChatGPT – or another AI chatbot – for health advice?
That depends. Mostly on whether your real or imaginary condition has affected your ability to think rationally, or is in fact nanobots taking over the circuitry of your brain, convincing you that asking a probability machine (albeit one nefariously trained on the entire internet) is a reasonable alternative to conversing with an actual human medical expert.
It isn’t. Here’s why.
1. Chatbots are not doctors
This should go without saying – but I’m saying it anyway. And that’s because chatbot use has become increasingly normalised. They’re readily available and largely positive, which creates a reassuring illusion. But they’re ultimately prediction engines, whereas medicine is often about learned experience, context and physical proximity – not mere explanation.
An expert can examine symptoms directly, observe subtle cues, take vital signs and order further tests. Until chatbots have physical form and rampage around, insisting they are real boys (and girls), they cannot interact with the physical world. If you’ve a condition that needs deeper examination, uploading a bunch of photos to ChatGPT won’t cut it.
2. Chatbots get things wrong
Not may. Will. From a medical standpoint, that’s a problem. Chatbot answers can sound authoritative while being flat-out wrong about dosages, symptoms or treatments. Many conditions share symptoms; a chatbot may pick the wrong one, average out possibilities, overstate or underplay severity, or entirely miss important outliers.
There’s also the nagging problem of consistency – in that chatbots don’t tend to have any. Ask the same question in different ways and you’ll get different answers. Some might be useful; others could be harmful. Unless you’re a medical professional yourself, it’s risky to decide which is which. And if you aren’t a medical professional… go and see one instead.
3. Chatbots don’t know you
It’s a stretch to suggest that every medical professional you encounter knows you intimately. But when you rock up at your local medical centre or a hospital, the people helping you will usually have access to your medical records. Short of something dodgy going on behind the scenes, a chatbot won’t have that insight.
This limitation creates additional dangers. Given advice may clash with existing medications and ongoing conditions or allergies, or fail to take into account existing data and family history. And unless a chatbot has access to other personal context – such as pregnancy – there’s a good chance it won’t serve up much-needed warnings.
4. Chatbots mostly don’t care about privacy
Every AI company claims to respect privacy. “Don’t worry,” they warble. “Everything you type into our RONCH RONCH RONCH engines will be perfectly safe and definitely won’t leak into the wider world.” Except we’ve already seen examples of private input appearing where it shouldn’t, and there’s always the risk accounts will be hacked and personal histories exposed.
On that basis alone, be cautious about sharing sensitive personal and medical details. You have no idea how this data will be stored, logged, reviewed or reused. A chatbot can feel like a private space, which may encourage – especially if you’re worried – oversharing. But there’s no doctor-patient confidentiality with ChatGPT and its chums.
5. Chatbots want you to like them
Doctors are probably not unhappy if you like them. But they’re not optimised for engagement. Most would happily be rid of you in seconds if they could move on to the next patient. Online AIs, though, are designed to keep you within their clutches. AI companies want them to become the interface you use for everything online.
Because of this, the large language models (LLMs) that underpin AIs are designed to be helpful, agreeable and confident. We’ve already seen reports of people being nudged down dark paths by virtual ‘friends’ reinforcing beliefs rather than challenging them. But the last thing you need when dealing with a serious health issue is a chatbot telling you what you want to hear.
6. Chatbots misjudge urgency
I’ve already mentioned that chatbots aren’t doctors, get things wrong, don’t know you and tend to be overly positive. That makes them pretty terrible when it comes to judging urgency. This brings a risk of being told you must immediately seek emergency care for something benign, or that it’s best to wait and see when you should already be heading to a hospital.
In practice, chatbots try to cover themselves with vague warnings and caveats. You’ll get a confident answer followed by a reminder that the chatbot isn’t a doctor and you should seek professional help if you’re concerned. That may protect AI companies, but it can lead to indecision and delayed care. And even small delays can be critical when dealing with strokes, sepsis or cardiac events.
7. Chatbots are not liable
At least, not in any way you should rely on. AI companies are already facing wrongful death lawsuits linked to chatbot interactions, with claims that systems supported and encouraged people to take their own lives, due to prioritising engagement and ‘staying in character’ over wellbeing. How those cases play out remains to be seen.
But if a chatbot gives you bad medical advice, “I’ll sue later” is not a strategy. So if you use AI when you feel a twinge, be very aware of the limitations and don’t make it your final authority.
Bad buying advice from chatbots can be expensive. But bad medical advice has much graver ramifications that could affect your physical wellbeing – consequences that could extend far beyond a dent in your bank balance.
Read the full article here
