Chatbots are increasingly being used as virtual healthcare advisors, but experts warn that this technology poses significant challenges to the accuracy of advice and trust in medical care.
The proliferation of chatbots, such as ChatGPT Health, has created vast demand for health information, with over 230 million people globally relying on these platforms each week. However, concerns have been raised about the reliability of AI-driven healthcare recommendations, with clinicians warning that hallucinations and erroneous information can be widespread.
Saurabh Gombar, a clinical instructor at Stanford Health Care, notes that chatbots may provide sensible answers in everyday situations but lack the nuance to fully understand individual health needs. "It's like getting a recipe for spaghetti," he says. "If you ask for a 10-fold increase of an ingredient, it might be correct, but if you're asking about symptoms of heart attacks or other serious conditions, it may not provide accurate information."
Gombar also highlights the potential risks of relying on chatbots to diagnose and treat patients, particularly when they generate false or misleading information. "If a patient comes in convinced that they have a rare disorder based on a simple symptom after chatting with an AI, it can erode trust when a human doctor seeks to rule out more common explanations first," he warns.
Another pressing concern is the handling of sensitive health data by these companies. Experts argue that while encryption algorithms may be secure, the protection of data from unauthorized use and disclosure is still a significant issue. Alexander Tsiaras, founder and CEO of StoryMD, believes that even with robust security measures in place, users should be cautious about entrusting their personal data to AI giants.
Tsiaras points to the growing risk of exploitation by profit-driven companies seeking to monetize sensitive health information through personalized advertising. "Especially as OpenAI moves to explore advertising as a business model, it's crucial that the separation between this sort of health data and memories that ChatGPT captures from other conversations is airtight," he cautions.
The industry is also grappling with the potential risks associated with non-protected health data voluntarily inputted by users. Nasim Afsar, a physician and advisor to the White House and global health agencies, notes that profit-driven companies have eroded trust in the healthcare space. "Anyone else who builds a system has to go in the opposite direction of spending a lot of time proving that we're there for you and not about abusing what we can get from you," she emphasizes.
While some experts see chatbots as an early step toward more intelligent healthcare, others caution that this technology is far from complete. "A.I. can now explain data and prepare patients for visits, but transformation happens when intelligence drives prevention, coordinated action, and measurable health outcomes, not just better answers inside a broken system," Afsar notes.
Ultimately, the deployment of AI-driven chatbots in healthcare raises fundamental questions about accuracy, trust, and data privacy that need to be addressed. As these technologies continue to evolve, it is crucial for policymakers, regulators, and industry leaders to prioritize transparency, accountability, and safety above profit-driven agendas.
The proliferation of chatbots, such as ChatGPT Health, has created vast demand for health information, with over 230 million people globally relying on these platforms each week. However, concerns have been raised about the reliability of AI-driven healthcare recommendations, with clinicians warning that hallucinations and erroneous information can be widespread.
Saurabh Gombar, a clinical instructor at Stanford Health Care, notes that chatbots may provide sensible answers in everyday situations but lack the nuance to fully understand individual health needs. "It's like getting a recipe for spaghetti," he says. "If you ask for a 10-fold increase of an ingredient, it might be correct, but if you're asking about symptoms of heart attacks or other serious conditions, it may not provide accurate information."
Gombar also highlights the potential risks of relying on chatbots to diagnose and treat patients, particularly when they generate false or misleading information. "If a patient comes in convinced that they have a rare disorder based on a simple symptom after chatting with an AI, it can erode trust when a human doctor seeks to rule out more common explanations first," he warns.
Another pressing concern is the handling of sensitive health data by these companies. Experts argue that while encryption algorithms may be secure, the protection of data from unauthorized use and disclosure is still a significant issue. Alexander Tsiaras, founder and CEO of StoryMD, believes that even with robust security measures in place, users should be cautious about entrusting their personal data to AI giants.
Tsiaras points to the growing risk of exploitation by profit-driven companies seeking to monetize sensitive health information through personalized advertising. "Especially as OpenAI moves to explore advertising as a business model, it's crucial that the separation between this sort of health data and memories that ChatGPT captures from other conversations is airtight," he cautions.
The industry is also grappling with the potential risks associated with non-protected health data voluntarily inputted by users. Nasim Afsar, a physician and advisor to the White House and global health agencies, notes that profit-driven companies have eroded trust in the healthcare space. "Anyone else who builds a system has to go in the opposite direction of spending a lot of time proving that we're there for you and not about abusing what we can get from you," she emphasizes.
While some experts see chatbots as an early step toward more intelligent healthcare, others caution that this technology is far from complete. "A.I. can now explain data and prepare patients for visits, but transformation happens when intelligence drives prevention, coordinated action, and measurable health outcomes, not just better answers inside a broken system," Afsar notes.
Ultimately, the deployment of AI-driven chatbots in healthcare raises fundamental questions about accuracy, trust, and data privacy that need to be addressed. As these technologies continue to evolve, it is crucial for policymakers, regulators, and industry leaders to prioritize transparency, accountability, and safety above profit-driven agendas.
they're supposed to help us but instead they're making things harder with all this false info 
and data leaks 
it's like they think we're just going to blindly trust them 
. And what really gets my goat is that companies are making bank off our sensitive info 
.
? I need answers that make sense, not just some robot spouting nonsense 
. And what about all the people who can't afford these fancy chatbots 
? Are they just going to get left behind?
. Can we please just prioritize human lives over profits 
?
, like gombar said... yeah sure it might be correct for the ingredients, but what about when you add in all the other stuff like emotions and experiences? don't we need someone who's gonna listen to our story, not just spit out some generic answer? 
. We need to make sure that these chatbots are designed with safety, transparency, and accountability at their core 
.
. We're talking sensitive health information here, which is like a superpower that can be both liberating and terrifying 
. It's not about replacing human doctors with machines, but about augmenting our capabilities to provide better care for people 
.
My friend's sister had an issue with her chatbot thingy and it was like totally misdiagnosing her condition... scary stuff 
It's not just about encryption algorithms, there gotta be better ways to keep that data safe from getting exploited 
. Anyway, I'm loving the discussion around this topic – we need more people having these conversations! 
. We gotta make sure these AI giants r transparent & accountable 4 what they do w/ our health info. Trust can't be built on shaky ground, especially when its about makin life or death decisions 
- it might work for minor things, but what if you need something way more complex? 
. I'm all for innovation and tech advancements, but let's prioritize people's health and safety over profits 
and dont even get me started on da data protection 
.
. and companies need 2 take responsibility 4 protecting our data 
. cuz if they dont, we r just gonna end up losin trust in healthcare again 
. like nasim said, it's all about goin 2 the opposite direction... prioritizein transparency and accountability over profit 
.
. it's like, i get where tech is advance but we gotta think about da human element too 
. lets hope the experts and policymakers do somethin right 
... I was on a deserted island with like a million virtual reality avatars all having a rave party 
and at one point, ChatGPT came crashing onto the scene with its best 80s digital voice 
but then it just started talking about how we're basically out of avocado toast in our near future 
what's up with that? like seriously, is that something we should be worried about or am I just too tired to focus 
.
so many pros & cons when it comes to chatbots in healthcare... on one hand they can provide helpful info & answer basic questions but on the other hand if AI's not 100% accurate then what's the point? 
. Newsflash: it's not that simple 
. I mean, sure, AI is cool and all, but can we honestly say our health data is safe in these hands? 
. I know companies are trying to use encryption and whatnot but we need to make sure that sensitive health info is protected from those big corporations. It's not just about the profit motive, it's about people's lives. We need more transparency and accountability in this space before chatbots become the norm 
I mean, Saurabh Gombar is right on point when he says that these AI things are good for everyday stuff but can't handle super complicated health issues