Can You Use ChatGPT with Patient Data? What HIPAA Actually Says
Using ChatGPT or any consumer LLM with patient data creates real HIPAA risk. Here is what the rules actually require before you connect one.
This is some text inside of a div block.
Using ChatGPT or any consumer LLM with patient data creates real HIPAA risk. Here is what the rules actually require before you connect one.
A clinician pastes a session note into ChatGPT to draft a summary faster. An engineer runs a support ticket containing patient details through a free AI assistant to troubleshoot an issue. Neither action necessarily feels like a compliance violation in the moment, and that is exactly the problem. Under HIPAA, both are almost certainly a disclosure that was never supposed to happen.
Consumer ChatGPT, the free or individual paid tier most people are familiar with, is generally not covered by a Business Associate Agreement and should not touch protected health information under any circumstances. OpenAI's business and enterprise offerings, along with API access under the right agreement, can be used compliantly, but only with a signed BAA and the correct configuration confirmed in advance, not assumed. The distinction between the consumer product and the covered enterprise or API offering is the single most important thing to understand here, and it is exactly the distinction most people miss.
This is not a hypothetical risk. Unauthorized or ungoverned AI tools were involved in roughly 20 percent of data breaches studied in one recent analysis, nearly all at organizations without proper access controls around AI tool usage in place. This pattern, often called shadow AI, tends to happen for an understandable reason. A workflow is slow, a free AI tool solves the immediate problem, and the person using it is rarely thinking about business associate agreements in the moment they are trying to finish a task quickly. The behavior is rarely malicious and almost never sanctioned, which is exactly why it tends to go unnoticed until an audit or an incident specifically looks for it.
The core issue is not that ChatGPT is inherently insecure. It is that the consumer product was never built with a HIPAA relationship in mind. Consumer tier usage typically has no BAA available at all, which means there is no legal agreement in place covering how patient data would be handled if it were sent there. Retention and training policies on consumer tiers may also allow inputs to be retained or used in ways that are simply incompatible with protected health information, regardless of how the data is used afterward. Sending PHI to a service with neither a covering agreement nor retention guarantees appropriate for patient data is a disclosure without authorization, not a gray area open to interpretation.
A case manager at a behavioral health platform is behind on notes at the end of a long day and pastes a client's session summary into ChatGPT to help draft a cleaner version for the record. It takes thirty seconds and solves an immediate problem. It is also, under HIPAA, a disclosure of protected health information to a third party with no signed agreement covering it, no matter how brief the interaction was or how helpful the outcome felt. Multiply that moment across a team of clinicians over months, and a platform can accumulate a meaningful amount of exposure through individually small, well intentioned actions that nobody flagged as a problem in the moment.
Using an LLM provider compliantly with patient data requires a few specific things to be true at once, not just one of them. A signed BAA needs to exist and needs to explicitly cover the specific service, endpoint, and tier actually in use, since a BAA on one product does not automatically extend to every other product the same vendor offers. The technical configuration needs to match what that agreement assumes, appropriate data retention settings, training opted out where required, and the correct region or deployment type. And the data reaching the model needs to be limited to what the feature actually requires, rather than an entire patient record passed along for convenience.
Sometimes, but it needs to be done properly, not assumed. If data reaching an AI vendor is genuinely de-identified under HIPAA's Safe Harbor method or a formal expert determination, it is no longer PHI, and no BAA is required for that specific flow. The complication is that free text clinical content, especially behavioral health narratives, resists reliable de-identification. Names, family details, and unusual circumstances can make a record identifiable even after standard identifiers are stripped out. Treating de-identification as a rigorous engineering task that gets tested against real data, rather than a preprocessing checkbox, is the difference between a legitimate strategy and a false sense of safety.
Policy alone rarely works, since the behavior usually comes from people trying to solve a real problem quickly, not from anyone intending to create risk. The more effective approach pairs a clear policy with a fast, sanctioned alternative, ideally an AI feature or approved enterprise tool that solves the same problem the free tool was being used for, so there is no reason to reach for an ungoverned option instead. Auditing what AI tools are actually in use across engineering, support, and clinical workflows is the only reliable way to find where this is already happening, since it rarely shows up on an official integration list. This is part of what our AI Acceleration Sprint checks for specifically, alongside the sanctioned AI features already built into a product.
An unauthorized disclosure through a consumer AI tool is treated the same as any other HIPAA violation once discovered, regardless of how small or well intentioned the original action was. Our pricing page lays out what a proactive audit of AI tool usage costs against the alternative of finding this gap through a breach or a customer's security review instead.
Resolve Health Tech audits exactly where AI tools, sanctioned and unsanctioned, actually touch patient data across a platform and a team. Contact us to find out where that gap exists in your own organization.
Author: Sana Fatima
Sana is a Technical Content Specialist at Resolve Digital. She specializes in breaking down complex architectural patterns, nearshore hiring trends, and software engineering workflows into actionable, human-friendly guides. Working alongside Resolve Health Tech's tech team, Sana ensures every piece of content is both highly readable and technically precise.
Sana is a Technical Content Specialist at Resolve Health Tech. She specializes in breaking down complex architectural patterns, nearshore hiring trends, and software engineering workflows into actionable, human-friendly guides. Working alongside Resolve Health Tech's tech team, Sana ensures every piece of content is both highly readable and technically precise.