Most AI features in behavioral health software have never had a real HIPAA review. Here is how to check yours before a customer or regulator does.
Your engineering team shipped an AI feature this quarter. Maybe it summarizes session notes, drafts treatment plan language, or powers a chatbot that checks in with clients between appointments. It works, customers like it, and it took less time to build than anyone expected. Here is the uncomfortable question behind all of that: has anyone actually reviewed whether it is HIPAA compliant? For most behavioral health software companies, the honest answer is no. The AI moved from prototype to production faster than the compliance review cycle could keep up, and now there is a feature in the wild touching protected health information that no one has formally evaluated.
HIPAA does not mention artificial intelligence anywhere. The Privacy Rule and Security Rule were written for a world of databases, fax machines, and file cabinets. The rules are technology neutral by design, so if PHI flows through a system, that system is in scope, no exceptions. What actually changes with AI is not the regulation. It is the architecture. Traditional software moves PHI through paths your team designed and can audit. An LLM integration introduces paths a team may not fully see: prompts that carry patient data to a third party API, model providers that may log, retain, or train on inputs depending on contract terms, generated outputs that can reproduce or infer sensitive details, and vector databases or embeddings that quietly store transformed PHI somewhere your data inventory never captured.
Each of those is a place PHI now lives or travels. If your risk analysis was written before your AI features shipped, it does not cover them. Risk analysis failures were the most common violation cited in HIPAA enforcement actions in 2024, appearing in more cases than any other finding, which makes an incomplete risk analysis one of the more common and avoidable ways teams end up on OCR's radar.
If you build software for behavioral health, the stakes sit above general healthcare tech. Therapy notes, psychiatric diagnoses, and substance use history are among the most sensitive categories of health data that exist. Some of it is covered not just by HIPAA but by 42 CFR Part 2, which layers stricter consent requirements onto substance use disorder records specifically.
A breach involving a lab result is bad. A breach involving therapy transcripts is a different category of harm, both for the patient and for a company's credibility with every clinic and health system it sells to. Behavioral health buyers already know this, which is why their security questionnaires keep getting longer and their diligence on AI features keeps getting sharper. Saying an AI feature is used responsibly is no longer enough on its own. Buyers want to know exactly what data goes where.
When reviewing AI integrations in behavioral health platforms, four questions tend to surface most of the risk quickly.
Does PHI reach the model provider, and under what agreement? If a product sends patient data to an LLM API, that provider is functioning as a business associate, and HIPAA requires a signed Business Associate Agreement before PHI flows. Many major AI providers now offer BAAs for their API products, but the details matter. Consumer tier products and default configurations are often not covered, and a BAA on one service does not automatically extend to every other service the same vendor offers. Sending PHI without one in place is not a gray area.
What happens to the data after the API call? Retention, logging, and training policies vary widely between providers and between tiers of the same provider. Zero retention options often exist but usually have to be explicitly configured or contractually arranged. A team that integrated with default settings during a prototype phase and then shipped may have patient data being retained somewhere its own privacy policy never mentions.
Can the AI feature leak PHI in its outputs? Models can reproduce information from their context in unexpected places, a summary meant for one clinician that includes another patient's details from a poorly scoped prompt, or a chatbot that surfaces fragments of conversation history across sessions. Cross patient contamination is an access control failure, and the Security Rule treats it that way.
Is any of this in your risk analysis and audit trail? Even if the first three answers are clean, that has to be demonstrable. Compliance is not only about the architecture being safe. It is about having reviewed it, documented it, and having audit controls that show who accessed what. AI features frequently skip the logging discipline the rest of a platform already follows.
There is no HIPAA certification for AI features, no stamp a vendor can hand over. Compliance is a posture a team builds and maintains. For an AI feature in a behavioral health product, that posture generally includes a current risk analysis that explicitly covers the AI data flows, BAAs with every vendor that touches PHI verified against the specific services in use, data minimization in prompts so a model only sees what it needs, retention and logging configurations that match stated policies, output handling that prevents cross patient exposure, and monitoring that would actually catch a failure. None of this requires abandoning AI. It requires applying the same engineering discipline a team already uses on its database and its APIs to a newer surface area, the model layer. Reviewing exactly that is what our AI Acceleration Sprint is built around.
Teams usually put off this review for an understandable reason. Everyone is busy shipping, and nothing has gone wrong yet. The timeline rarely cooperates for long, though. The most common forcing functions are a large customer's security review before a contract renewal, a prospect's diligence process during a sales cycle, or, worst case, an actual incident. In the first two scenarios, an unreviewed AI feature stalls revenue. In the third, breach notification for behavioral health data becomes a public event with a long tail. Teams that come out ahead are the ones who find their own exposures first, on their own schedule, and can walk into a security review with documentation instead of improvisation. Our pricing page lays out what that review costs against what waiting tends to cost instead.
Is it HIPAA compliant to use ChatGPT with patient data? Consumer chat products are generally not covered by a BAA and should not touch PHI. API and enterprise offerings from major providers can be used compliantly, but only with the right agreement and configuration confirmed in advance.
Does de-identifying data before sending it to an LLM solve the problem? It can, if the de-identification actually meets the HIPAA standard under Safe Harbor or Expert Determination. In practice, free text clinical notes are extremely hard to reliably de-identify, and behavioral health narratives are full of quasi-identifiers. Treat de-identification as a rigorous engineering task, not a preprocessing checkbox.
We are a vendor, not a provider. Does HIPAA even apply to us? If customers are covered entities and the software touches their patients' PHI, the vendor is a business associate, directly liable under HIPAA for Security Rule compliance and breach notification. Any AI subprocessors then need their own agreements in place too. More of these scenarios are covered on our FAQ page.
How long does a HIPAA review of an AI feature actually take? A focused review of a single feature can surface the major gaps within the first two weeks. A full sequence to close everything found, including build work, typically runs closer to six weeks.
If the four questions above cannot be answered confidently for every AI feature in a product, that is the gap. Resolve Health Tech runs a six week AI Acceleration Sprint built for exactly this. Contact us to find out where your own AI features actually stand.
Sana is a Technical Content Specialist at Resolve Health Tech. She specializes in breaking down complex architectural patterns, nearshore hiring trends, and software engineering workflows into actionable, human-friendly guides. Working alongside Resolve Health Tech's tech team, Sana ensures every piece of content is both highly readable and technically precise.