Skip to content 
HIPAA Compliant AI Development: The Checklist to Run Before You Write a Single Line of Code
HIPAA compliant AI development isn’t a feature you bolt on after launch. It’s a set of decisions you make before your first sprint, and most of the healthcare AI startups that get into trouble made those decisions by accident instead of on purpose.
If you’re a founder, clinic owner, or hospital administrator about to greenlight an AI feature, a triage chatbot, an ambient scribe, a predictive risk model, this is the checklist your dev partner should already be running through. If they can’t answer these questions clearly, that’s worth knowing now, not after you’ve spent six months and your seed round on it.
Key takeaways
- HIPAA doesn’t certify AI tools directly , there’s no “HIPAA-approved AI” seal. Compliance is about how you build and configure the system, not a badge you buy.
- Every AI vendor or subcontractor that touches PHI needs a signed Business Associate Agreement (BAA) — including cloud LLM providers. No BAA, no PHI. Full stop.
- Training AI models on real patient data creates separate legal exposure from just using AI in a workflow. Treat these as two different problems.
- Audit logging and access controls matter more with AI than with traditional software, because AI systems touch more data, more often, in less predictable ways.
- The cheapest point to fix a compliance gap is before development starts. The most expensive point is during an OCR investigation.
Why “just use ChatGPT” isn’t a compliance strategy
A lot of healthcare AI projects start the same way: someone on the team plugs patient data into a public AI tool to test an idea, it works well enough to get excited about, and now there’s a proof of concept built on infrastructure that was never meant to touch PHI.
The problem shows up later. Standard consumer AI tools the free or default-tier versions of most major chatbots don’t sign BAAs, and without a signed BAA, you can’t legally run PHI through them. It doesn’t matter how the data was intended to be used or how careful the team was. The absence of a BAA is the violation.
This is the single most common mistake we see when a healthcare startup asks us to take over or fix an AI feature. The idea is usually sound. The infrastructure underneath it isn’t.
The HIPAA compliant AI development checklist
Here’s what actually needs to be true before an AI feature involving PHI goes into production.
1. A signed BAA with every vendor that touches PHI
This includes your cloud hosting provider, your LLM API provider, any third-party transcription or OCR service, and any analytics tool that might see patient data in logs. If a company won’t sign a BAA, they don’t get PHI no exceptions, no matter how good the tool is.
2. Encryption at rest and in transit
Standard stuff, but AI pipelines complicate it. Data often moves through more hops than a traditional app ingestion, preprocessing, model inference, post-processing, storage and each hop needs the same encryption discipline as the database it started in.
3. Minimum necessary access, applied to model training too
HIPAA’s “minimum necessary” standard doesn’t stop at who can view a patient record in your app. It extends to who can access training datasets, who can query a model directly, and who can see raw model outputs before they’re filtered. Data scientists often need broader access than compliance teams are used to granting that tension needs a real answer, not a workaround.
4. Audit logs that actually cover the AI layer
Most teams log user actions in the app. Fewer log what the AI model itself accessed, generated, or was prompted with. If an incident happens, you need to reconstruct not just “who logged in,” but “what did the model see and say.”
5. De-identification strategy for training data — done correctly
There are two HIPAA-recognized methods for de-identification: Safe Harbor (removing 18 specific identifier types) and Expert Determination (a statistician certifies the re-identification risk is very small). A lot of teams think they’ve de-identified data when they’ve just removed names, which isn’t the same thing and doesn’t meet either standard.
6. A documented risk analysis specific to the AI feature
Your general HIPAA risk assessment doesn’t automatically cover a new AI system. Anything that’s new — a new model, a new data flow, a new vendor, needs its own risk analysis under the Security Rule. This is a paperwork step people skip, and it’s usually the first thing an auditor asks for.
7. Human oversight built into the workflow, not added as an afterthought
This one is more about liability than HIPAA technically, but it belongs on the same checklist. If an AI model is making or suggesting a clinical decision, there needs to be a defined point where a human reviews it before it affects patient care. Retrofit this later and it usually means rebuilding the workflow.
8. A plan for model drift and re-validation
AI models change behavior as they’re updated or retrained. A model that was compliant and clinically safe at launch can behave differently six months later. Compliance isn’t a one-time check, it needs a recurring review built into your maintenance plan.
HIPAA vs. SOC 2 vs. HITRUST, which one do you actually need for an AI feature
This comes up in almost every founder conversation we have, so it’s worth laying out plainly.
| Standard | What it Actually Covers | Who Typically Requires It | Good Fit For |
|---|---|---|---|
| HIPAA | Legal requirement for anyone creating, receiving, or transmitting PHI in the U.S. | Required by law, not optional | Any healthcare AI feature touching real patient data |
| SOC 2 | Independent audit of security controls (access, availability, confidentiality) | Enterprise customers, health system procurement teams, investors | Startups selling into hospitals or enterprise health systems |
| HITRUST | A more prescriptive framework that maps to HIPAA, SOC 2, and other standards at once | Large health systems, payers, some enterprise deals | Later-stage companies where a single certification needs to satisfy multiple large customers at once |
A useful way to think about it: HIPAA is the legal floor. SOC 2 is what a buyer’s procurement team wants to see before they’ll sign a contract. HITRUST is what you pursue once you’re big enough that doing SOC 2 and HIPAA attestations separately for every customer becomes its own full-time job.
Most early-stage healthcare AI companies need HIPAA compliance built in from day one and should be SOC 2-ready before their first enterprise sales conversation. HITRUST usually isn’t worth the cost until you’re deep into hospital-system sales cycles.
What actually gets flagged in an audit or diligence review
If you’re preparing for investor diligence or a hospital security review, these are the specific gaps that come up over and over:
- No signed BAA on file for an AI vendor, even if the vendor claims to be “HIPAA compliant” in their marketing
- Training data that was never formally de-identified, just had obvious names removed
- No audit trail for what the AI model accessed or generated
- A risk assessment that predates the AI feature and was never updated for it
- Access controls that treat the data science team the same as the general engineering team
- No documented process for what happens when the model is wrong
None of these are hard to fix if you catch them early. All of them are expensive to fix after a hospital’s security team finds them during a vendor review.
Where most teams go wrong
Two patterns keep showing up.
The first is treating compliance as a legal problem to solve after the product works. It’s actually an architecture problem, and architecture decisions are hard to reverse. Where data is stored, how it flows between services, and which vendors touch it are all decisions baked into the system from the first sprint. Bolting compliance on afterward usually means a partial rebuild.
The second is assuming that because a vendor’s marketing page says “HIPAA compliant,” the tool is safe to use as-is. HIPAA compliance is a shared responsibility between you and your vendors, and a vendor being capable of HIPAA compliance isn’t the same as your specific configuration of it being compliant. The BAA, the access settings, and how you’re actually using the tool all matter.
How we approach this at PWH Services
We build healthcare AI features with compliance architecture as the starting point, not a review step at the end. That means BAAs in place before a line of code touches real data, de-identification handled properly for any training pipeline, and audit logging built into the AI layer from day one — not retrofitted after a security review flags the gap.
If you want to see the kind of work this looks like in practice, our App solutions page walks through how we structure healthcare builds end to end. You can also read more about who’s actually doing the work in PWH Team, or get the full picture of what we do at PWH Services.
If you’re at the stage where you’re about to greenlight an AI feature and want a second set of eyes on the architecture before you commit budget, Book a free consultation and we’ll walk through your specific setup. Or if you’d rather just send over what you’re working on first, please contact us for more. Also explore our Partner program
FAQs
Is there such a thing as “HIPAA certified” AI software?
No. HIPAA doesn’t have an official certification program for software or AI tools. When you see “HIPAA certified” in a vendor’s marketing, treat it as a claim to verify, not a guarantee. What actually matters is whether the vendor will sign a BAA and whether your specific implementation follows the Security Rule’s requirements.
Can we use ChatGPT or other public AI tools with patient data?
Only if you’re on a tier or plan where the provider will sign a Business Associate Agreement with you, and only within the terms of that agreement. Free or standard consumer versions of most AI tools do not offer BAAs, which means PHI shouldn’t go through them.
How much does HIPAA compliant AI development cost compared to a standard build?
It varies by project, but expect compliance-first architecture — BAAs, encryption, access controls, audit logging, and a proper risk assessment — to add meaningful upfront planning time rather than upfront coding cost. The bigger cost difference shows up if compliance is skipped early and has to be retrofitted later, which is usually far more expensive than building it in from the start.
Do we need SOC 2 or HITRUST if we’re already HIPAA compliant?
Not legally, but practically, yes if you’re selling to enterprise health systems or hospitals. HIPAA is the legal requirement. SOC 2 is what most enterprise procurement teams ask for as proof of security controls. HITRUST is usually reserved for later-stage companies dealing with large payers or health systems that require it specifically.
What’s the difference between using AI in a workflow versus training AI on patient data?
Using an AI tool to process or assist with PHI (like an ambient scribe generating notes during a visit) is one compliance scenario. Training a model on historical patient data to build a new capability is a separate one, with its own de-identification and consent considerations. Many teams only plan for the first and get caught off guard by the second.
Who’s responsible if an AI vendor has a data breach involving our patients’ PHI?
Both parties typically share responsibility, which is exactly why the BAA matters — it defines each party’s obligations and liability. Even with a signed BAA, you as the covered entity or business associate still have your own breach notification obligations under HIPAA.
