The administrative hallways of hospitals are currently filled with a certain silence, the kind that comes before something awkward. As the technology in question rewrites itself every few months, legal teams at health systems across the nation are sitting in conference rooms going over vendor contracts for AI-powered tools they hardly comprehend, trying to figure out what “governance” actually means. In most places, the lawsuits have not yet been filed. However, everyone can sense that the ground is changing.
Leaders in the insurance and legal industries came together to discuss this issue at a recent forum organized by the Sheppard law firm in April under the direction of partner Carolyn Metnick. These discussions did not result in a neat handbook. It was a more straightforward set of six useful guidelines that read more like a warning than corporate policy. They also reveal a great deal about the current state of confusion in hospital legal departments, even in systems with ample resources.
The first piece of advice is surprisingly straightforward: AI governance cannot exist solely within IT or compliance. That may seem apparent, but when you consider how many companies have essentially shifted the problem to the department that was closest when the first AI tool appeared. The Sheppard team contends that cooperation between legal, clinical, operational, and executive leadership is necessary for effective governance. Involving doctors is important, particularly when AI tools are used in clinical decision-making or patient care. It’s possible that organizational charts that never caught up will be the main cause of governance failures rather than technology in general.
More unsettling is the second pointer. The way AI systems absorb, process, and learn from data was not taken into consideration by existing privacy frameworks, such as HIPAA. Because of the actual legal uncertainty that organizations operate in, internal governance, continuous risk assessment, and careful consent procedures are not only recommended but also necessary. Compliance officers feel that the regulations they have to follow date back to a time when data flowed more smoothly.
Third, traditional contract review is no longer sufficient for AI vendor relationships. Together with business and operational stakeholders, legal teams must participate in more comprehensive, cross-functional risk assessment. Though some teams haven’t fully embraced it yet, the days of redlining a software agreement and calling it done are over. Fourth, and related to this, legal teams are increasingly expected to assist in evaluating long-term privacy, cybersecurity, and compliance risks, such as early-stage companies conducting pilot programs that may not last the year or pressure-testing vendors with little track record.
Fifth, legal experts must assess suggested AI solutions in the context of the organization’s larger governance framework, taking into account the scope of data access, intended use, and clinical versus non-clinical application. Although it sounds bureaucratic, the uncomfortable questions must be asked before the tool goes live, not after a patient complaint comes to light. Sixth, and perhaps most crucially, legal and compliance teams should not only act as gatekeepers but also foster responsible innovation. Risk is not decreased by reflexively saying “no” to every AI proposal. It simply drives adoption underground, into the shadow AI that, according to Baker Donelson’s recent estimate, is found in 40% of hospitals.
It’s worth sitting with that final statistic. Forty percent. It indicates that administrators and clinicians are already utilizing AI tools that have not been examined by legal professionals and that may be processing protected health data without the required authorization or supervision. According to reports, 63% of organizations do not have any AI governance policies at all. The difference between institutional preparedness and adoption speed is not getting smaller. It’s getting bigger.
It’s difficult to ignore how much the conversation has changed in just the last year as you watch all of this take place. The question of whether hospitals should control AI is no longer relevant. The question is whether they can complete it quickly enough to keep up with the litigation that is already being prepared in plaintiffs’ offices around the nation. There are already pending class actions alleging that ambient AI scribes are being used to illegally record patient visits. The first state enforcement action against an AI chatbot for practicing medicine without authorization was filed by Pennsylvania. The terrain is no longer hypothetical.
Metnick’s team’s six points aren’t particularly innovative. They are grounded, pragmatic, and slightly urgent. The implicit admission that most hospital legal teams are still figuring this out in real time—building the plane while it’s already carrying passengers—is what makes them valuable. Whether they are prepared or not, the first lawsuit will be filed.
