Henry Meier, Esq., The Law Office of Henry Meier, Esq.
Let’s say your credit union is ready to take a cautious step into the AI world. It has heard great things about how chatbots can improve customer service and increase credit union efficiency. You reach out to several vendors, each of whom assures you that their product can do exactly what you are requesting without any mistakes. They show you their draft contracts and promise that, as soon as you sign on the dotted line, they can get to work building the AI system of your dreams. As a matter of fact, they have made the approval process even easier by including a 1-page addendum to their standard contract which addresses the integration of AI.
What AI-specific issues should be addressed in the contract? Can they be handled with the simple 1-page addendum the eager salesperson has provided you as an addition to the standard contract? This is one situation where the government can actually help.
A recent nine-page addendum or “clause” to the government’s standard procurement contract for its own use of AI services has been examined by legal experts, and as I read through some of the commentary, it strikes me that some of the issues they highlight are issues that credit unions should analyze when examining AI vendor services. These key issues include but are no means limited to: defining data and its ownership, the strength of representations and warranties, and the facilitation of human oversight.
When it comes to defining who owns what data, AI platforms open up a can of worms. For example, whereas traditional core computing software deals with the input of static information and predictable outputs such as a disclosure or an updated ledger, no one can be sure precisely what your chatbot will say to your membership.
Consequently, the government addresses this issue by defining government data to include not only inputs, but also data outputs. Specifically, the government defines data inputs as including “prompts, queries, instructions, system prompts, source data, documents, knowledge bases, and any other information or content submitted to the AI System by or on behalf of the Government.” Data outputs are also defined to include responses, “analysis, (…) and any other action or output produced by the AI system.”
In the context of your proposed contract, an expansive definition of data helps to ensure the credit union owns not only its data, but the outputs of that data. We have all heard stories about vendors refusing to hand over reports when a contract is terminated, arguing that the output of the software actually belongs to them. This issue becomes even murkier when dealing with AI. What the government is doing with its language is making sure that it has the right to not only its traditional data but the prompts and output resulting from this information.
A second novel issue raised by your AI customer service contract is an unambiguous understanding of what your vendor is guaranteeing. The government does this in a straightforward manner by mandating that the contractor make “commercial efforts to ensure the AI system is ‘truthful’ in responding to user prompts, seeking factual information or analysis,” and that it implements continuous improvement processes to enhance detection and mitigation of performance trustworthiness and bias.
As you negotiate with your vendors, you should evaluate how willing they are to actually guarantee the outputs they are providing. For example, are they willing to warrant that their customer service bot will not discriminate against members, or that the system will be run in conformity with federal laws and regulations? While similar language has always been put in contracts, the dynamic nature of AI makes these guarantees all the more important.
Since the advent of widespread AI use, regulators have stressed the need for human oversight over the platform. For many credit unions, this may be a daunting challenge. The government addresses this issue by making the AI provider responsible for providing summarized processing actions and decision points; model routing decisions with the accompanying rationale; and the data retrieval methods used including “complete source attribution including direct links and relevant excerpts from material used in the response generation.”
The bottom line is that while your credit union will have to train at least some of its employees to thoroughly understand AI models, it can enhance its oversight abilities by delineating specific information about the AI process that must be provided by your vendor. After all, there have been plenty of examples of AI hallucinations, and the quicker you could understand how your AI information output is generated, the quicker you can resolve any issues.
A constant refrain I hear from credit unions big and small is that their contracts are too long and too restrictive, preventing the credit union from quickly adopting innovative products and cheaper services. As AI is integrated into your credit union, you will once again be confronted with new safety and soundness and operational concerns. Recent amendments made by the government to its baseline procurement contract may not seem like a major regulatory development; but as your credit union ponders the approaches to take to AI integration, these recent changes can provide a blueprint for your credit union.
Trust me, I understand that the typical credit union does not have the same negotiating leverage as does the federal government. But an understanding of the key issues confronting AI integration and how they could impact your operations will help your credit union prepare for the worst and hope for the best.
The insights shared represent Henry Meier’s professional opinion and are intended for educational purposes only. This content does not constitute legal advice or an attorney-client relationship.