How (And Why) to Vet Vendor Software for AI Use and Security Risks

Practically overnight, artificial intelligence made its way into organizations in every industry. Third-party software is integrating AI engines and chatbots into their basic subscription packages, whether you ask for it or not. Artificial intelligence is getting packaged in office products and search engines, and your employees are using them without considering the potential security implications of AI.

Artificial intelligence has permeated the workplace, and it’s here to stay. That’s an issue, because AI presents unique security challenges that most organizations aren’t fully prepared to address. Depending on your organization’s security stance, you may need to consider how to lock down software so that personnel can’t inadvertently share inappropriate data with third parties.

To protect your company, you need a framework of three essential controls: AI Policy, Approved Software Lists, and Vendor Vetting. I’ve already written about AI policies and approved software lists, so I’ll spend most of this article on the final piece: how to vet your third-party software vendors on their use of AI.

Related: Your Company’s AI Adoption Is a Security Nightmare

Implement AI Policy Boundaries Today

Because AI is ubiquitous, every organization needs to have something in its policies governing the use of artificial intelligence. If you don’t define standards, your people are going to “paint outside the lines” — and that will inevitably introduce security risks you aren’t prepared for.

Before you can vet a vendor, your internal policy must clearly define acceptable uses of AI and what constitutes “sensitive data” for your organization. This is a critical first step. You need a well-communicated standard to ensure that proprietary or protected data don’t end up in an AI’s systems.

Defining Your AI Vetting Strategy

When employees request to use a particular software, you need to validate whether it’s the right tool for the right purpose (at the right price point). But you must also decide on your security stance for that specific tool. Some areas to consider:

Public vs. Private Instances

Will you allow the use of a publicly available AI platform, or do you require a private instance? Some providers offer private instances, but “private instance” can mean many things in the AI world. Don’t assume anything. 

Your organization needs to walk in with eyes wide open, whether you use a private instance, a public instance, or open-source AI capabilities. 

Risk Mitigation

If you aren’t leveraging an AI platform for sensitive or internal data, it greatly mitigates the level of risk you’ll take on. But “sensitive data” must be well defined, and you’ll need a very clear and well-communicated policy in an attempt to ensure that no sensitive or internal data ends up in an AI’s systems.

However, if your organization uses AI-enabled software in association with Sensitive or Internal Use data, you’ll need to take extra care to properly and thoroughly vet every piece of software that your company uses.

Your Company Is at Risk with AI Software

I often call AI adoption the “AI zombie walk” because many organizations throw all caution to the wind for the wonderment of what AI promises — with little to no consideration of the risks. As your personnel use AI to do their daily work, they may be unknowingly sharing sensitive data with insecure AI engines. Your standard vendor vetting reviews should now include a thorough awareness of how AI is being implemented in their tools.

Inevitably, there is an element of trust involved in vendor relationships, but that trust must be verified. I still remember Edward Snowden blowing the whistle on big tech companies that violated their own privacy statements by allowing back doors and sharing information with unapproved parties. It’s not much of a leap to imagine AI organizations inappropriately or even surreptitiously using your data to feed back into their engines for continuous improvement.

One specific risk to be aware of is the “trust but verify” approach to coding. Bad actors are actively attempting to influence learning models to inject backdoor vulnerabilities into generated code. They want security holes to appear as an expected outcome in the program so that unaware developers will plug them directly into their environment. Never take code from a machine and throw it straight into production, just as you wouldn’t with any other unknown developer.

AI Vetting Questions to Ask Every Vendor

In early rounds of vetting, eliminate the vendors that aren’t suitable for your purposes. Once you have a shortlist, go into progressively deeper levels of questioning based on your company’s specific risk appetite.

Ask vendors about their security and compliance posture. Sounds nuts to need to put this in writing, but actually read their security documentation and confirm the services and locations you leverage are part of the scope.

What specific controls have they implemented and have within scope? Has the vendor recently added AI capabilities? How are they protecting the environment and your data? Specifically, understand how they handle their AI modeling:

• Are we entering data into a public pool?
• Can we garner a private or semi-private instance of the AI engine?
• Do we have the capability to host it ourselves so we know precisely where the data is?
• If we use a private instance, does that mean nothing touches our data or metadata?
• Is the private model being influenced by learnings from elsewhere or a baseline engine improvement feed?
• Can I configure my instance of the vendor software to exclude involvement with AI?

Related: How to Audit Your Vendors for Security and Compliance

Major Red Flags and Dealbreakers

Do not simply assume you’ll get a transparent response from massive AI companies—demand evidence that verifies their claims. In my experience, the following are immediate dealbreakers:

1. A Risk Appetite Mismatch: The vendor does not possess a model that works in accordance with your risk profile (e.g., they only offer a public interface for sensitive data).
2. The Transparency Gap: The vendor is unable to provide acceptable answers about how they use customer information or who they share it with.
3. Vague Responses: If a vendor can’t answer your technical questions in detail to your satisfaction, it’s a sign they either don’t know or they’re obscuring their real data practices.

Take seriously the responsibility to be a good steward of your organization’s security. Your customers, employees, and partners are counting on you to walk into the AI era with your eyes wide open.

Take Your AI Responsibilities Seriously

The convenience of AI is seductive, but the security risks can be permanent and catastrophic. Once your sensitive data is ingested into a public model, there is no Undo button. To stay ahead of the curve, you must be proactive and skeptical of any software your company considers using. 

Establish (and enforce) your AI policy, lock down your approved software list, and vet every vendor like your company’s reputation depends on it — because it does. It’s time to snap out of the AI zombie walk and start leading your organization into a secure future with a clear, risk-appropriate AI strategy.