9 Questions FIs Should Ask a Machine Learning Vendor

1. Is the platform future-proof?

The AI space moves fast and evolves rapidly. Unfortunately, this means some technologies in use today will become obsolete tomorrow. Think about how banking habits rapidly shifted to digital channels during the pandemic. Any system you invest in must be flexible enough to respond to changing market activities and new customer banking habits. 

Make sure to ask the machine learning vendor if the solution has built-in limits. If the solution can’t easily adjust to meet new conditions, it could become a blocker for your business’ scaling priorities. 

2. How much control do you have?

Any machine learning solution you implement should be able to support multiple models and custom models. Supporting multiple models is important because fraudsters behave differently depending on where they are and what they’re doing. A global model, for example, won’t reflect the nuances of fraud patterns unique to a specific region like North America or Latin America. Meanwhile, custom models consider the unique fraud patterns that the banking and payments industries regularly face. Ask the vendor if their product supports both custom and multiple models.

3. Do you understand how the system arrived at its risk score?

You’ll need to trust the machine learning system you select. But how can you trust the system if you or your team of fraud analysts don’t understand how it reached its outputs? This will be an issue if the machine learning system relies on black box solutions that do not reveal how a decision was reached. Make sure to invest in solutions that ultimately keep humans in control and allow them to review transactions to make informed decisions. Only choose a vendor who offers black box solutions if you’re willing to fully trust the system’s outputs without understanding its decision-making process.

4. Does the system enable omnichannel data?

Machine learning is not a set-it-and-forget-it solution. The technology needs to evolve with the latest trends in order to deliver the most reliable outputs. Ask the machine learning vendor if the solution can quickly intake data from different customer touchpoints – mobile, online, ATMs, etc. With these data points, your bank can get a 360-degree view of customer behaviors, empowering your team to adjust the model accordingly to improve its performance. Remember, a good model can also pull data from both external as well as internal sources.

5. Is it scalable?

Imagine your business successfully scales rapidly and you process twice as many transactions as you once did. Your machine learning model should be able to keep up with its new workload with only minor adjustments. If the system can’t keep up with its new demands, it could become a blocker for your organization’s future growth. Make sure to ask the machine learning vendor how the system responds to new developments. They should list different options on how the system can be adjusted. Meanwhile, a vendor who claims their system is “the only one you’ll ever need” should raise a red flag.

6. Is the machine learning vendor’s expertise part of the deal?

A machine learning system is arguably one of the most complicated purchases your bank will ever make. That’s why it’s important not to think of your dealings with a machine learning vendor as a one-and-done transaction. You’ll need access to the vendor’s expertise in machine learning and data science to make the most of your investment. Ask the vendor if access to their expertise is part of the purchase agreement. Look for a vendor who is offering a partnership, not just a sale.

7. How specific does it get?

Machine learning platforms create profiles of normal behavior and then flag instances of unusual or abnormal behavior. But some machine learning platforms can zoom into greater detail than others. For example, instead of looking at “women between the ages of 30 and 35” it looks at “Susan.” The more specific a platform is in building a profile, the more fraud it can prevent. Ask how the vendor’s platform builds customer profiles to establish baseline “normal” behavior. Make sure the vendor doesn’t use data too broadly. If they do, your organization could lose more money to fraud.

8. Does it offer flexible case management?

If your bank has operations around the globe and operates on a 24/7 basis, you could have several hundred people logging onto the system at different times. Look for machine learning systems that offer flexible case management tools that enable your teams to create self-configurable workflows for their priorities. You’ll want case management tools that automatically distribute work based on different criteria, prioritize transactions by importance, and offer various team members (fraud analysts, branch employees, or customer service personnel) exactly what they need from the system. 

Ask the vendor how the system distributes work across different departments and how team members can use it to prioritize transactions. Consider different teams’ perspectives and how the platform will work for them as they perform their responsibilities. Be wary if the vendor can’t address how the system works for different departments.

9. Is it self-configurable?

Let’s say you want to measure a specific department’s, product’s, or team member’s churn rate. Normally, you’d have to ask the machine learning vendor to adjust your dashboard’s reportings to pull off these goals. A platform with self-configurable dashboards, however, lets you customize specific metrics to focus on your preferred business KPIs.

In a world of gas-powered cars, machine learning systems are the new electric cars. A platform that’s self-configurable goes one step further. It means you’re not just buying the car; you’re buying the car factory. Ask the vendor how long it takes to adjust the dashboard to your liking. A red flag should go off if they can’t guarantee you’ll be able to configure it to your organization’s specific needs at your convenience.

Investing in machine learning can be an intimidating step. But you don’t have to do it alone. Use this guide to make your machine learning selection process as smooth and informed as possible.

How can you be certain you’re working with the right machine learning vendor for your business? Download or eBook, How to Choose a Machine Learning Platform to Detect and Prevent Financial Crime, to learn the red flags to watch for and more.