Building Deeper Customer Profiles in Banking with Computer Vision- with Kelly Harlin of Sharp NEC

With so much competition in the market, it is easy for customers to switch their banking partners. Data and analysis of customers’ financial data help banks reduce churn by channeling insights behind personalized solutions and offerings for their customers. Yet the demand for more personalization in financial services hardly stops there.

To offer more profound levels of personalization, banks need a better idea of customers’ lifestyles. Lifestyle data recognizes how people live and can help banks determine the critical influences behind what they purchase, as much as their age and income. Moreover, lifestyle data enables financial institutions to consider customers’ interests, activities, and the effect they have on their buying behavior. 

For instance, residents in college towns frequently eat out at fast-food restaurants and pizza outlets during the week. When many college students are new town residents, then bedding, bath, and cooking products usually become popular purchases. Lifestyle metrics then show distinct growth trends in local music and nightlife venues during college semesters.

Emerj CEO and Head of Research Daniel Faggella recently spoke with Kelly Harlin, Director of Solutions Marketing and Commercialization at Sharp NEC, on the AI in Business podcast. Sharp NEC specializes in information displays and camera systems for gathering data from physical environments. These capabilities place them at the forefront of bank demand for more information from their physical branches. 

In the following analysis of their conversation, we examine two key insights that Kelly brings out from the banking sector:

• Capturing customer lifestyle data: Tracking banking data about consumer lifestyles and purchases with computer vision to personalize customer offerings.
• Delivering digital-first experiences in bank branches: Leveraging data to understand why customers are coming to banks after the pandemic to maximize the value of physical branches in a digital-first age. 

Listen to the full episode below:

Guest: Kelly Harlin, Director of Solutions Marketing and Commercialization at Sharp NEC

Expertise: Insights and analytics, product marketing, integrated marketing

Brief Recognition: Kelly has been with Sharp NEC for four years now. Before which, she held senior marketing roles at Reality Interactive and Microsoft. 

Capturing Customer Lifestyle Data

Today, banks have a lot of data about their customers’ financials but want to know more about their lifestyles. They need more data in terms of the following:

• What cars do they drive?
• What clothes do they wear?
• Where do they live
• What do they need when they walk into the branch?
• How long did they wait in line at the ATM or the drive-thru?

But even once they gather the data, how does it help the bank? Kelly expresses to Emerj how banks can ascertain aspects of customers’ lifestyles relevant to financial decision-making as the banking sector gets data-rich with customers’ financial data: 

“Understanding the lifestyle and potential financial status of the customers around the location of banks or credit unions can help banks serve the customers better. If the data gathered indicated that there are a lot of luxury cars going through the cameras, it means that the banks should not hype or promote luxury car loans in that neighborhood.”

– Director of Solutions Marketing and Commercialization at Sharp NEC, Kelly Harlin

She emphasizes that knowing your customers beyond their bank balance can tell us if and what they are aspirational for and their affinity for different brands. 

That will mean banks are leveraging the entire property of their physical branches as data collection centers. For a retail bank with a large parking lot, an ATM, and a physical branch in the same space, the bank needs to look at two main elements: the person themselves and their car. The data the banks are trying to collect is:

• Customers’ lifestyle data by looking at what clothes and logos they are wearing
• What cars do they drive- are they luxury cars or SUVs?

Small detection objects and cameras inside and outside the bank branches can gather data that can help banks shape and tailor their products and offerings. Computer vision, a field of AI, helps banks in this scenario. 

Kelly assures that this can be done without invading customers’ privacy. For instance, the Canadian government allows the private sector to record their customers if they follow the policy framework. The guidelines demand businesses establish a reason for conducting video surveillance and use video surveillance only for that reason. It also asks them to develop a policy on the use of video surveillance.

What sets the guidelines aside is how the government asks businesses to prepare. Namely, in requiring businesses to answer questions from the public. Individuals have the right to know who is watching them and why, what information is being captured, and what is being done with recorded images.

Kelly tells Emerj that hardware setups can be flexible in these situations, meaning banks can use their existing security cameras for the same results – the key is how the information is gathered and analyzed. The basic framework of the hardware should include the following:

• Visual data is collected through the branch cameras, and additional cameras can be leveraged using Raspberry Pis. 
• The video feeds come into the  Intel Smart Display Module (SDM), which enables a cable-free installation and display of multiple camera feeds to a single dashboard or data-gathering source. 
• Then the data and related aggregations through APIs are sent to the database in a data lake for the bank to gather data into their own dashboard.  

Deciding what type of hardware needs to be placed at what locations depends on the use case. If the bank just wants to capture traffic from outside the branch, then the existing security cameras fulfill the purpose. Still, for a use case where the bank needs to detect the logo, Kelly recommends using Logitech cameras. 

Other than the security cameras, a bank would need analytics software to capture the data about customer attributes like people count, logo detection, wait times, vehicle classification, clothing detection and object color detection.

Delivering Digital-First Experiences in Bank Branches: 

Following the pandemic, two-thirds of adults worldwide make or receive a digital payment, so how do the banks attract people back into the branches, and what services do they need to offer? 

Kelly mentions how Credit Unions have adjusted to the change, aspiring to attract their customers back into the physical locations. Hence they want to know as much about the customer as to offer them value when they visit the branch.

People are used to a balance of functions between their FI digital experiences and their brick-and-mortar experiences. Before the pandemic, physical bank branches had very little in the way of offering — or even complementing — available digital experiences.

Now banks are challenged to offer brick-and-mortar experiences that complement the digital-first approach most customers are taking to their financial interfacing. 

Kelly describes one data-driven approach for solving the above challenge during her appearance on the AI in Business podcast. In gathering and processing enough data beforehand, banks can predict why customers are going to their physical branches in the first place — whether it be a problem with their cards or a loan for a vehicle, home, or business.

Part of this involves looking at apparent challenges, like waiting lines, as opportunities — or rather, means for acquiring customer lifestyle information. Kelly tells Emerj that when customers visit the branch to see the loan officers, waiting in line can be seen as incredibly unproductive – for both the branch and the customer. To solve this, Kelly shares an interesting use case:

“One of the lenders had put an interactive screen in the bank branch. So now, instead of just sitting and waiting, the customers would interact with the content on the screen, and while they do, the bank can gather the other essential data like the brand they are wearing.”

– Director of Solutions Marketing and Commercialization at Sharp NEC, Kelly Harlin

Kelly further mentions how banks can use these information-gathering displays creatively. While a customer waits to see the loan officer, they can get information on their account from the same interactive screen. 

In another example, someone looking for an auto loan would have some questions which could be answered via the digital screen. Even a loan for a recreational vehicle (Kelly cites a customer taking out a loan for a boat as an example) can be completed over a digital screen, with no essential need for a human agent in the customer interfacing process.

Finally, both the customers’ personal data that was submitted discreetly via a waiting line interface screen and the lifestyle data gleaned from the physical branch waiting area by camera systems can be transferred to a management dashboard for decision-making based on deeper trends therein.