Irina Kolesnikova
December 19th, 2022
The growing interest in computer vision among retail businesses shouldn’t be surprising since many retail operations generate large amounts of data and require visual feedback. However, Retail Info Systems experts claim that, although only three percent of retail companies have implemented computer vision, forty percent plan to do it within two years.
Turning to computer vision services, retailers can resolve many pain points and, furthermore, transform employee and customer experiences. For example, store layout improvements based on real-life visual data and heat maps work much better than ones based on intuition.
Consumers today expect the same personalization and convenience from brick-and-mortar stores as they encounter online. That is also the reason why the popularity of virtual reality and computer vision in retail industry is growing. Virtual mirrors in fitting rooms provide offline retail stores with a previously unseen level of personalization. Furthermore, self-checkout with computer vision cameras becomes smooth, while automating other routine tasks saves more time for customer-oriented activities.
With machine learning consulting and an integrative approach to computer vision implementation, a digital transformation of retail companies becomes much more attainable. Let’s take a closer look at computer vision technology and its top seven applications in retail sector.
Computer vision is an umbrella term that refers to the branch of machine learning models aimed at extracting information from visual media. In other words, computer vision systems combine cameras, software, and artificial intelligence to empower machines to “see” and identify objects.
They utilize deep learning to train systems to analyze images. Once fully trained, computer vision models can recognize objects, detect and recognize people and even track their movements.
Self-checkout has already solidified its importance for brick-and-mortar stores, paving the way for a more streamlined, AI-assisted shopping experience. However, as automating customer services is becoming a priority, retail businesses have to update it and other processes as well to make them more efficient.
Today, self-checkout in most retail stores means that customers manually scan the barcodes of individual items. In contrast, computer vision-powered cameras identify goods without barcode scanning, thus simultaneously enhancing customer experience and security and speeding up the checkout process overall.
In the last few years, with the proliferation of machine learning in the retail industry and automated visual inspection, software companies have recognized the demand for computer vision-enabled self-checkout systems and now offer a range of variations of this concept. For example, Amazon’s Just Walk Out system combines cameras, sensors, and deep learning. It allows customers to pick up the needed products and walk out of the store without waiting in line to pay. Notably, customers don’t need an Amazon account or an app. Instead, computer vision systems use cameras tracking objects and a person’s movements, and shelf sensors identify goods removal or return. Then, as the customer leaves the store, their card is automatically charged for items they picked up.
Another Amazon project, smart shopping carts called Dash Carts, was launched in 2020. These carts, embedded with cameras, sensors, and a smart display, automatically track items as shoppers add them and charge the linked Amazon account when they exit the store, allowing them to skip traditional checkout lines. The carts, designed for small- to mid-sized grocery trips, build on Amazon’s “Just Walk Out” technology from its cashierless Go stores. Shoppers use their Amazon account and smartphone to activate the cart, which tracks and displays the total cost of their order as they shop.
It’s common knowledge that online reviews matter for building trust. We and, most probably, you, are among the 89% of consumers who check product reviews in detail. Computer vision technology allows retailers to enjoy the benefits of online reviews and ratings at their physical stores.
Guitar Center, one of the world’s largest musical equipment stores, applies its online store’s well-established features in physical outlets. The company utilizes an application allowing access to product information and reviews by enabling customers to scan an item’s barcode with the smartphone’s camera. Customers can then use the mobile app to look up reviews, ratings, similar products, and alternative colors of scanned products.
Computer vision also found its way into retail inventory management. According to Retail Technology Study 2020, 64% of retailers are looking to deploy various data-driven solutions, computer vision included, to optimize inventory in the next two years. The retail industry can utilize computer vision to update its inventory management system in real-time and develop an omnichannel retail experience, integrating different methods of interaction available.
Computer vision cameras can be mounted on top of standard retail equipment to notify staff about gaps on shelves or misplaced products. This provides more time for shopping floor staff to focus on customer care. At the same time, retail store analytics can also use these real-time computer vision data from cameras to dynamically react to product movement, replanning the position of goods on the shopping floor to match consumers’ propensity to purchase.
For example, Tally, the mobile robot for inventory management and the all-in-one inspection system designed by Simbe Robotics, captures visual data from more than 12 high-resolution cameras. Besides notifying staff about out-of-stock products, Tally can detect damaged packaging and incorrect pricing or even accompany customers to the right products.
A retail heat map technology is a shaded matrix description where values in the matrix are pictured as different colors. Retail heat maps apply real-time movement detection and allocate colors related to traffic volume to each area. Retailers can use heat maps to study customer movements, test new merchandising strategies, and experiment with layouts.
By installing computer vision cameras, retailers can identify hot areas of the store, customers’ movement and purchase patterns, and behavior concerning certain products. By analyzing this information, retailers can make informed decisions about merchandising, store layout, as well as staff positioning.
For example, Legend World Wide, a high-end Serbian fashion retailer, built a ‘connected store’ in collaboration with Deloitte. The company installed computer vision sensors and cameras in the physical store to track customer journeys and gain better product-related insights overall.
Customer movement heat maps revealed a flaw in the store’s layout. Right after entering the shop, most men quickly skimmed through the clothes on the left. After immediately realizing they were in the women’s clothes department, they turned around and left the store. The situation improved with clear navigation signs pointing to the men’s section upstairs. An experienced retail analyst would likely notice this problem; however, computer vision technology allows arriving at such conclusions with greater certainty and much faster.
Computer vision and augmented reality (AR) implementation is not just a technology but a game-changer in the shopping experience.
We all want to try products before buying them online, and that’s now possible thanks to computer vision and augmented reality.
Particularly in sectors like fashion, cosmetics, and eyewear, where fit and aesthetics are crucial, this combination, known as virtual try-on, offers significant benefits for both retailers and consumers, streamlining the decision-making process and enhancing the shopping experience.
For instance, Warby Parker uses AR and computer vision in its app to map a user’s face and overlay eyeglass frames virtually, providing a realistic preview and boosting customer confidence in online purchases. Similarly, L’Oréal employs AR in smart mirrors and apps to let customers virtually apply makeup, increasing engagement and conversion rates.
These technologies let you virtually try on not only clothing, accessories, cosmetics, but interior items using just your smartphone.
With augmented reality, you can see how a table or lamp will look in your room, rotate the item, try different colors, and decide on the best spot before purchasing. IKEA and other brands already offer this service, helping shoppers make confident decisions.
This feature boosts customer satisfaction, reduces hesitation, and increases product engagement.
Smart fitting rooms equipped with computer vision can identify items and suggest complementary products, mimicking a personal stylist’s role. These technologies also help reduce return rates by improving the accuracy of online shopping, addressing a significant challenge in retail.
Virtual mirrors may become the primary driver of personalization and customer experience enhancement due to computer vision in retail. A virtual mirror is a mirror with a display hidden behind the glass. Powered by computer vision algorithms and AR, it shows the consumer what the product would look like on him or her as well as matching clothing items, available sizes, and colors. Moreover, it allows the customer to ask the staff for other items, without the need to leave the fitting room.
Complementing virtual mirrors, AI-powered recommendation engines and customer advisers can enhance consumer satisfaction. Computer vision algorithms can be so advanced that they can suggest you the perfect match for your outfits. Soon they will become fully-operational customer advisors.
For instance, Ralph Lauren has implemented an in-store virtual fitting room solution. The ML-powered engine behind this product provides customers with available sizes and colors of the items and real-time fashion recommendations based on their current looks. So, for example, when a person tries on a skirt in the fitting room, the engine will recognize the image of the skirt and recommend jackets, hats, bags, and other items that will complete the look.
To achieve that, the retailer feeds the artificial intelligence algorithm with the data on each product from the catalog. The system then analyzes each product’s characteristics, including gender targets, color, description, and price, to suggest other suitable items available at the store or online. As a result, the amount of clothes consumers tried on increased by 90%, influencing revenue growth.
Moreover, recommendation algorithms can also significantly impact the eCommerce sector. Products can be suggested in real-time through a mobile app as customers shop or through personalized promotions and coupons sent after their visit. Retailers can send targeted offers based on a customer’s in-store behavior, significantly increasing the chances of a sale. By recognizing each customer’s unique tastes and preferences, retailers create a sense of exclusivity and belonging, which is key to building customer loyalty.
For example, John Varvatos, a US-based luxury menswear brand, turned to computer vision-powered algorithms to upgrade the experience of their virtual store. By implementing the “Complete the Look” algorithm on their products’ pages, John Varvatos reported an 83% increase in conversions, a 74% increase in the average order value, and a 107% increase in time spent on the website.
By providing crowd analysis, computer vision systems may become “eyes” not only for marketers but also for security. As computer vision observes consumer behaviors, ML-based algorithms identify patterns and make decisions, leading to retail loss prevention via computer vision applications. One of the most common applications is to detect suspicious behavior correlated with theft and fraud. For example, by recognizing every item in the checkout area and connecting it with a transaction, it can reduce employee theft at counters by detecting cashiers who don’t scan each product or ring them up at other prices.
These computer vision systems continuously improve themselves, learning from each interaction to enhance their predictive accuracy. This ongoing refinement makes the technology more reliable and reduces the chances of mistakenly accusing innocent customers.
In times of rapid digitization and continuously changing consumer lifestyles, adapting to client needs and behaviors on the fly becomes crucial. Computer vision helps keep your fingers in the pulse of every slight customer behavior fluctuation. Its application even expands to complex use cases resolving technical issues such as poor illumination. You may discover more details by checking out our guide for detecting objects in video.
Despite being one of the most mature AI technologies with multiple use cases, computer vision applications in real life have yet to realize their full potential. Nonetheless, AI technology offers competitive advantages in delivering products and services, so computer vision technology is developing tremendously in retail, manufacturing, and many other sectors.
Unfortunately, far too often, retailers (and other businesses) implement technologies to enhance just an isolated part of store operations. Such a vision is usually driven by niche software vendors, who provide off-the-shelf solutions and are often focused on solving a particular problem. However, Amazon’s Just Walk Out system succeeded primarily because of a holistic vision of transforming the retail industry standards and customer experiences in its integral entirety.
Even though this strategy might be too complicated for many companies in terms of processes or finances, considering long-term transformational strategy is crucial. Taking small steps could be the best way to tackle big problems and pave the way to grand success. In ensuring that your company tries new tools and takes a data-driven approach to business, you foster experimentation and discovery aligned with business goals. With computer vision and AI, companies can follow customer experience and engagement while making knowledgeable decisions about enhancing store operations integratively.
