Retailers, particularly in the apparel, home goods and home improvement categories, are in the early days of using the technology. For example, the giant appliance chain Home Depot just added visual search to its mobile app by adding a camera icon to the search bar. When a customer takes a photo of a part they may need, machine learning and computer vision technology scan every product in their 1-million product catalog to render the most accurate results matching the image and tell you where in the store it is, or send you to the online page it can be found.
Amazon, Target, Bed Bath & Beyond, West Elm, Wayfair, Neiman Marcus, Macy's, Asos, H&M and Forever 21 have visual search capabilities built into either their e-commerce sites or mobile apps to answer the search queries that can't easily be asked in a text field, like "What kind of chandelier am I looking at?"

Some companies, like Wayfair and Farfetch, have built visual search technology in-house. At Farfetch, a 15-person computer vision team is working on powering the capability, not just to render accurate search results but to build product recommendations off of the context of an image, to act as a styling guide around what jacket matches which dress. It also works with the platform Syte to boost the technology.

"There's a huge intellectual challenge with computer vision, which is finding meaning. But that's where we unlock a huge level of consumer engagement. So we need to nail it. We can't just be a luxury search engine. We need to be able to break down the context of an Instagram, or any inspirational image, and say if you want to achieve this style, here's what to buy." Farfetch CMO John Veichmanis.
Customers who use visual search on the Farfetch app spend more dwell time in the app than other customers, and that since launching the tool three months ago, the company will continue to improve upon the results to get personalized recommendations down to the individual level. That takes a lot of data, and metadata.
"It's a heavy technical cost to launch and upkeep this technology.. We're telling retailers to start now, whether you build it yourself or partner with someone. Because this will shake out the way text search did: You'll have your big scale platforms where searches start, but everyone will need their own version of it, too." Vic Drabicky, founder of the data agency January Digital

The race is also on among platforms.
Pinterest wants to be the first platform to crack visual search at scale. Several iterations of shoppable pins have passed through Pinterest's feed, but the most recent version uses computer vision to pull up a feed of "products like this" below the pin. These products are all in stock online. Since launching that feed last year, traffic that Pinterest sends to retailers increased by 40 percent.

"We feel like this is what we are uniquely able to do, be a bridge between products in stock and images. More and more of Pinterest will have shopping mode. We'll never force it. But when you have the desire to shop, we want it to be so easy to start shopping." Tim Weingarten, head of shopping products at Pinterest.

Right now, products can appear in Pinterest visual searches one of three ways: Retailers hand Pinterest the keys to their e-commerce catalogs so Pinterest can build product pins, tag them, and track details like availability and price accordingly; retailers upload their own lifestyle imagery and then tag their own products, so if a customer comes across the pin on the site they can shop the exact items in the photo; or brands and retailers can buy an ad in the shoppable product feed to be featured in results.

Pinterest wouldn't disclose what it costs to be featured in product search results, but as of now, it's the only platform that's begun to monetize the tool, according to eMarketer. Google's similar Style Match tool, and Amazon's Lens results, don't have sponsored product slots, yet.

As growth rates climb, and as marketers look for alternatives to the duopoly, Pinterest could finally have its moment…the platform has long been considered as hidden gem, but it hasn't been able to break through and become a have-to-do-it type of platform. Visual search has the opportunity to fill that spot, and especially as Facebook's use continues to decrease, consumers will be looking for that. Google will be great at visual search as well, but its approach has covered a lot more ground than retail. Pinterest has a crack in the door to be a mainstream partner.

As platforms make headway in bringing visual search to scale, retailers will have to be ready on the back-end. In general, visual search will push e-commerce sites to better catalog their imagery and improve checkout pages so that people can easily purchase when they land on a product, especially on mobile. Wurmser predicts that in the next two years, many more visual search ad products will surface.

There's more motivation than the platform partnership. Wayfair's head of product, Matt Zisow, sees value in owning the end-to-end digital experience as it navigates visual search, computer vision and machine learning technology. Wayfair has a catalog of 10 million products, so visual search plays a role in everything from augmented reality interior design tools, to product searches, to replacements for out-of-stock items, to personal recommendations. Overall, making sure that every single product image on the Wayfair said is coded with a computer vision algorithm has lifted the user experience of the site, by increasing the relevance of results.

Veichmanis said that visual search will have long-term effects on how Farfetch predicts trends and inventory buys because it will compile visual data around customer interests earlier on.

The only thing left is to get more people to use it!