The retail shelf is ‘ground zero’ for retailers and brand manufacturers, the last point in the retail supply chain. And, if product isn’t on the shelf, well, it can’t be sold. Stockouts are estimated to cost brands and retailers upwards of $1T per year. The root cause? Lack of visibility into the actual condition of store shelves – i.e., you can’t re-stock what you can’t see.
It’s not for lack of trying. Retailers have experimented with a variety of technologies, from sumo-wrestler-size ground robots roaming store aisles to tiny cameras affixed to store ceilings. Amazon and others are instrumenting the shelves themselves with sensors in an attempt to be smarter about what they contain.
Meanwhile, brand manufacturers send armies of people into the stores to spot-check shelves by hand. Each approach, whether automated or manual, comes with an assortment of issues related to accuracy, scalability and complexity. They also are capital intensive, and the resulting data is too incomplete and stale to be actionable.
The Pensa Approach
Pensa’s system is different. It is uniquely effective at understanding real product and shelf conditions, collecting data faster, more frequently, with greater accuracy (98+%) and at a lower cost per location than any other approach, whether manual or automated. (You can read our official launch announcements, here and here.)
So, how have we been able to finally solve a problem that has vexed brands and retailers for decades?
Pensa breaks through the complexity and overcomes the limitations of prior attempts by marrying a small low-cost drone with powerful, patent-pending AI in the cloud to process the input and learn just like a person would. We call this combination an “autonomous perception system,” and it’s the core of how we are able to achieve breakthrough results.
Agile, efficient, highly-accurate data capture. The Pensa drone, equipped with a small camera, navigates the store autonomously and “sees” the products on the shelves. Agile and capable of seeing from different heights and perspectives as it moves, it is as if 100 cameras were fixed on every product on every individual shelf, everywhere in the store.
The drone is just the tip of the iceberg, a nimble and cost-effective method of collecting shelf inventory data for processing in the cloud. I’m talking about orders of magnitude in terms of cost improvements, with Pensa’s per-store location costs averaging one-hundredth or less that of other automation approaches.
Truly seeing real-time shelf conditions. The real magic is what happens next. It turns out to be hard, even with the latest and greatest ‘neural network’ research from academia, to train computer vision systems. Neural network state-of-the-art is more often trained to recognize “a chair” or “a table” or “a pedestrian,” not “a 12-oz can of Campbell’s Chicken Noodle Soup” down to the level of individual SKUs. Pensa’s computer vision, however, is able to precisely interpret this kind of detail — a key requirement, given that an average retail store today can easily have 100,000 products, with a huge variety of different SKUs, on the shelves at any given moment in time.
AI that learns, analyzes and concludes. Pensa’s patent-pending AI takes the next step in understanding what is actually happening on the shelf, learning by observing rather than having to reference a planogram.
First, the system first figures out how to integrate the best view of the products on the shelf from various angles. Our machine learning cloud then uses our own patent-pending advances in deep learning to process the vision input. As products are seen on the shelf, our system recognizes what it recognizes and trains automatically on what it doesn’t. The next time those same products are seen, whether there or in an entirely different store, Pensa will recognize them. The technology is able to do this for the broad range of consumer-packaged-goods formats, so it can distinguish, for example, the Campbell’s Chicken Noodle from the Tomato soup.
Then, another key Pensa breakthrough is applied, namely, the system’s ability to learn from repeatedly looking at the shelves and drawing conclusions. We call this an “observed planogram,” as it captures how the shelves are actually being managed. From this, the system can sense stockouts even where there are no gaps on the shelf – for example, if a store clerk fills an empty shelf with different products or a customer puts an item back in the wrong place. These are common occurrences and they confound many automation systems and human spotters alike. The Pensa system handles these situations, automatically – learning from artificial intelligence algorithms tuned to understand the intricacies of the retail shelf. For people, these tasks are tedious and error-prone, but they are perfect for advanced artificial intelligence and automation with the proper techniques.
Compare this with other attempts at automation, which look for “voids” or holes in the shelf that might signal something missing. They then, rather than understanding the shelf, must receive exact store maps (called planograms) to determine what is supposed to be on the shelf in that particular spot. Unfortunately, such planograms themselves are often inaccurate or out-of-date.
Similarly, on the autonomy and robotics side, we’ve borrowed the best from autonomous vehicle research results. Here also, we’ve tailored our approach to tame some of the more “bleeding edge” problems faced by self-driving cars that need to decide in real-time when to swerve to avoid an accident. Our autonomous drone is comparatively slow, quiet and methodical – essentially a very agile hovering camera which can move into an unoccupied area to scan the shelves then fade away to check other aisles or categories as necessary. (Recognizing the type of soup can is important but unlike an autonomous car, it doesn’t need to happen within a fraction of second to save a life.) We’ve also spent considerable time working with retailers and consumers to ensure the system is safe and comfortable for both workers and shoppers.
Pensa is fortunate to have some of the world’s experts on our team and on our Board. We’ve been able to adapt and scale deep-learning artificial-intelligence algorithms, robotics technology and autonomy – pushed to the limits of research but combined with large scale software engineering and product delivery needed for the highly demanding commercial retail environment. The end result is a groundbreaking new service that delivers actionable, real-time data to consumer brand manufacturers and retailers alike.
