Fruits of machine vision

Visual search is often touted as being a way to get consumers to research products online while in a store. Moral issues aside, this has never appealed much to me as a model as I reckon that the immediate gratification outweighs any price benefit. However I've now seen a different use of visual product recognition that was stunningly good.

While buying holiday supplies in Carrefour TNL in Nice, I was rather surprised to find that there were no queues at the fuit and veg scales. For those unfamiliar with the process, you have to weigh the produce yourself before heading for the checkouts. Normally this involves a lengthly hunt through the user interface for whatever you were buying, leading to lines building up.

First thing I weighted were green beans - boom, no sooner had I put the bag of beans on the scales than the display offered me green beans as first choice! Next was a bag of flat peaches - again, that was the first choice offered.

Curiosity raised, I looked underneath the screen, and sure enough there is a camera focussed on the scale pan. Clearly there is some very nice software behind the camera as recognising flat peaches through a translucent plastic bag is impressive.

But my next purchase foxed it. The system was unable to identify six Royal Gala apples. Not surprising since they could have been one of several red apple variants all about the same EU-regulated size, but shows there is some way to go. I was taken back to the default menu to hunt through several screens of different types of fruit.

Despite that limitation, I am still deeply impressed by the practicality of the system and the smartness of recognising odd-shaped items through a translucent material. From a retail perspective it is a big step forward towards the holy grail of increasing throughput.