Mashgin aims to shorten the cafeteria line with computer vision

You know what I’d really like to do for my lunch break today? I’d like to wait in line. That $12 SoMa salad will just taste so much better if I feel like I’ve worked for it.

I’m kidding, of course. No one likes lines. Especially not Mukul Dhankhar, a Bell Labs alumnus who was so sick of waiting in lines he decided to do something about it.

That something is the Mashgin, a kiosk that can ring up a plate of food in seconds and then take your payment via credit card, no human cashier involved. A few hundred people have tried it so far, and many have the same question: How the heck does this thing work?

The Mashgin kiosk.

The Mashgin kiosk.

In short, it’s computer vision.

“We believe we are the first ones to solve the problem, to see as a human would see,” co-founder Abhinai Srivastava said. The technology is only now coming together. 3D reconstruction and 3D cameras became available very recently. We tried to marry the two and it turns out that is what gave us the accuracy.”

Before it is deployed in each new cafeteria, the Mashgin kiosk is trained on every single food item customers might put in front of it. It can recognize the difference between a Fuji apple and a Honeycrisp. Did you mix 10 different items together into a salad? Not a problem, it can visually sort them out. Just place your tray in the kiosk, remove your hands, and it will automatically ring up your tray.

The Mashgin team: Rajeev Prasad (left) and Abhinai Srivastava (right). Mukul Dhankhar is not pictured.

The Mashgin team: Rajeev Prasad (left) and Abhinai Srivastava (right). Mukul Dhankhar is not pictured.

The Mashgin team will test the kiosk in potentially hundreds of Bay Area cafeterias next year when it partners with Compass Group. Compass Group operates tens of thousands of dining halls worldwide, including Google’s.

Mashgin was 98 percent accurate in a recent test. Since then, the team has been working to smooth out the software kinks to make it even more accurate. They anticipate few errors. And even then, the machine is trained to spot its own struggles and ping a human for help.

The startup will also be rolling out kiosks in small-volume grocery stores. Eventually, it wants to move into large grocery stores, where the system could hover over a conveyor belt and ring up items as they pass underneath. It also has plans to expand into general retail, and maybe even beyond sales into other fields like robotics.

But for now, they are taking things one scan at a time. And that leads us to the next bottleneck: bagging.