Can your phone count calories? Boston company takes AI to food photos | Crain's Boston

Can your phone count calories? Boston company takes AI to food photos

The creators of the Boston-based nutrition app “Lose It!” say the world’s most powerful dieting tool is already in your pocket: your smartphone. Since 2008 the company, which is incorporated as FitNow Inc., has emerged as one of the leading weight loss apps by making it easy for smartphone users to keep track of what they’re eating.

Now they’re expanding on that premise with a photo-based feature that uses artificial intelligence to take some of the tedium out of journaling your every meal. Think of it as Facebook’s facial recognition technology—which seems to "know" who your friends are in photos posted to the site—only for food.

Millions of users already use the app to count calories, searching for everything they eat in the company’s database of dishes. Hundreds of thousands of them also pay for the premium version of Lose It!, which includes additional features. So while the photo-assisted feature, which the company has dubbed Snap It, might seem like an incremental step, CEO Charles Teague says it’s actually tackling the biggest challenge they face in getting and keeping customers.

“It can feel tedious,”  said Teague. “The challenge has always been that there’s a set of people who find it frustrating or too time-consuming to log their food, and so they’re not successful with it. What we really want to do is make it as easy as Fitbit has made activity tracking.”

The company’s founders knew there was a growing of body evidence that showed calorie counting helped people lose weight, but saw many people got frustrated maintaining a paper journal or a manual spreadsheet. Smartphones seemed like a natural fit, and they jumped into the market at the right time—the company’s user base surged after Apple featured Lose It! in their “there’s an app for that” commercial campaign. Since then Lose It! has raised $7 million and claims to have helped its users lose a combined 50 million pounds.

In addition to fast-tracking the process of logging the calories in that salmon sashimi, the new smartphone features include contextual data gathered from the user’s GPS. The new app is still in beta testing, but Teague said it may soon be able to tell if you’re in a Panera, for example, and that could help it identify how many calories are in the meal you’ve just photographed. The company also lets users scan barcodes to get better nutrition information about packaged foods.

With so many smartphone-attached young people already photographing their food for social media, there's a lot of data to be mined. But Teague said his company isn’t sharing any data its customers don’t want it to.

The company employs 20 people in the Boston area. Engineers at Lose It! first started playing around with the Snap It idea as a side project a few years ago, Teague said. Interested in machine learning, they built a prototype of what would become Snap It in a web browser. From there they helped it "learn" to identify 250 types of food, from fried chicken to raw oysters.

There are still kinks in the neural network. It’s no substitute for a human when it comes to distinguishing, say, borscht from tomato soup. But that will improve over time, Teague said.

Lose It! isn't the first company to explore food recognition technology. Google has researched it, as has the National Institutes of Health, and there's even a plate outfitted with cameras "that instantly analyzes everything you eat. But none of them has produced a consumer-facing product. In 2011 a company called Meal Snap piloted a very similar idea but pivoted into a kind of crowd-sourced photo album for restaurant menus.

Image recognition technology has gone from science fiction to big business—worth nearly $30 billion, according to market research firm MarketsandMarkets—in just a few years. One of the biggest players in that market is Facebook, already famous for its photo recognition technology. At Facebook's annual developer conference in April, CEO Mark Zuckerberg told a crowd that the same intellectual property at the base of that feature could also power "an [artificial intelligence] system that can now tell from a photo whether a lesion on your skin is skin cancer with the same accuracy as the best doctor in the world." 

Beyond facial imaging and calorie counting, there is a wide range of applications for photo recognition technology.

"Whether you're developing social tools or you're diagnosing diseases," Zuckerberg said in April, "you're using a lot of the same AI [artificial intelligence] technology.


September 29, 2016 - 11:29am