Microsoft Patent Targets Virtual Nutrition Labels

By Jeremy Hsu | January 31, 2016 9:55 pm

A concept video shows how a Microsoft Hololens wearer could call up a virtual recipe in the kitchen. Credit: Microsoft

A Microsoft patent idea sees the future of eating for smart glasses wearers going something like this. You’re about to chow down on a juicy hamburger when a virtual nutrition label pops up that says: “Typically a hamburger of this size may have 800 – 1200 calories.” Ignoring the information, you pick up the greasy burger and take a large bite. As you chew, another floating warning pops up in your vision: “This is your second big meal today — are you sure you want to do that?”

Having your smart glasses scold you for disregarding your personal diet goals may not sound like a lot of fun. But someone at Microsoft thinks an augmented reality app focused on health and nutrition could translate into big business down the line. A Microsoft patent application titled “Wearable Food Nutrition Feedback System” was recently discovered through research provided by legal technology firm ClientSide. The patent application covers the concept of a wearable head-mounted device—basically smart glasses—that can recognize a wide variety of food and display virtual information such as virtual nutrition labels, food allergy warnings floating in midair or even menu recommendations posted by friends on social media.

It’s always worth keeping in mind that a patent application or grant does not mean Microsoft will actually develop such a nutrition-focused product for augmented reality. But Microsoft has been hard at work on unveiling an augmented reality device: the Hololens wearable that can display interactive holograms overlaid in the real world. A food nutrition app seems like a no-brainer for the Hololens or a next-generation version of the smart glasses.

The Microsoft patent application includes an example of the smart glasses wearer looking at a veal special dish on a restaurant menu. In one case, the wearable device might display an “avoid” allergy warning about the veal dish containing nuts. That assumes the wearable system already has information about the human wearer having specific food allergies.

In a second example, the patent application envisions the wearable device telling the wearer: “You didn’t like the Veal Special last time…” That case assumes the system can store data about specific restaurant dishes a person had tried before and recorded their reactions or ratings for each dish.

A third example involves the wearable device displaying the following: “Marsha was here last week and had the Veal Special and gave it four stars!” In that case the wearable device would presumably pull the friend’s food rating and recommendation from something like an online social media service.

The biggest technical issue for the nutrition label app may come from creating the software capable of recognizing different types of food and deducing the appropriate nutritional information. A piece of fruit such as an apple or a packaged product may not present the greatest challenge for visual recognition algorithms, but a casserole might prove otherwise.

Still, Microsoft seems to have anticipated that problem. Its patent application envisions the wearable device asking for the human wearer’s help in labeling foods that it can’t identify. Many computer algorithms and programs have improved their capability to identify certain images based on that time-honored tactic of learning from human feedback and input. In that sense, it’s likely just a matter of time before a virtual nutrition label app similar to what Microsoft envisions becomes a reality.

This particular Microsoft patent application, published on Jan. 28, 2016, was first filed on Oct. 8, 2015. It builds on a previous patent application that was originally filed in 2012 and approved in 2015.

ADVERTISEMENT
  • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

    The Nanny State is about 100% surveillance, 100% control, 100% confiscation. If you want Michelle Obama’s fist squeezing your nutritional privacy, have at it. When I go to Carl’s Jr., I arrive to repeatedly stuff my face with gustatory ecstasy. As for the Obamalette, Youtube v=S2EKzoh6AAo

    The unabled maximize self-esteem to be credible – effectively selecting for the deluded, hence government.

    • Michelle Bailey

      Your going to show her “Uncle Al” – see you in the cardiac care unit.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Lovesick Cyborg

Lovesick Cyborg examines how technology shapes our human experience of the world on both an emotional and physical level. I’ll focus on stories such as why audiences loved or hated Hollywood’s digital resurrection of fallen actors, how soldiers interact with battlefield robots and the capability of music fans to idolize virtual pop stars. Other stories might include the experience of using an advanced prosthetic limb, whether or not people trust driverless cars with their lives, and how virtual reality headsets or 3-D film technology can make some people physically ill.

About Jeremy Hsu

Jeremy Hsu is journalist who writes about science and technology for Scientific American, Popular Science, IEEE Spectrum and other publications. He received a master’s degree in journalism through the Science, Health and Environmental Reporting Program at NYU and currently lives in Brooklyn. His side interests include an ongoing fascination with the history of science and technology and military history.

ADVERTISEMENT
ADVERTISEMENT

See More

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+