AR Cooking Experience Design
ChefAR
The design is about a learn-to-cook experience in AR that immerses a home chef fully in the cooking process. No more switching between ingredients and websites or videos (and touching your phone or tablet with dirty hands). The cooking instructions are overlaid on real food as you cook, making learning a much more hands-on and engaging experience. The product completes the entire cooking cycle from learning what’s available in the fridge, recommending recipes, to demoing step-by-step animated instructions with voice. Because our AI-enabled product handles the logistic hassles in the background, the home chef can focus on her relationship with food and fully enjoy the learning experience.
Project Type /
Reality Virtually Hackathon at MIT Media Lab
Role /
Strategy,
UX & UI Design
Team /
Lei Xia,
Dasiy Zuo,
Sharon Yan,
Yaqin Huang,
Year /
2017
Problem Statement
Ever aspired to become a celebrity chef featured on Chef’s Table? Well, we did - but let’s face it, cooking is REALLY HARD. From picking ingredients, creating a recipe, adjusting flavors, to final plating, each step requires knowledge, focus, patience and even talent. And NO, tutorials on Youtube or BlueApron don’t help much - they do not care about your grocery, schedule, or cooking pace. You may have to keep hitting the replay button with fingers covered in both grease and keyboard-harbored bacteria while trying to rescue yourself from a potential house fire...
Solution
Our application leverages AR and AI to solve all that, and more. It provides step-by-step assessment, recommendation and instructions to guide you through the entire cooking experience through Hololens. It starts by recognizing the ingredients in your fridge, and based on that recommends multiple recipes that are both health-conscious and customizable. After a recipe is selected, it teaches you how to cook the dish, step-by-step, with verbal instructions and interactive animations at your own pace. It values flavors as much as presentation. Towards the end, it will provide interactive plating examples to make sure that your dish looks restaurant-ready. Plus, you will have the chance to take a picture of your completed dish and share it with your friends, family, and fellow chefs via the social network.
ChefAR
Platform: Microsoft Holo Lens, Development Tools: Unity, SDKs: Hololens SDK, APIs: N/A