...
Our preliminary experiments indicate that while a mobile product look service would be useful and impact, user interfaces are slow and awkward. We believe that by developing an intuitive user interface for mobile image annotation and recognition that we can build a useful product lookup service. In
Task Analysis
Inspired by our experiences after interviewing subjects, we identified three primary tasks for a mobile phone application: quickly locating reviews for a product, efficiently comparing products, and clearly showing results to the user. In this section, we discuss these tasks for each user demographic.
How do users lookup products?
Current systems either have no method for specifying the object (instead, they assume it is centered), or they have (obtuse) interfaces for drawing a rectangular bounding box. Due to the sensitivity of state-of-the-art computer vision algorithms, poorly placed localizations lead to significantly incorrect identification results. Both cause losses in efficiency which compel users to use typed queries, or to not use their mobile device at all. Indeed, we need more efficient methods for specifying objects of interest in a photo.
How do users compare products?
*Goal: *The primary task for users is to lookup products using a mobile phone application. Our user studies reveal that all demographics desire an efficient interface for identifying products in a store.
Subtasks: To accomplish this, users must first take a photo of the object they wish to identify. Next, the next annotate the object by drawing a box around the object of interest. After deploying modern computer vision algorithms, the product is identified from a massive database. The user is then shown a screen with the product information.
Research Question 1: What is the best way for a user to annotate an object? Bounding box, ellipse, tapping, or enclosing hull?
*Research Question 2: *What is the best way to display results to the user? Popup or new screen?
How do users compare products?
Goal: Users (especially experts) want a system to quickly compare products with a side-by-side chart.
Subtasks: Users again take a photos of the objects they want product information for. In order to specify which objects to compare, they must annotate multiple objects. After recognizing both objects, a comparison screen is shown.
Research Question 3: What is the best way for a user to annotate multiple objects?
Research Question 4: What is the best way to display comparisons to users on a mobile screen? Column wise or row wiseCurrent systems do not support looking up multiple items in a single image, causing the user to have to repeatedly do the same task for multiple products. What is a visible, learnable and efficient interface for allowing users to specify multiple products in the same image?
How do users get unbiased results?
...