Speech and Touch (Katya)

 

 

STEP 1: Arjun talks to his son, Raj, on iPad or iPhone. Raj describes how amazing it was to drive to LA from SFO through Highway 1. Teleport automatically stores spatial descriptions mentioned by Raj.

STEP 2: After talking to Raj, Arjun scrolls through spatial descriptions stored and picks one. Avatar ask whether he wants to travel by foot or car and then how would he like to describe the route: 1) by start and end 2) by landmarks 3) by a reference object.

Step 3: Depending on the option there are three possible moves: 1) say start and end locations; wait while the animation is generated; watch the animation. 2) say landmark locations; wait while the animation is generated; watch the animation. 3) say the reference object; use 'move forwards','move backward', 'turn right' and 'turn left' commands to navigate around the object.

STEP 4: Interaction viewing of animation. While watching the animation by tilting the phone to the right or left Arjun can interactively change viewing angle of the animation.

STEP 5: By Tapping over the objects Arjun can see additional information, such as name of the object, the weather at the moment of the conversation or even additional images if added by his son.

Learnability

  • Interface extensively uses visual clues that make interaction extremely intuitive for elderly people who are often not accustomed to the use of conventional desktops. Visual clues take advantage of conventional metaphors and colour.
  • The use of metaphors and colour scheme are both internally and externally consistent.
  • Furthermore, speech based interaction reduces cognitive load by sequentially offering relevant options.
  • The interface implements animated icons that symbolise receiving speech or avatar's speech, giving a clear visual representation of system's state.
  • The problem with learnability might occur if the system fails to recognise the speech command because it can be extremely difficult to explain to the user what is wrong in the way they say things: users might be pronouncing words not sufficiently clear, there may be background noise, users can use out of vocabulary words.
  • In order to convey what verbal input is expected, questions are designed as either 1) restatement of the phrase used by the avatar (e.g. "Choose one of the following options…" expects a direct restatement of one of the options), or 2) when possible questions prompt a one word response (e.g. "Name the first landmark" or "Name the start of the route").

Efficiency

  • Speech based interaction alone may be inefficient for users who use the system very often. The fact that the system asks the same questions can become irritating after some point.
  • Nevertheless, the targeted user group is elderly people for whom physical manipulation often may pose additional challenges. From this point of view the interface is considered efficient.
  • The interface combines speech and touch which allows to delegate problems that would be difficult to solve with speech only to a touch based interaction. For instance, In order to get additional information about an object the user needs to tap over the object. Another example is that the system records a history of spatial descriptions that the user can scroll through because memorising all the location is not feasible.
  • Difficult to convey which statements will work. To help the situation the system asks questions that prompt one word response, such as "Name the first landmark" or "Name the start of the route". The system also provides and animated graphic representation of the expected input.
  • Generating an animated video of a route can be very time consuming.

Safety

  • The weakness of the system is that there is infinite number of speech commands that users can try.
  • The interface improves safety of the system by making it to ask questions sequentially and in case of noisy input ask additional questions that help to disambiguate input.
  • Generating animated videos of the route can be time consuming, therefore, there is a risk that users can get impatient and start tapping the screen or giving additional commands.
  • No labels