Robo-Monitor - User Testing

Design

In our project we created an Android application called Robo-Monitor. Android applications consist of a number of activities, where each activity has its own GUI. The Robo-monitor consists of 3 activities:

Channels overview screen

This is the initial screen which opens when the user opens the application. It shows all channels for which there is data arriving, with some statistics about the data like bandwidth, frequency, and values.

Since there could be many channels, the user can "star" certain channels and hide all the rest. When the user stars a certain component, the text label of the component gets bolded - to increase visibility.

Main screen with no items starred and the control channel collapsed (left). Starring items makes their entry bolder (right).

A menu allows the user to view only starred items. Often many channels have various timestamps and debug information that is helpful to hide. Once selected, the list is much shorter (right).

Graph overview screen

This screen presents a graph widget which allows the user to observe how data has changed over time.

It allows the user to use touch gestures to scroll and zoom. We also provided Track functionality so that they can watch the data stream as it appears in real time.

The graph axis have deliberately small labels.  In GR4 we received feedback on GR4 that these may be hard to read, but when we experimented with bigger font sizes they made the screen cluttered and were very distracting.  While not perfect, we believe that small labels is the right usability choice.

Graph view and view with the menu option. When the user drags the graph, controls appear allowing for a zooming and tracking the current plots. Note that multi-touch zooming is also implemented, which is a key feature for our system.  During testing, some users had trouble finding the menu option on this screen, so we are considering moving it or adding another method of access.

Plot with multiple traces. The legend in the upper right displays the channel and variable names. We chose to use a font similar to the axis labels to be as small as possible but still readable.

Graph traces selection

The graph widget can display multiple graphs at the same time.

We provide a utility for the user to select which traces are shown and easily change that

When this screen opens, channels that currently have variables being plotted are automatically expanded. In all tests, users quickly understood the purpose and usability features of this screen.

Implementation

Our implementation uses combination of standard and custom widgets.

• Graph widget: we based our main visualization tool -- the graph widget -- on a open source calculator app called Arity. We added improvements to their widget and fixed several bugs. We plan to submit patches upstream for enhanced muti-touch zoom and infinite scrolling. The graph widget is implemented as a canvas screen which is redrawn periodically depending on a software timer.  The plot itself is drawn with the versatile Android Path object.

• Star widget: the star widgets on the channel screen are implemented with an ImageView and a corresponding boolean variable. Their functionality is almost identical to a checkbox, but it was easier to recreate this functionality that to modify the checkbox object's layout and integrate with expandable lists.

• Expandable lists: To create the main channel views, we use expandable lists.  We have custom list adapters which create each view (or row) in the table.  These list adapters each have their own layout and internal click listeners which allows us full functionality in their look and feel.
  • A drawback of this method is that when the data is updated, each row in the expandable list is redrawn, meaning that the onClick listeners for the stars are recreated. If the user has clicked during this time, that input will be lost.  This factor prevented us from exploring table update rates faster than 1Hz.  We may be able to work around the issue by providing a custom update function.

• Add trace screen: The add trace screen is implemented in a similar manner to the main screen's expandable lists, with different custom list adapters to reflect the need for checkboxes.

Backend

• We run a background service that generates data and stores it in the main Application class (which is shared among all activities.)  We do this for two reasons: (1) we wanted the data must be consistent when the user switches between activities, and (2) we plan to hook real LCM data streams into the application so we wanted an architecture that could support this.

• To show real-time information about the data we have a periodical, background asynchronous thread which "pings" the main GUI thread of the channel overview screen. This way we assure that the GUI will remain responsive even if there is a lot of data to be processed.

Evaluation

We tested our interface with 3 robotics researches from MIT. All of them were possessed Android phones. We think this is important, so that they are familiar with Android's app conventions (such as using the menu button to explore what you can do).

Briefing

All of our users had the appropriate background knowledge that channels are dependent on a robot, and understood that we were using randomized data we needed only a simple briefing:

"We have built an interface for visualization of LCM streaming data. We support only few types of channel data structures: those, which have only one layer of sub-components which take float values. Try using the app in order to explore the data stream."

In order to simulate the users' first or second use, we did not create very specific task lists -- instead we wanted to see which parts of the app the users would discover.  We ensured that before testing was finished, each user had explored the main parts of the interface:

• Use the stars
• Explore a channel
• Plot multiple graphs at once

User 1:

Initial Android skills: high

Observations:

The user didn't use stars right away, but rather selected a channel component and started looking at the graph. He commented that the graph seems very intuitive, and liked the zoom out and track ideas. He mentioned that clicking on a component and getting the graph immediately is a very direct way to observe the data and liked that it feels fast. He explored the menu in the graph which allowed him to select traces to be shown in the graph. We didn't observe any difficulty when he added traces.

When he returned to the main screen we asked him about the stars but he was not sure what the stars were for. He commented that it would be nice if there was a way to add traces using the star functionality. He also suggested that one should be able to long-tap a channel to see possible options (such as plotting all components, or starring all components, etc.)

Usability Issues & solutions:

• Initially unclear what stars do
  • Not necessarily a substantial issue since it is quickly resolved when the menu is opened.
• Multiple trace support was expected on the main screen, not on the graph
  • Implement a way to plot starred items at one.

User 2:

Initial Android skills: medium

Observations:

This user spent more time than the first one on the channel overview screen. He noticed the channel statistics and the component values and tried folding/expanding different channels. He was starting to repeat his actions and asked us if there is a way to plot some of the data in 3D when he clicked on a channel component (called "x") and that brought him to the graph. He was then delighted with the graph and used the buttons to track the graph and to zoom-in / zoom-out. He also moved the graph up and down but did not discover that it is possible to use multitouch to zoom. He did not observe that it is possible to add multiple traces to the graph so we asked him how he might go about it. He tried a long-tap on the graph without success. He eventually moved back to the channel screen and looked at the stars where he figured out that he could show a subset of the channels. He suggested that long-tapping allow for graphing all starred items.  After exploring this avenue, he eventually located the menu option on the graph and successfully completed the task.

Usability Issues & solutions:

• Starred items should allow for multi-trace graphing
  • Menu option or long-tap
• Confusion about _Track_ button
  • "Clicking anywhere on the screen does the same thing as track"
  • Track should change its label when tracking/not tracking

User 3:

Initial Android skills: high

Observations:

This user looked quickly at the channel overview screen and easily found the graphing functionality. He was very active, trying many interactions rapidly. He immediately turned the phone on the side to look at the graph, a feature only a few others users have tried. He seemed lost in the graph and then discovered "Oh, there's the track button." We then asked the user to plot several traces at the same time. He first tried to star the traces in the channel overview screen and hold, but without success. He then tried holding a channel which did not work either. Finally, he then went to the graph and tried the menu option leading to success.  When browsing the graph he tried using the _Search_ soft-button on the side of the phone, but this didn't do anything.

This user suggested that we allow for plotting quantities against each other, allow for mixed scale Y axis (where scaling can be applied to certain traces to ensure they fit on a single axis).

We asked the user to try using the stars, and he was able to hide certain channels. He mentioned that this will be very worthwhile when there are more channels.

Usability Issues & solutions:

• Search unimplemented
  • This is not a critical feature, but we may want to implement search, especially when the number of variables because large
• Starred items should allow for multi-trace graphing
  • As before. This is clearly our largest usability issue
• Long-tap on channels is a common interaction
  • We likely should provide options for this use case, including graphing an starring.

Reflection

Fast, iterative user testing was key to our success throughout the process. Even in our final round of testing, for example we discovered usability problems and solutions that we did not anticipate, such as the fact that most users would connect starring items with plotting them together. We were so close to the application that we had completely separated the features in our minds.

We were happy with our initial prototypes and felt that they generated good variation in design, allowing us to combine the best of each. In the future, however, we must concentrate more on constraint identification. In Robo-Monitor, an important constraint that we did not consider (until we started computer/phone prototyping) was that information comes as a stream, instead of as a static set, forcing the UI to change continuously. The form of our initial prototypes rendered them unable to alert us to this factor, so we were forced to rethink parts of the interface late in the process. That being said, we believe that the advantages of rapid iteration in paper prototyping are well worth the static nature. Overall, we found that method to be the most informative for our system.

While there is no "correct" solution in UI design, we gained an appreciation for UX designers' ability to evaluate user interfaces in a concrete manner. We found the user testing protocols to be effective in providing good converge of the features in the application while simultaneously ensuring that it was clear where the interface was at fault, missing, or incorrect. When evaluating changes to the interface, we found that asking the users themselves for suggestions was an effective tactic because, even when their solution was not optimal, their suggestions provided further insight into how they were thinking about the interface.

Finally, we would like to thank Mason Tang, the other TAs, and Professor Rob Miller for their guidance and knowledge throughout the course.