...
We also spent much time deciding which features to include in our early prototypes when a simple user test could have decided for us quickly and easily. For example, we put a lot of time into designing an editing mode in which the user could focus the textbox and navigate between characters already typed to insert or delete characters. However, when users played around with our prototype we observed that no one ever tried to edit the text - we had to tell the users to deliberately edit some text in order to evaluate our design of the edit mode. Had we gathered user feedback earlier, we would have decided to not include an edit mode.
After encountering many problems with our early prototype seen in Fig 1, we considered two alternatives. One is the final design seen in Fig. 2. The other is a variant of our final design that uses a tertiary search instead of a binary search. We thought this would further reduce the number of keystrokes, but were not sure how intuitive the controls would be. Here, we applied what we learned from our early prototype and immediately gathered user feedback and decided not to proceed with tertiary search.
Our evaluation also taught us the relative importance of subjective versus quantitative evaluations. We knew our keyboard required less keystrokes, so in one sense our design was better. Our keyboard was nevertheless slower to use, so in another sense it was worse. We still received much positive feedback about our design, suggesting that with more practice, our keyboard could become a more popular alternative.