Prototype

Briefing

You and the facilitator, Stephen Jones, are collaborating on a revolutionary user interface project. As you would not want secret information about your project to leak out to competing 6.813 students, you must be very careful about any conversations you have with Stephen, whether in person or online. Stephen provides you a phone with a Secure Messaging application to allow you to communicate securely with him when you need to discuss sensitive topics.

Remember:

• You are using SecureMessage because you care deeply about sending and receiving sensitive messages securely
• This is a phone application and is designed to act similarly to other mobile chat/texting applications
• Think out loud and let us know how comfortable you are performing these tasks given your need for security

Tasks

Task 1

Stephen hands you a new phone. Log into the Secure Messaging application, and add Stephen (who is physically standing next to you) as a Secure Contact. Confirm that he in your list of Contacts. 

Task 2

Stephen leaves the room. After a few hours, you receive notification of a new message. Go to your message inbox, determine who the message is from, and read the message. Since you are comfortable with whom you are communicating, you decide to reply to the message.

Task 3

Stephen has added Rob Miller to the chat. Since you trust Stephen, accept Rob as a chat participant. Review Rob's message, then add Rob as a new secure contact.

Task 4

You receive word that Rob Miller is actually a 6.813 student looking to steal project ideas. Since it is an imposter, you must remove Rob Miller as a contact. In addition, you want to remove all trace of communication with the imposter so remove all messages received from Rob Miller.

Round 1 Observations

User 1

Observations during trial

• Initially hit "manual" instead of "QR" when adding contact
• Got new mail when on contact screen, initially clicked their only contact to try to read the message. When the option menu popped up, they realized this was wrong and clicked the inbox icon.
• Clicked directly on the lock icon when adding Rob to contacts (this is correct, but the whole name was acceptable)
• Quickly clicked straight to contact list hen told to remove contact (correct)

User comments

• Was not clear how to remove all messages for a user initially
  • Looked for some remove-all command before trying to delete the contact
  • Found the remove-all button on attempting to delete
• Concerned about how to delete all messages after removing a contact (We didn't think of this yet--so something to keep in mind)
• Described it as "Overall very straightforward"
• Multiple/single person chat icons were fairly clear
• Wants to know how they would revert to single chat with a person when in multi-chat, having both run in parallel.

User 2

Observations during trial

• Took a couple seconds to find the "Add contact" button on the contact page, though it's the first/only they tried
• Asked "do you have a QR code?" to the 3rd party when prompted
  • Maybe we should make this step more obvious or explicit? Many people would have no idea what QR code means, even if they were looking at one.
• Wasn't too certain initially clicking "identity" to show QR code, though remarked "not immediately obvious, though makes sense. 'Identity' is good."
• On receiving message, took a little while to decide to click the message. Wasn't sure if it would bring to a chat window or if they would reply somehow from that page.
• "How would I add Rob to the chat?"
  • The other person added them to the chat, so I should be able to--otherwise that seems sketchy in a 'secure' application.
• "How do I know it's Rob?"
  • Concerned with adding contact they have not personally verified; notion of contact & trust tied in users mind
    • Should add a new color, indicating an added contact that is not explicitly trusted. (Yellow)
• Clicked lock icon next to Rob's name to add. Was concerned that the area was just the lock (it's not)
  • May be worth making it somehow clear the action area is the lock and name by tying them visually.

User comments

• Confused as to trusting vs adding as contact
  • "I can never message someone I don't trust--I consider a contact/trust to be different things"
• "Why would I want to have all-traces of a conversation deleted?"
  • Later remarked they understand it may be important for different users, just not them personally
• Linguistically confusing with using delete and remove interchangeably in the interface
• Make it more obvious that "Remove all messages" isn't a "confirm action" checkbox for safety, as it's a completely separate option
• I can't message someone I can't trust
  • Goes back to adding a new contact type, untrusted but on contact list with yellow icon.
• We should maybe add a "verified badge in addition to the lock icon" -- better clarity for n00bs.
• The lock icon has a good trust affordance

User 3

Observations during trial

• Was very quick opening contacts and clicking the add contact button; found them easily
• Remarked we may want to show a picture of a QR code instead of just saying "QR code" for clarity
• Was quick to hit identity when sharing contact information
• Clicked on name to open chat
• Surprised that when adding Rob, GUI elements turned red
  • "looks sketch" -- "does not look secure at all"
• Adding Rob to contacts was "so easy" compared to adding someone in person
• "Worried if I delete [Rob] first then I can't delete messages"
  • Was happy to see the open there on delete all messages when removing contact

User comments

• "Worried about paper trail" -- record on other party's phone despite deleting
  • e.g. group chat, wiping their messages leaves them on the other person's phone. Also, any messages said to them would still be there in a group chat setting, potentially giving stuff away.
• It's way too easy to add someone who is added to a chat, esp in comparison to normal addition of a contact
  • "Felt weird to add" via added introduction
  • Would "rather add everyone securely"
• Add an added-but-unverified category

Overall Observations

• Add a 3rd category of icon, a user who's a contact but not 'trusted' -- yellow icon would be a good indicator of this
• Allow inviting people to chat to make everything make sense
• Make it clear it's possible to wipe messages w/o deleting
• Make the 'wipe all messages' checkbox more obvious as to it's purpose--it's not a 'confirm action' checkbox, which it may be mistaken for by an inattentive user
• Make it clearer that the lock icons AND names above messages in conversations are clickable by tying them together somehow, so user isn't trying overly hard to aim on the lock only

Iteration 2

Changes

• A distinction between verified and unverified contacts was added. Contacts added by QR code are automatically marked as verified while others are not. This allowed users to add contacts with out fully trusting their identities.
  • Unverified contacts/messages were marked in orange and verified in green.
• A button for inviting a user to a chat was added.
• The 'delete all messages' checkbox text was changed to "Also remove all records of communication with this user?" to indicate that it was an additional action.

User 1

Observations During Trial

• Was under the impression that users added to a chat would be able to view past messages in that chat.
• Thought that the chat icon (1 silhouette for individual chats, 2 for group) would include one silhouette per person in the conversation and each silhouette's color would match each user's trust level.

User Comments

• Change the color of the back of the screen to match the current "security" of the chat.
• Don't make messages from unverified contacts stand out so much (they were highlighted in orange).
  • The user noted that she would probably mark all of her contacts as verified so that the application wouldn't bug her (which would defeat the purpose of verified v. unverified contacts).
  • She suggested that the lock icons stay orange but that the background for messages from unverified users should be a neutral color.
• In contrast to a previous user, she wanted to keep the ability to add contacts easily (that is, not requiring that all contacts be added by QR code).
  • She suggested that contacts should be verifiable by QR Code. If she has a contact and later meets the person face to face, she wanted to be able to verify the person by taking a picture of their QR code.
• Change the "Identity" section to "Me".

User 2

Observations During Trial

• The user appeared confused about the meaning of "verified".
• Tried to wipe messages from a contact by editing the contact.
• Then tried to wipe the messages through the settings page.

User Comments

• Make the function of the plus button on the chat window (add user to chat) more obvious.
• Add a way to easily return to the message browsing screen (back button).
• Add actions to messages (allow users to copy, forward, etc.).

User 3

Observations During Trial

• When adding a contact, this user didn't know what QR code meant.
• Wanted the ability to to defer an "add to chat" request to discuss it.
• Was confused about the distinction between adding a user to a chat and to the contacts.
• Felt that red (on unknown messages/user) indicated "angry" (not untrusted).
• Didn't understand the significance of orange messages/contacts (unverified contacts).
• Didn't understand the difference between verified and unverified contacts.
• The user took a while to realize that he would need to click on the username above Rob's message to add him as a known contact.
  • He initially thought that he would need to go out and find Rob.

User Comments

• "I have decided that I have no security interest at all."
  • The user didn't feel the need to distinguish between verified and unverified users and didn't like the overhead.
• The user noted that the username above messages would appear more clickable if it looked like a link.

Overall Observations

• The method for wiping messages is not obvious. In reality, this may not be a problem as this task is extremely rare. However, one possible solution is to add a "Erase messages from user" option to each user's "User Actions" menu.
• The distinction between verified and unverified is confusing for some users. Making it less intrusive and adding help text may prevent some of this confusion.
• All of the users hesitated slightly when switching to the identity section when sharing their identity. Changing this to a "Me" section should help.