BioMate

Lab-bench biologists find it difficult to use many existing tools for their data analysis. These tools are generally command-line computer programs written by computational biologists. Computational biologists do not have time to create user-friendly interfaces for their programs, and often find themselves spending a lot of time helping lab-bench biologists run their programs. This creates a burden for all involved: lab-bench biologists cannot move forward with their data analysis, and computational biologists cannot move forward with their research.

Click on a page above for a specific section.

Paper Prototyping

Prototype Photos

Briefing

Lab-Bench Biologist

You are a lab-bench biologist. You frequently collaborate with a computational biologist in another lab who writes scripts to analyze the data that you have generated in your lab. Sometimes you want to be able to run these scripts yourself because you want to analyze many different datasets and don’t want to bother your collaborator each time you have some new data. The problem is that you are not comfortable using a command-line interface, but your collaborator does not have time to create a fancy GUI for you to run each of their scripts.

We have provided a tool here that allows your collaborator to easily create a GUI for their script and share it with you. You will use the GUI to input the parameters that you want to run the script with. After specifying the parameters, you will be able to press a button to generate the correct command-line syntax for running the script, which you will then cut and paste into the terminal to run.

Compuational Biologist

You are a computational biologist. You frequently collaborate with lab-bench biologists in another lab and write scripts to analyze the data that they have generated in their lab. Sometimes your collaborators want to be able to run these scripts themselves because they may want to analyze many different datasets and don’t want to bother you each time they have some new data. The problem is that your collaborators are not comfortable using a command-line interface, but you do not have time to create a fancy GUI for them to run each of your scripts.

We have provided a tool here that allows you to specify the command-line structure of your script and generate a user-friendly GUI for your biologists. The biologists will use the GUI to input the parameters they want to run the script with. After specifying the parameters, the biologists will be able to press a button to generate the correct command-line syntax for using your script, which they will then cut and paste into the terminal to run.

Scenario Tasks

Lab-Bench Biologist

1. You would like to run a simulation using a Monte Carlo script which has been written for you. First, find the interface for that script.
2. Now, make a note for yourself saying that you are going to try running the script on the file ‘abc.txt’ in your home directory.
3. Next, generate the command for the script which prints verbose output to the terminal, runs for 2000 iterations, and is run on the file ‘abc.txt.’
4. You realize that the command as generated above causes the script to run for too long. Make a note for yourself saying that 2000 iterations takes too long. Also include the command you just used.
5. You are leaving for the day, but would like to return to work on this tomorrow. Save your current configuration so you don’t have to input all the settings you currently have all over again when you come back.
6. It’s a new day! Now, go back to the Monte Carlo script interface, but this time generate the command which does the same thing as you did yesterday except runs the script for only 100 iterations.

Computational Biologist

1. Create a new interface for a Monte Carlo script you have written.
2. Build the interface for the command ‘perl /home/X/montecarlo.pl [-v] <INPUT_FILE> | less’, where “-v” is a flag for verbosity and “<INPUT_FILE>” is a required parameter and specifies the file path. Make sure that the lab-bench biologist knows that this path is absolute.
3. Edit the Monte Carlo script interface you just created to add an optional parameter which will allow the lab-bench biologist to specify the number of iterations the script runs for. The flag for this is “-n” and the default value should be 100.
4. Make sure the lab-bench biologist is aware that this command will print its output to the terminal. Also, it should be suggested to not run the simulation for more than 10,000 iterations. Finally, add a description explaining what a Monte Carlo simulation does, and leave your contact information for the lab-bench biologist in case they require clarifications.
5. Share the Monte Carlo script interface you have created with your collaborator “V”.

Observations: Monday's User Testing Session (Round 1)

Takeaways

• Most users do not notice the help buttons until they are really stuck, and sometimes not even then. Important help information should be displayed up-front.
• The interface for computational biologists is extremely hard to figure out if you don’t see the tooltips, but the task flows smoothly after the help buttons are found.
• The most helpful parts of the help buttons for the computational biologists are the examples.
• The ‘drag and drop’ in the interface for computational biologists constructing the command structure has a visible affordance.
• A ‘save as’ option for the computational biologist might be nice.
• Most users do not notice the “Browse” button if they already have a file name in mind.
• The “Notes” button on the lab-bench biologist interface could be made more noticeable.
• The people testing the lab-bench biologist interface had a very easy time. However, our scenario may have been too simple.

User Testing Descriptions

User 1 (Role: Computational Biologist)

Observations

• The name “Add/Edit Script” was effective for creating a GUI for a script, user clicked it appropriately.
• Could guess correctly the purpose of “Add Text” and “Add Parameters” button in the “Command Structure” panel
• Only noticed the “?” (help button) in front of “Alias” when unsure about the meaning, found the help message confusing, asked for clarification.
• Was confused by the “| less” part of the command we presented as part of the sample task, thought of it as a parameter first and clicked on “Add Parameters” button.
• Set the type of inputfile to “string” (we had expected the type to be set to “file”).
• Asked, “What happens when you double click on the parts in the command structure?”. This feature was not implemented for the prototype. But we plan to give the user ability to modify that part when it is double clicked.
• Was confused whether clicking “Save” will automatically save the GUI since there was no feedback (this was an error of the computer forgetting to write “saved @timestamp” onto the screen)

Feedback

• Make the help message clearer (was adjusted on the fly).
• When type “Boolean” is selected the “default” should change to drop down list since it has only two choices (our failure to do this was a computer error).
• Give feedback after clicking “Save”.

User 2 (Role: Lab-bench Biologist)

Observations

• Was clear about the goal, clicked on “Run a Script” button appropriately.
• When the GUI for “monte carlo script” was shown, was overwhelmed by the things and temporarily forgot about the task.
• Took a while to notice the “Notes” button. Once noticed, clicked correctly on it to see the notes associated with the script.
• Did not pay attention to the “Browse” button in front of the “input file” field. Typed the name of the input file instead.

Feedback

• Liked the wheel and icons in the welcome screen.
• Found the interface intuitive and quite easy to use.

User 3 (Role: Computational Biologist)

Observations

• First thought the “Construct Command” panel as a textbox and tried to type the sample command in it.
• Did not find it obvious to click on “?” for help. Wanted to see examples of what is “text” and what is “parameters”.
• Was confused about what to put in the “Prefix” field. Did not click on help. Put “-n” first, then changed it to “-100” then looked at the “Default” field and figured out what is the correct thing to do.
• Was confused about what to put in the “Alias” field. After spending some time finally clicked on “?” and found the help message helpful.
• Was not sure whether a flag is a “parameter” or “text”. First looked at help for “Add Text”, then for “Add Parameters” and figured out what to do.
• Was unsure about the type of a flag, clicked on help and was able to figure it out.
• When presented with the task of reordering the parameters, the user’s first attempt was to click on the part in the “Command Structure” panel and drag it. The user was pleasantly surprised by the fact that the parts are actually draggable.

Feedback

• Make the help icon more visible.
• Give examples of what is text and what is parameter in the context of this interface.

User 4 (Role: Lab-bench Biologist)

Observations

• Did not notice “Browse” button for input file. Typed “input.txt” into the textbox.
• When told to make notes about “1000 iterations”, remembered seeing the “notes” button in the “monte carlo” interface and could locate it appropriately.

Feedback

• Found it intuitive to use the interface.

User 5 (Role: Computational Biologist)

Observations

• Faced no trouble finding the “Add/Edit Script” button.
• Noticed the “more” button next to the instructions.
• Was confused whether to put “-” before a prefix, looked up help for examples.
• Was not sure what to put as “Prefix” for input file, looked for help.
• Found the task of dragging the parts in the command structure panel and reordering them intuitive.

Feedback

• Found the help messages wordy. The examples should be clearly visible.
• There should be a “Save As” option so that the computational biologist can create slightly different GUIs for same script.

Changes for Round 2

Changes to the interface that allows computational biologists to create a GUI

We realized users did not notice the question marks next to the fields, and thus tried to use the interface without any help (which was very difficult). We therefore made the following changes:

• Examples are now visible next to all the appropriate fields (previously, the user had to click on the question mark next to a field to get a help message about the field which included an example).
• The question marks (which provide help messages) next to the fields are now in a different color. This also applies to the interface for lab bench biologists who are running a script.

In response to a user suggesting that a “save as” option might be nice (to save slightly modified versions of a script):

• We added a “save as” button to the interface.

To improve the safety of the design by explicitly encouraging programmers to describe the caveats of the interface:

• We introduced a field labeled “caveats” and modified our test to encourage the computational biologist to supply information appropriate for a “caveats” field.

An additional safety concern was that lab-bench biologists who mount drives to the server might browse for a file on the server through their mounted drive, and thus obtain a path to a file on the server that is only meaningful in the context of a mounted drive, and not when a job is being submitted to the server directly (as is usually the case). To solve this:

• We omitted the “file” parameter type; all “files” are now strings. It is not possible to browse for a file to obtain the file path.

Changes to the interface that allow lab-bench biologists to use a GUI

Our previous test for lab-bench biologists went very smoothly and did not test very complicated tasks. We thus decided to make the scenario for lab-bench biologists more complicated so that we would get more valuable feedback. We modified our scenario as follows:

In our first iteration, a user would click on “Notes” and would be presented both with “Programmer notes” (effectively instructions for running the script) and “My notes”. In retrospect, it was not obvious that clicking on “Notes” would provide both instructions for running the script and a place to take personal notes. The Notes button was also too small. We therefore made the following changes:

• The “Notes” button was replaced with “My Notes”, it was made bigger, and clicking on it only provided the user with the notes they had taken for the script.
• Underneath the “Notes” button, there was a “Caveats/Instructions” button (note: we got rid of this button after it failed miserably in a subsequent user test; see below). Clicking on this button provided the user with what was formerly under “Programmer Notes” . The programmer’s notes were now divided into “caveats” and “instructions”.
  We included important information for running the script under “caveats/instructions”, and tested whether lab-bench biologists were likely to click on this button to obtain the relevant information.
• When we performed our first test of this interface on a lab-bench biologist, we found that the biologist did not notice the button at all, even after being prompted, in part because it wasn’t aligned with the “My Notes” button above it. The biologist also said he was unlikely to click on such a button. We thus decided to make information on caveats more prominent in the following ways:
  • For every input field, we decided we would display caveats pertaining to the field in red (e.g.: “file paths must be absolute”). This would also require modifying the interface for computational biologists to include describing caveats for every individual field.
  • We also decided to display caveats pertaining to the whole script in red at the top, with a button for “More”. Pressing ‘more’ would bring up the caveats that pertained to the script as a whole.
  • The caveats/instructions button was now just “instructions” and aligned well with the “My Notes” button above it. Clicking on instructions still brought up a page that had both the caveats and the instructions listed.
• The popup after pressing “Generate command” now contains a “Copy to Clipboard” button.
• If a lab-bench biologist presses “save to notes” on the “generate command” popup, they are taken to their notes page and can actually see that the command has been inserted into their notes. They are then free to provide surrounding annotation for the command. This was done to improve feedback for the ‘save to notes’ command and to also improve its usefulness since one rarely saves a raw command without surrounding information on what the command did.

Observations: User Testing on People From B Lab (Round 2)

Takeaways

• Once the examples are visible up-front, computational biologists rarely need to click on the help buttons to figure out what to do.
• The interface for computational biologists to add more parameters might be a little tedious if there are many parameters to add. It would be nice to be able to duplicate parameters and modify just a few fields.
• “Script Name” at the top of the computatonal biologist interface should be renamed to “Title”, so that it is more clear that the name here can be distinct from the name of the script on file.
• It would be nice to be able to edit the parameters by clicking on the parameter “widget” in the command structure window (currently, parameters can be edited by clicking on them in the list underneath the command structure window).
• It is important to be able to explicitly specify the order of the parameters, since many scripts that are written do not use prefixes for their parameters.
• Once the caveats are presented up front in red text (rather than hidden behind a button click), they are noticeable to lab-bench biologists.
• While lab-bench biologists in the Boyer lab have some familiarity with UNIX and know how to get absolute paths, this is not true for biologists from other labs. Some support for how to get absolute paths should be provided.
• The landing page for lab-bench biologists is confusing, particularly when the lab-bench biologist wants to load parameter settings for a script they were using earlier. There is a tendency to click on “Add/Edit a script” (there is a misunderstanding about what ‘script’ refers to).
• It would be nice to access recent scripts that the lab-bench biologist was using.
• It would be nice for lab-bench biologists to have the option of inserting the current time-stamp when creating a note.

User Testing Descriptions

User 1 (Computational Biologist)

Observations

• Was confused about what to put in the “Script Name” text field. He wasn’t sure if it should be the actual file name (e.g., montecarlo.pl), or a higher-level description.
• When adding static text to the command, he thought he should separate all the parameters by commas. He didn’t initially realize the difference between static text as defined in BioMate and the actual “text” of the command itself. He quickly realized he should click “Add Parameter” for parameter options, however.
• On the “Add Parameter” pop-up, he initially ignored all the text fields except for the flag “-v”. He thought this would be sufficient to specify the command.
• Given the parameter ‘widgets’ in the command display area, he wondered whether he could right-click on the widgets to bring up an edit window.
• When saving his script interface, he wanted to control the file name by selecting “Save As” instead of “Save”. He wasn’t quite sure what the file name would be were it to just be “Saved”.
• Felt that adding more than three parameters one-by-one in the provided fashion would be tedious. Wanted a way to batch-entry parameters, or at least copy rows in the table for faster editing.

Feedback

• The order of the parameters for his scripts is often important.
• Would like to be allowed to edit the parameters by right-clicking on the widget, or at least providing some affordance for doing so (such as a pencil icon).
• Rename “Script Name” to “Title”
• Would like an option for batch input of parameters, as mentioned above; however, his scripts usually have about six parameters.

User 2 (Lab-Bench Biologist)

Observations

• The user was not sure how to bring up “monte carlo” script. He looked for a way to to find the available scripts. He was not sure about the difference between “Add/Edit Script” and “Run Script” (he commented that “Run Script” is something he would expect to click on after a script had already been loaded; after his test, we amended the names to “Add/Edit a script” and “Run a script” for clarity). Clicked on “Add/Edit Script” first, figured out this is the wrong place and then went back and clicked on “Run Script”.
• Did not notice “Notes” button at first glance, instead clicked on the “My Notes” button in the top toolbar which leads to all notes of the user rather than script level notes.
• The “Save Parameters” and “Load Parameters” buttons were effective, the user could appropriately use them.

Feedback

• The user suggested it might be more intuitive to keep one note per day instead of one note per script, in accordance with “lab notebook” mentality (however, this contradicts our need-finding, as we found that the act of keeping several separate notes pertaining to a script can quickly get out of hand as the number of notes builds up).
• The position of “Caveats/Instructions” button was not very intuitive (may be because of being a paper prototype interface; after this user test, we altered our interface to get rid of the “caveats/instructions” button altogether).
• Might look at instructions but is not quite used to seeing caveats. Wants the caveats to be shown on the face.
• The “?” help icons should stand out more.

Users 3 & 4 (Lab-Bench Biologists, two testers at once)

Observations

• Finding the Monte Carlo script interface was intuitive.
• The “?” help button affordance was noticeable and gave good feedback.
• One said that using ‘pwd’ would give the correct pathname they needed, but this may not be well-known to others. The other tester wanted to browse their file directory instead (but this would not work either if the job was to be submitted to the server, as is usually the case).
• “File path must be absolute” was not understandable, and even if it were it is not clear how to obtain that information from a UNIX server.
• No issues with finding the “Save Parameters” button to preserve the current configuration.
• However, considerable confusion when wanting to return to the script. They first chose “Add/Edit Script” since they intuitively thought they were editing their past modifications. Eventually, they realized they should do “Run Script” then “Load Parameters”, but this was far from clear to them.

Feedback

• It would be nice if clicking “save to notes” saved not only the raw command line, but also explicitly specified what the parameter values were (since many biologists are not used to looking at a command line to figure out the parameter values).
• “Load Saved Parameters” instead of “Load Parameters”
• When wanting to return to a script, have something like a recent activities on the home page to load what they were just working on. Perhaps an option for “Open Script”.
• There should be a clear separation between which buttons on the home page are for programmer-facing tasks and lab-bench biologist-facing tasks.
• The home page felt a bit cluttered; it would be better if there were fewer options.

User 5 (Lab-Bench Biologist)

Observations

• Had no trouble hitting finding way to the monte carlo interface.
• Clicked on instructions first thing, without being prompted to.
• When making the note, first thing she wrote down was the date. It may be good to have an “insert time-stamp” function to automatically put down the date.
• Was familiar with the concept of an absolute path, so did not struggle with that.
• After she input her parameters, hit save parameters without being prompted to.
• When she got the “generate a command” screen, she also pressed “save to notes” without being prompted.
• Had the same problem as previous testers with trying to load the old session. The first thing she pressed was “Notes” on the home screen, selected “my notes for monte carlo” from the list of notes, reviewed what was written, clicked close, and was taken back to the list of notes. She said “uh oh”, and went to the home screen. There she clicked Add/Edit a script, just as previous testers had made the mistake of doing. When she saw it wasn’t what she wanted, she pressed home. Then, she clicked run a script, selected monte carlo, and clicked “load parameters”.
• Rest of the test went smoothly.
• In between, she said “can you just live inside my computer?”

Feedback

• The names for “Add/Edit a script” and “Run a script” should be changed; she suggested “Use a script”.
• Other than that, it was a fun user testing experience.
• When told she was the only user who had clicked on “instructions”, she exclaimed “why does no one click on instructions?!”