Versions Compared

Old Version 31

changes.mady.by.user Alan Q Huang

Saved on

compared with

New Version 32

changes.mady.by.user Alan Q Huang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Final Design

"Describe the final design of your interface. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, and user testing)."

Screenshot

Explanation

Originally, the main screen looked like the extended song list below. Our final design adopts a splash-screen-like design in order to highlight the name of the app and reassure users that they're in the right place, while remaining functional with a prominent "new song" button. The background is intended to be aesthetically pleasing but unobtrusive.

We limited the main screen to three saved songs to avoid overwhelming users. The "new song" and "about" buttons remain prominent. Tapping on a song launches the compose screen, shown below. Our original design included small "rename", "duplicate", "export", and "delete" buttons on each song, but our paper and heuristic evaluations noted that such nested buttons were confusing and difficult to use on a small screen. We therefore replaced them with a menu accessed by tapping and holding for a second or so; this method is nonobvious, so we also included an instructional note.

Renaming a song works by replacing the name in the song button with an editable text box; when the user submits (presses enter) the song is saved. Duplicating a song creates a new song named "Copy of NAME" and enters rename mode on that song. Deleting a song asks for confirmation, then slides the song button rightward off the screen. (It would perhaps be clearer for the button to shrink to nothing vertically with the buttons below sliding up, but technical properties of the Android API make implementing this animation difficult.)

Exporting uses native Android sharing functionality to display a list of possible transfer methods for the user to pick from. A MIDI file containing the song is sent via the specified method.

If the user clicks the "see all" button, this screen is shown. It lists all saved songs, allowing scrolling and searching (the list updates as a user types in the search box).

Here and above, each song button shows in approximate units the time since the last edit. Our original design used an absolute timestamp with second granularity, but testing convinced us that the extra precision wasn't useful. Currently the minimum interval shown is "<1 minute"; the display refreshes periodically in order to remain accurate.

The "about" button takes users to this screen, which is just a static message.

This is the compose screen for a song with no notes in it (e.g. a newly created song). Swiping horizontally scrolls the song and two-finger pinching zooms it, as is common for touch interfaces.

On the left is a vertical black line, which is the cursor showing the current temporal position in the song. Tapping on a blank section of the display moves the cursor to that point.

Along the top are various control buttons:
- "Play" starts playing back the song as audio; it's currently disabled because the song has no notes.
- "Record" starts recording notes from the microphone; this is described below.
- "To Start" moves the cursor to the beginning of the song, and is disabled because the cursor is already there.
- "Undo" and "Redo" work at the level of 'actions', either record or edit; these are described below.
- "Save" saves the song to private storage in the device memory.
- "Back" returns to the list of songs.

Along the bottom are track-manipulation buttons. The four colored buttons switch the current track, and the right button brings up the track tools menu. These are all described in more detail below.

A click track plays while the user is recording. The cursor turns the color of the current track, and moves rightward across the screen (the display scrolls so that the bar is never more than 3/4 of the way across). Notes are created at the cursor position when the app detects singing, and lengthen until they change pitch or stop; notes are deleted if the cursor hits them while recording into the same track. All buttons are disabled here, but "Stop" is available to stop recording.

Pressing "Stop" commits to the undo stack a record action, which incorporates all deleted notes and created notes. Thus, no matter how many notes a user records before stopping, all will be undone by one press of "Undo".

Here the user has stopped recording on the green track, and started recording on the purple track. All four tracks are drawn on top of each other; notes on the same track can't overlap, but notes on separate tracks can.

Though it's not pictured here, the play mode behaves similarly; all buttons except "Stop" are disabled, and the cursor moves rightward, playing each note it hits. However, the cursor is black, reflecting that notes from all tracks are played. (From heuristic feedback, if "Play" is pressed when the cursor is at the end of the song, it starts playing from the beginning instead of playing to the end i.e. doing nothing.)

Here notes have been recorded into all four tracks. Notes from the currently selected track are bright and large; other notes are dim and small.

The 'new' editing interface. Tapping on a note causes these icons to appear. Tapping the trash can deletes the note. Dragging the side arrows left or right changes the start and end points. Dragging the middle icon vertically changes the pitch of the note, and dragging it horizontally moves the whole note without changing its length. As notes in the same track can't overlap, the note stops when it hits something, and a red glowy thing appears to signal that it can't move or extend anymore.

Tapping away from the note deselects the note (and may select another note if that note was tapped). All changes made before deselecting or deleting the note are committed to the undo stack as an edit action.

This was approximately the original editing interface in our paper prototype. We decided to implement it because the 'old' interface below was confusing for some users, and direct manipulation seemed more intuitive. However, this interface is also somewhat suboptimal because the icons are small and hard to tap, and multiple accurate taps are required to successfully edit a note.

The 'old' editing interface. Tapping on a note causes this overlay to appear. Tapping the trash can deletes the note. Swiping left or right on the left side of the screen moves the start of the note backward and forward in time; swiping horizontally on the right side does the same for the end of the note. Swiping up or down in the middle of the screen changes the pitch. There's no way to move the note without changing its length. If the note tries to expand but hits something, it stops and a red glowy thing appears; also, the arrows that don't do anything are crossed out.
Tapping the back icon resets the change and commits nothing to the undo stack. Tapping the save icon commits all changes made as an edit action (as does deleting the note).

This was the interface we switched to for our second paper prototype iteration. It was intended to make it easier to edit notes by only requiring one accurate tap, and then giving users the whole screen for swiping. However, the overlay makes it difficult to see what's going on with the note, and even which note is selected; there's not enough feedback for actions the user takes.

Tapping on the track tools button brings the user to this screen, which has color-coded controls for each track. A muted track doesn't appear in the compose screen and produces no sound during playback.

The instrument menu allows the user to choose what musical instrument the track will be played as. These selections are saved with the song, so they persist across editing sessions.

If there are unsaved changes to the song when the user tries to leave the compose screen, the app prompts the user to save. There are three options: save then exit, exit, or don't exit.

The saved state of the song is saved as a location in the undo stack, so e.g. if the user opens a song, makes a change, then undoes it, they can quit without being prompted to save.

When saving a newly created song, the user is prompted for a song name. After that, saving uses the existing name, and renaming from the song menu is necessary to change it.

Originally, we saved an empty song named "Untitled ##" (where ## is a unique number) when creating a new song, and required renaming to specify a custom name at all. Our heuristic evaluators noted that it was weird to create empty untitled songs, and recommended letting users manually save and name new songs.

Implementation

Most of our app is written in Java using the Android APIs, but there are native libraries to perform playback and recording. (We used native code because there were no convenient Java libraries, and implementing the functionality ourselves would have been difficult and possibly resulted in slow and buggy code. We used a pure Java / Android API implementation for playback in the prototype, and the quality was fairly terrible.) There is also a third-party library (com.leff.midi) that implements the MIDI message protocol and file format. Recording grabs audio samples from the microphone and outputs pitches, from which the compose code can create notes. Playback takes notes and sends MIDI messages to a system audio library, which produces audio output.

Beyond those, the only important backend class is Song, which encapsulates the concept of a song with a name, a mapping of tracks to instruments, and a set of notes, each of which has a track, pitch, a start time, and a duration. Songs can be written to and read from file storage; the app creates a file in private storage for each saved song. Songs can also export themselves to MIDI files.

An activity is an Android concept described as "a single, focused thing that the user can do". It's typically a single screen or view that the user can see and interact with. Our app has four activities: the main screen, the about screen, the full song list screen, and the compose screen. The main and song list activities contain SongWidgets, which are the buttons in the lists; each one corresponds to a song and has code for interacting with that song.

The compose activity implements all the various button actions, interacting with recording, playback, and file storage. It has various state including cursor position, selected track, and zoom level. It also contains a list of notes. Each note is a custom GUI widget that draws itself onto the compose screen; it keeps track of its track, pitch, start time, and duration, and adjusts its color, position, and size as appropriate.

Each note listens for touch events; if the note is tapped, it starts the edit mode. In the 'old' interface, this is a full-screen semi-transparent widget that detects swipe events and modifies the note based on their starting point, direction, and length. In the 'new' interface, this makes the various icons visible; they listen for touch events and modify the note when dragged.

The compose activity contains the undo stack, which is a list of UndoItems. An UndoItem is added whenever a change is made, and the stack is modified when the undo and redo buttons are pressed. When the user saves a song, the compose activity converts its internal data structures into a Song, and saves it to file storage.

We planned to play back notes in other tracks while recording, à la overdubbing, so that users could more easily create harmonies when recording different tracks, but the microphone picked up the notes and it messed up the pitch detection. We considered something like disabling that feature unless headphones were plugged in, but concluded it wasn't worth it."Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface."

Evaluation

Our users were friends from our living groups who had varying levels of interest in music. While they're all MIT students and so might not be generally representative, MIT students seem to have a pretty wide range of musical inclination and so weren't a particularly bad sampling of our user population, though most of them were programmers (don't think any of them had programmed on Android before, though). Our app aims to be usable by people with varying levels of music experience and who are probably somewhat younger, creative and tech-savvy enough to carry a smart phone.

...