Scenario
Joe is a student taking Japanese 2 at MIT. The class uses the Genki 1 textbook, and has studied the grammar, characters, and vocabulary up to Chapter 10. The textbook has taught Joe some useful grammar and vocabulary, but not enough to read most of the Japanese-language content on the web or in books. The textbook contains some example sentences for reading practice, but Joe has already read them and still doesn't feel like he's mastered the material. Joe want to practice reading additional sentences to solidify his understanding of the grammar, characters, and vocabulary that have been taught in Genki 1 up to Chapter 10.
Following his friend's suggestion, Joe heads to the Reading Practice website. Joe specifies to the website that he's studying Japanese from the Genki 1 textbook, and has studied up to Chapter 10. Then, the website provdes Joe with novel sentences in Japanese that use the grammar and vocab that he's learned, and he practices reading them.
After reading a few sentences, Joe encounters the sentence りんごを食べたいです。 He's unfortunately forgotten what the vocabulary word りんご meant, but he doesn't want to look it up in a dictionary, so he uses the interface to determine that it means "apple". Joe then see 食, and while he knows it has something to do with eating, he doesn't know how it should be pronounced in this particular context. He therefore asks the system to romanize the sentence, and seeing "Ringo wo tabetai desu", he now knows that 食 is pronounced "ta" in this context. However, he still doesn't understand what the sentence means, hence he asks the system to show him an English translation. Seeing the translation "I want to eat an apple", it dawns on him that he's forgotten that the -tai verb conjugation is used to indicate want/desire. He wants to practice reading this sentence pattern more, hence he asks the system to show him additional sentences that use the -tai verb conjugation.
After reading a few more sentences that use the -tai verb conjugation from the list, Joe notices a sentence that isn't classified correctly. He had been searching for sentences using the -tai verb conjugation, but this sentence doesn't include it. He therefore removes the "-tai verb conjugation" tag on the sentence, so that it won't be displayed to users searching for that type of sentence.
A few years later, Joe is now fairly fluent in Japanese. He'd like to contribute back to the website that helped him practice reading Japanese back when he was starting to study Japanese. Hence, he logs in and enters a new sentence. He types in, ビールを飲みませんか。 and the English translation, "Would you like to drink beer?". The system helpfully uses some heuristics to suggest that the sentence be tagged as having a "-masen verb conjugation", and Joe additionally adds the "making an offer" tag. Thereafter, students looking for these types of sentences, either because they're explicitly searching for sentences with that type of tag, or the textbook chapter they're studying has covered the usage of a sentence type using that tag, will see Joe's newly added sentence.
Designs
Design 1
This design focuses on minimizing the amount of time needed from when the user enters the website, and can start reading sentences. For this reason, it lacks registration, and requires the user to select only a language before getting to the main interface where sentences are displayed. The interface is a single list of sentences, each of which have an associated accordian menu for displaying romanization, English translation, and information about the individual vocabulary words and grammatical patterns used in the sentence. The user doesn't ever leave this list of sentences; finding additional examples of sentences using certain vocabulary words or grammatical patterns is accomplished by having the list re-order itself such that the sentences of interest are injected below the reader's current position. Likewise, if the user doesn't want to see sentences that use a certain grammatical pattern, they are moved down to the end of the list.
Storyboard
The user enters the website, and is asked to select a language. In Joe's case, he selects Japanese from the drop-down menu, and clicks the "go" button. Now the user is brought to a page with a list of all sentences in the database for that language (which can be scrolled through and will be dynamically loaded as needed, a la Google Images). At first, sentences using all grammatical structures are shown. If the user is studying from a textbook, he can optionally select the textbook and chapter he is studying from - Joe would thus select Genki I, and chapter 10. Once the textbook and chapter is selected, the list of sentences is filtered to those that use the grammar and vocabulary that has been covered up to that point in the textbook. The user can now start reading the sentences, scrolling down to find new sentences. The sentences are initially ordered such that those sentences that use the vocab and grammar that has been covered most recently in the textbook (which is likely of the most interest to the reader) is displayed first, and sentences are ordered such that vocabulary and grammatical structures vary from sentence to sentence (so, for example, if りんご occurred in the first sentence, then it shouldn't be seen again until a few sentences later). In the event that the user didn't select a textbook, sentences are ordered randomly.
Eventually, while reading, the user might find a sentence that he has trouble understanding, either because it contains some vocabulary he doesn't remember, or uses some grammatical construction he doesn't understand. In the event that it's some particular word that the user doesn't understand (in our scenario with Joe, りんご), he can expand out the "vocabulary" accordian menu, where he will find all vocab words used in the sentence, their romanizations (ringo), and their definitions . Alternatively, Joe can hover over the word りんご, at which point the mouse cursor will change shape to indicate a clickability affordance, and then Joe will click the word. This will cause the Vocab accordion menu to be expanded, and りんご and the romanization (ringo) and definition (apple) will become highlighted. There is also a Grammar accordian menu, which can be expanded and will list the grammatical constructs that this sentence uses, and again, there are "prioritize" and "deprioritize" buttons next to these.
The user would click prioritize on a vocab or grammar item if he wants to see more sentences of that sort (because he wants to practice reading more sentences that use that particular vocab or grammar), and deprioritize if he doesn't want to see sentences that use that vocab item or grammatical pattern. If the user clicks on prioritize for some vocab word or grammatical pattern, then among the sentences later in the list (which haven't yet been read), the ones that use the vocab or grammar that the user wants to prioritize, will be moved up and injected immediately below the sentence that the user just read (the sentences associated with the accordian menu from which the "prioritize"/deprioritize button was clicked). That way, the next sentences that the user reads will all use the particular grammatical pattern that was prioritized. If the user clicks on deprioritize, then later sentences using that grammatical pattern will be moved to the end of the list, so that the user will see those only after having read all other sentences.
If Joe wants to see the entire sentence romanized, he can expand the "Show romanization" accordian menu. Likewise, if Joe wants to have the English translation of a sentence displayed, he can expand the "Show English translation" accordian menu. If Joe does not understand some grammatical aspect of the sentence, or simply wants more practice with a grammatical form (in the scenario, the -tai verb conjugation), he can expand the Grammar accordian menu. Here, he will find the listing of grammatical aspects that are used in this sentence, and he can press "prioritize" on "-tai verb conjugation" so that the sentences using that grammar get moved up to where he is reading at, so that the next sentences he reads utilize the grammatical pattern.
If Joe encounters a sentence whose grammar patterns listed are incorrect, then he can expand the accordian menu, and click the "edit" button. Here, he can remove any categories that are incorrect (or add any that need to be added), and click "done editing" to save changes.
If Joe wants to contribute a sentence, he goes to the home page, and switches to the "contribute" tab. Here, he selects the language of the sentence he'll contribute from a pull-down menu, and enters the sentence itself and its translation in a textbox. Then, he selects any additional grammatical tags that would be used to help users find his sentence, clicks on "add" to add them, then clicks on "contribute sentece" to contribute the sentence. Now, any user who subsequently clicks the "more sentences" option on that type of grammatical tag will have Joe's sentence displayed to them among the list. (The usage of tags for representing the grammatical patterns found in a particular sentence is used to simplify the backend, to avoid needing to use NLP to determine the grammatical structures found in sentences. Locating vocabulary in a sentence should be considerably easier from a backend standpoint, and hence sentence contributers don't need to tag sentences with the vocabulary they contain, only the grammatical patterns).
Usability Analysis
Learnability
The lack of needing to perform any registration or select options other than the language before being able to read sentences enhances the ease of use. Additionally, the interface for browsing sentences is simply a list of sentences, hence the user doesn't need to know what he's looking for (ie no need to know the name of the particular grammatical structure) to find reading material.
The prioritize/deprioritize model for filtering sentences is also more easily learnable than something like an explicit search for sentences containing particular words, as the user doesn't need to be able to recall, the vocabulary words and sentence patterns that he wants to review - he just needs to be able to recognize them when they come up in the list of sentences.
The per-sentence information gathering process makes heavy use of accordian menus. These are becoming fairly common in web apps, and hence learnability with their usage should not be an issue. The usage of the clicking on a part of a sentence to expand out the accordian menu for that vocab item, however, is somewhat inconsistent with the users' experience of how accordian menus work. Namely, users may not expect accordian menus to expand by themselves if the user clicks elsewhere. Additionally, the highlighting of an item in an accordion menu may not be noticed by the user, because it occurs far from where the focus of the user's attention (the mouse) is. However, this is a tradeoff for internal consistency, as it ensures that no separate popup or hover-over information dialog is needed if the user wants information about some particular word in the sentence - all supplemental information for a given sentence is displayed in its accordian menu.
Another learnability issue is that it may be difficult for a sentence contributer to know what each of the grammatical tags mean, particularly because differing textbooks may describe different sentence patterns using different terminology.
Efficiency
This interface is not efficient for users who already know what particular type of vocabulary or grammatical pattern they want to practice, as they must first find an example of a sentence using that vocab or grammatical pattern before they can click on the "prioritize" button to bring up additional sentences using that particular vocab or grammar. However, because of the choice to order sentences such that the ones using grammar and vocabulary introduced in recent chapters (which is likely what the user wants to review) come first, then finding such a sentence will not take long in the case when the user is practicing material introduced in recent chapters (which is likely the most common use case).
Another issue with efficiency on the initial screen is that a selector is used to locate the textbook, which, while necessary if the user is able to recognize but not recall their textbook name, may be less efficient than typing a name with autocompletion for users who already knows the name of their textbook .
The choice of a selector box for adding grammatical tags to sentences, while easily learnable because all of the grammatical tag types are visible, may be inefficient for users who already know the category the word belongs to. While the grammatical could be sorted alphabetically, a user might know the particular grammatical pattern under a different name from the terminology that is used in this website. Hence, it may make sense to also allow a search box with autocomplete to be used for searching for particular tag types. Optimally, users shouldn't even have to input what types of grammatical structures in a sentence, but this would unfortunately involve complex NLP work for detecting grammatical patterns in sentences.
Visibility
This design's state is encapsulated in the ordering of the list of sentences, which is always visible to the user. While information about the sentence's vocabulary, grammar, romanization, and translation are hidden in the accordion menu by default, this is by design, because it is meant to be relied on only if the user has trouble reading the sentence without assistance.
Error Prevention
One particularly problematic error a user might encounter is if he accidentally presses "deprioritize" on a vocabulary item or grammatical structure that he actually did want to review. What happens in this case is that sentences containing it will go to the very end of the list, and he won't see them until then. Thus, such an error is not easily undo-able (the user would need to go to the end of the list, and press "prioritize" on the item again, but in the process would lose his reading position). This is an example where design 3, which uses popups to display supplemental materials (and can thus just be closed if accidentally opened), has an advantage over this one.
The editing functions - namely, adding and removing grammatical tags from sentences, and changing translations for sentences, are unfortunately not undoable. For this reason, because this model functions much like a wiki - in that any user is able to edit or contribute any sentence - it may be necessary to introduce some form of history into the interface should accidental editing or vandalism become a problem.
Design 2
Storyboard
This design provides the student user with a ‘home’ view (Figure 2) of all the textbooks that he is learning from. A user can add and search textbooks from any page by using the ‘Add/Search’ bar labeled in Figure 2. The user can either search for a title (Figure 4), or see an alphabetical sorting of all the textbooks in the system (Figure 3). Also, there are language filters along the right to allow the user to narrow his search. After the student has added the book to his textbooks, he can return to his ‘home’ via the ‘My Home’ link that is located at the top of every page and then pick a book from his selections.
When he picks a book, the page updates with a chapter listing on the left-hand side and a translator on the right (Figure 5). The translator accepts a word as an input and translates the word either ‘From English’ or ‘To English’. The information displayed depends on the language of the textbook. For example, Chinese words might include information on tones, pinyin representation, character representation, traditional and simplified forms, etc. Japanese words might include information on the different text representations (eg. Kanji, Kana, Romaji) of the word.
When the user selects a chapter or section from the outline of chapters, the page loads all the vocabulary words learned in that chapter or section (Figure 6). The vocabulary words are shown in an accordion-style format. The user can sort the words by one of the fields at the top of the accordion listing. When the user expands one of the folds in the accordion, practice sentences using the word are listed. If the user has privileges, there will be text boxes and a button used for adding new sentences.
Other features demonstrated in Figure 6 include the ‘Look Up’ feature that lets the user filter the accordion-style listing of the words in the current chapter. If the user wants to practice the vocabulary in this chapter, he clicks the ‘Practice’ button, and the system will switch to a flashcard like interface (Figure 7). The user can mark a card as correct or wrong to indicate whether he has mastered the sentence. Additional options are ‘Previous’, ‘Show Answer’, ‘More Like This’, and ‘Next’. The ‘Previous’ option lets a user backtrack to the previous word that he practiced.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Usability Analysis
Learnability
Pros:
The user interface has many affordances. A plus sign is used for adding a book (Figure 3 shows this) to your collection, a minus sign for removing (Figure 2), and a ‘return’ arrow for restoring a removed book (Figure 2).
There is an omnipresent ‘My Home’ button at the top of every page that allows a user who is exploring the website to easily return to his ‘home’ view. This enables a user to explore the site knowing that he can easily return ‘home’ if he gets lost.
The Logout button is consistent with most interfaces that support logging in and out. The logout button is located in the common logout link location at the top right of the page.
The ‘Add/Search’ bar has an auto-complete feature to assist users in entering a title. This helps users who cannot remember a title completely.
Cons:
The procedure to add a new sentence is not very obvious to the user. In order to add a sentence, the user must list all the words (Figure 6; he may also use the look up feature) and select the vocabulary word for which he wants to add a sentence. This feature is visible only to users who have privileges, so novices cannot learn this feature until after they have mastered a part of the language and have acquired privileges.
Visibility
Pros:
The state of the system is visible to the user. In Figure 7, the current title is displayed in large text alongside a picture of the book’s cover to remind the user what book he is currently looking in. Along the left hand side of the interface, the selected chapter is highlight and boxed to indicate which chapter is being studied.
Cons:
The visibility of a user’s mastery of each word or section is not displayed unless the user explicitly lists all the words (Figure 6). Thus, an unaware user might feel that he needs to keep practicing sentences from a particular chapter even though he might have a high mastery rating for all the vocabulary in that chapter.
Efficiency
Pros:
The following items are in the same location on every page when logged in: ‘My Home’ button, ‘Add/Search’ bar, Logout button. Thus performing these tasks is easy from every page.
Cons:
Performing a search for a book can take a long time. Although there is a language filter and alphabetical sorting of the books, the user must still linearly scan through the items. If there are many books, the filter and alphabetical sorting are not enough.
Searching for a word in a chapter (Figure 6) can be tedious and time-expensive task if there are many words in the chapter. The interface attempts to counter this negative effect by providing a word look-up search bar to narrow the search.
Error prevention
Pros:
A user may make changes to his collection of textbooks including removal of unused textbooks and addition of new ones. If a user accidentally removes a textbook from his ‘home’ view, he has the option of returning it to his collections. That is, permanent removal is a delayed process (shown in Figure 2 by the ‘Time Until Deletion’).
The ‘Log Out’ button is located in a corner away from other features of the interface. This prevents the user from accidentally logging out.
Cons:
If a user has many textbooks, the removed textbooks are pushed down and out of the user’s locus of attention (Figure 2). A user who is unaware of a removed textbook might never learn of the book’s removal until after its deletion because the removed textbooks are located out of sight.
Design 3:
Key Concepts: Every new state (any new page brought on by a button/link click) will be displayed in a new window (pop-up window)
Page 1:
This is the homepage that greets users coming on to the site. The page will have 2 options: for old users to log in with username and password, or for new visitors to sign up by entering a username, email, and password (with a password confirmation). The button at the right of the confirmation password textbox will be labeled "register." If the username is already in use, the page will prompt the visitor to try a different username. After an account has been created, the visitor will have to log in using the normal log in. 
Page 2:
The homepage will enter into "page 2" after visitor successfully logs in. The user now has two options. Clicking the "Language QuickGo" button will lead the user to "page 4." The user can also click the "textbook go" button to enter "page 3." The "Language QuickGo" button is only available for clicking after the user enters the language in the language drop-down list. The "textbook go" button is only available after the user selects the language, textbook name, and chapter. Note, this will be almost identical in format to the MIT coop textbook selection.
Page 3:
The user will arrive upon this page after filling in the appropriate selections and clicking the "textbook go" button. This page will contain a series of boxes, each containing a single practice sentence provided by the textbook publisher as supplementary material for that chapter. Each sentence box will contain a button labeled "?" at the bottom left corner. Clicking on this button will cause a page to pop up. The page will contain three headings: tags, romanized, and english translation. Below the tags heading will be a series of hyperlinked tags pertaining to each work in the sentence. Below the romanized heading will be a phonetic description of the sentence (allowing the user to pronounce the sentence). Finally, below the english translation heading will be the english translation of the sentence. Clicking any of the hyperlinked tags under the Tags heading will take the user to "page 5." The "cancel" button closes the pop-up page and returns the user to the original "page 3" containing the list of supplementary sentences.
Page 4:
The user can arrive upon this page after clicking the "language go" button instead of the "textbook go" button. This page serves two function. The first function is to allow the user to find example sentences after entering certain tags in the textbox with the heading "Tag Search" and hitting the "Go" button. This will take the user to "page 5." The second function of this page is to allow experienced foreign language speakers to add sentences to the database. The user must type in a sentence in the preselected foreign language, enter a list of words to tag the sentence in the textbox labeled "Tags," and finally enter the same sentence in English in the textbox labeled "English Translation." The user can then hit the "Add" button to add the sentence. The "Add" button only becomes available after all three textboxes are completed. Hitting "Add" will clear the textboxes (information scent for user to know the system has added the sentence) and add the sentence/sentence info to the database.
Page 5:
This page can be arrived on by the user in two ways. The first way is when the user clicks on a hyperlinked tag. The second way is by the user entering tags in the tag search box on "page 4." There are several notable components to this page. First, the Tag: label will contain the tag(s) that were used to take the user to the current page. Next, definitions (translations) will be provided for the Tag in the "Def" field. Below will be a scrollable list of example sentences associated with the tag. Each sentence will have an "edit" button at the end. Note that these user-provided sentences differ from the textbook-provided sentences on page 3 because the tags can only be added/removed in user-provided sentences. Clicking the "edit" button will bring up a pop-up page with three notable components and two buttons. The first component is the Tag List, which has an editable text box containing all the tags associated with the sentence. The user can add and delete tags in the textbox and click the "update" button to confirm those changes. The second component is labeled by the heading "English Translation" and is a translation of the sentence. Finally, the third component is the Tags (hyperlink) component allowing the user to go to more "page 5"-type pages. Clicking the "update" button will also update the hyperlinks available in the Tags field (the third component of the pop-up page). Clicking the cancel button will close the pop-up page. 
Learnability
Pros:
The interface is very simple. The existence of cancel buttons give users a more clear understanding of how to exist out of a pop-up page. The existence of pop-up windows allow the user to switch views easily via the interface provided by the user's OS for switching among windows (something user should be used to and familiar with). Certain buttons are only clickable when the necessary fields are completed, enforcing certain actions.
Cons:
A lot of button presses.
Visibility
Pros:
Everything is clearly labeled. The textboxes, drop-down selection lists, scrollbars, and buttons all provide very visible affordances to the users. Buttons that should not be clicked will be grayed out and unclickable.
Cons:
Not all information is directly visible, requires button click. Need to ensure button labels provide sufficient information scent (formative evaluation).
Efficiency
Pros:
The simplicity of the design allow users to need only skill-based actions. Pop-up pages save users from having to click back/home buttons to navigate to previous pages. All sentences have a list of hyperlinked tags allowing users to easily obtain more information from sentences without having to open search boxes or type in search words.
Cons:
Lack of breadcrumbs prevents users from tracing their series of actions that led to the creation of the pop-up pages.
Error
Pros:
Pop-up window utilization allows users to explore new pages and features without risk of losing previous pages. No log-out problem, users simply close the window. The interface does a great job of separating states/modes as new pop-up windows.
Cons:
Allowing users to freely modify the tag list of user-contributed example sentences can be dangerous. Does not have a safety for when users accidentally modify and update the tag list field of user contributed example sentences.
Differences:
Overall Interface:
1. Design 1: This design consists of a single and simple standalone page. The vocabulary and grammar that the user wants to read is represented by the order of the sentences in the list. Vocabulary and grammar included in the sentences directly below the currently being read sentence are the ones that the user wants to practice reading.
2. Design 2: This design provides the user an organization of all his reading material, which aids in navigation to his desired task. For example, the user has his own textbooks, which are divided into ‘Removed’ and ‘Not Removed’. Navigation is performed via searches and quick look-ups, and available actions are controlled by what state the user interface is in (eg. using the practice cards, selecting a chapter).
3. Design 3: This design provides the user with an interface that hides complexity. A common theme is that extra information is displayed only upon actions (such as clicks) by the user (e.g. if a user wants to reveal additional information about a sentence, he can click the question mark button to open a new window, this window in effect encapsulates the extra information so that is easy for the user to hide/show).
Finding Reading Material:
Design 1: The user does not explicitly search for particular vocab or grammatical items. Rather, the user is displayed a list of sentences, and he uses prioritize and de-prioritize operations on vocabulary and grammar items to reorder the reading list to make grammar and vocab he wants to practice appear below where he is currently reading, and to exclude vocab and grammar he does not want to read.
Design 2: The user has the option of either initiating a search for vocabulary, or manually going through the list of words in a chapter or filtered look-up. Similarly, to find a book, the user can do a search or search via full listings. Users also have saved books.
Design 3: The user fills out a form (textbook, chapter, tags) to identify what types of reading he wants. Also available is a search option that can be used to search specific tags.
Reading Material:
Design 1: Information about sentences are arranged in accordion menus.
Design 2: The user can read from a full or filtered listing of words, or read words via flashcards, one at a time, which is the intended way of practicing.
Design 3: Readings are listed in reading blocks. Each block has one sentence and an options button, which provides more info on the sentence in a new window. So reading sentences and extra information is separated.
Contributing Reading Material:
Design 1: Sentences are added via a tagging system.
Design 2: To add words, the user must be look up the word in the chapter (either by list view or search view), then make an entry for the specific vocab.
Design 3: Adding reading material occurs in its own part of the quick search screen. The user fills out a form and contributes the sentence.