Overview of Application User Experiences' Metrics Development

Contents

The Metrics Framework table below organizes the current list of AUX metrics into a quasi-balanced scorecard format that is consistent across the areas in CSS.   Columns in the table are individual lines of business, usually headed by a Team Leader.  A second piece of the framework is a sub-section on metrics definitions and further documentation if needed about how they are produced. 

The actual values for these metrics are shown in a series of tables in the next section, where we present up to five quarters of data if it is available. 

.

Metrics Framework

In the table below, the left column marks off the "balanced scorecard" quadrants in our metrics framework.  Adjacent columns list by "line of business" the current roster of existing or potential metrics.  The metrics are in various states of development --
- bold face indicates "operational" metrics, with a defined means of production, and a historical record;
- normal face are for "possible" metrics, that aren't well established now but could be with a small development effort using existing systems;
- italic face  shows "prospective" metrics, desirable metrics that don't know have a path to becoming established.

New metrics ideas can be introduced at any time.  Continuous improvement efforts seek to move prospective metrics to possible, and possible to operational, within the normal operations of the teams involved.

Metrics Quadrant

Training

Accessibility & Usability

Documentation

Client Satisfaction or
Programmatic Outcomes

Satisfaction with Hands-On Training
("smile sheets")

Could be developed -- apparent Uptake of Review recommendations as demonstrated in
Change in Site Assessment Scores
(e.g., 60 before, eg., to 91 = explicit and effective intervention,
vs to 62 = natural drift from emerging cultural awareness).  

To be developed  

Resource Utilization
... of Team Services
(market penetration)
(utilization of the team
by MIT)
(uptake) (event demographics)

Seats Filled, total N 

Training Taken per MIT Organizatioal Area

Training Taken per Cohort

ATIC Lab Student Users, N of 


Accessibility Consults, N of 

Usability Reviews, N of 

To be developed

Resource Utilization
... of Tools and Resources

Classes Offered

Classes Offered per Topic Category

Adaptive Device Loans

Speech recognition software distributions

Pages Updated Per Topic Category

Process Performance

Average Seats Filled per Class

Training Room Rate of Utilization

Accessibility Reviews, avg duration or rate of completion (how long do they take?) 

To be developed

Finances

to be developed

no revenue expectations

no revenue expectations

Definitions and Means of Production

The AUX Metrics Definitions page holds a table listing metrics we have in some detail, and identifying measures we'd like to build. 

Five (or more) Quarters of Measures Presentation

{viewxls:name=AUX TRAINING metrics.xls|viewgrid=true}

Loading

Loading


Team Leaders upload fresh versions of their metrics documents here.

  File Modified
Microsoft Word 97 Document AUX ATIC metrics.doc Jul 06, 2010 15:22 by Robert W Smyser
Microsoft Word 97 Document AUX ATIC metrics fy2010 q3.doc Apr 08, 2010 15:49 by Robert W Smyser
Microsoft Word 97 Document AUX metrics.doc Jan 27, 2010 09:22 by Robert W Smyser
Microsoft Word 97 Document AUX training metrics.doc Jul 06, 2010 15:17 by Robert W Smyser
Microsoft Excel Sheet AUX TRAINING metrics.xls Jul 06, 2010 15:09 by Robert W Smyser
Microsoft Excel Sheet Copy of Training FY2010-2QMetrics-and-Reports (2).xls Jan 27, 2010 09:27 by Robert W Smyser
PDF File Training FY2010-2QMetrics-and-Reports (2).pdf Jan 13, 2010 14:05 by Robert W Smyser
PDF File Training-graphs.pdf Jan 13, 2010 15:10 by Robert W Smyser
PDF File Training-graphs-again.pdf Jan 13, 2010 14:16 by Robert W Smyser

These macros are experimental attempts to get the metrics, and *charts* that steve winig prefers to have,  to actually render on the wiki page.  These PDFs generate JIRA exception requests, and MIT's wiki-support has already escalated them to the confluence people.  So we wait. One solution to try is to render the Present page of the spreadsheet as a Bitmap in Word, and post that, just for presentation. {viewpdf:name=Training FY2010-2QMetrics-and-Reports (2).pdf}  save as PDF from inside Excel {viewpdf:name=Training-graphs.pdf} (PDF printed from Excel 2003 format {viewpdf:name=Training-graphs-again.pdf} (created by PDF printed again to a PDF) {viewxls:name=Copy of Training FY2010-2QMetrics-and-Reports (2).xls|sheet=Present|row=41|col=13}

  • No labels