Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin

Overview of Application User Experiences' Metrics Development

Contents

The Metrics Framework table below organizes the current list of AUX metrics into a quasi-balanced scorecard format that is consistent across the areas in CSS.   Columns in the table are individual lines of business, usually headed by a Team Leader.  A second piece of the framework is a sub-section on metrics definitions and further documentation if needed about how they are produced. 

The actual values for these metrics are shown in a series of tables in the next section, where we present up to five quarters of data if it is available. 

.

Metrics Framework

In the table below, the left column marks off the "balanced scorecard" quadrants in our metrics framework.  Adjacent columns list by "line of business" the current roster of existing or potential metrics.  The metrics are in various states of development --
- bold face indicates "operational" metrics, with a defined means of production, and a historical record;
- normal face are for "possible" metrics, that aren't well established now but could be with a small development effort using existing systems;
- italic face  shows "prospective" metrics, desirable metrics that don't know have a path to becoming established.

New metrics ideas can be introduced at any time.  Continuous improvement efforts seek to move prospective metrics to possible, and possible to operational, within the normal operations of the teams involved.

Metrics Quadrant

Training

Accessibility & Usability

Documentation

Client Satisfaction or
Programmatic Outcomes

Satisfaction with Hands-On Training
("smile sheets")

Could be developed -- apparent Uptake of Review recommendations as demonstrated in
Change in Site Assessment Scores
(e.g., 60 before, eg., to 91 = explicit and effective intervention,
vs to 62 = natural drift from emerging cultural awareness).  

To be developed  

Resource Utilization
... of Team Services
(market penetration)
(utilization of the team
by MIT)
(uptake) (event demographics)

Seats Filled, total N 

Training Taken per MIT Organizatioal Area

Training Taken per Cohort

ATIC Lab Student Users, N of 


Accessibility Consults, N of 

Usability Reviews, N of 

To be developed

Resource Utilization
... of Tools and Resources

Classes Offered

Classes Offered per Topic Category

Adaptive Device Loans

Speech recognition software distributions

Pages Updated Per Topic Category

Process Performance

Average Seats Filled per Class

Training Room Rate of Utilization

Accessibility Reviews, avg duration or rate of completion (how long do they take?) 

To be developed

Finances

to be developed

no revenue expectations

no revenue expectations

Definitions and Means of Production

The AUX Metrics Definitions page holds a table listing metrics we have in some detail, and identifying measures we'd like to build. 

Five (or more) Quarters of Measures Presentation


Anchor
AUX-metrics
AUX-metrics

comment
{viewxls:name=AUX TRAINING metrics.xls|viewgrid=true}
Office Word
nameAUX training metrics.doc
Office Word
nameAUX ATIC metrics.doc



Panel

Team Leaders upload fresh versions of their metrics documents here.

Attachments
uploadtrue
oldtrue

comment
These macros are experimental attempts to get the metrics, and *charts* that steve winig prefers to have,  to actually render on the wiki page.  These PDFs generate JIRA exception requests, and MIT's wiki-support has already escalated them to the confluence people.  So we wait.

One solution to try is to render the Present page of the spreadsheet as a Bitmap in Word, and post that, just for presentation.







{viewpdf:name=Training FY2010-2QMetrics-and-Reports (2).pdf}  save as PDF from inside Excel

{viewpdf:name=Training-graphs.pdf} (PDF printed from Excel 2003 format
{viewpdf:name=Training-graphs-again.pdf} (created by PDF printed again to a PDF)

{viewxls:name=Copy of Training FY2010-2QMetrics-and-Reports (2).xls|sheet=Present|row=41|col=13}