| Wiki Markup |
|---|
h2. Overview of Application User Experiences' Metrics Development |
...
Contents
The Metrics Framework table below organizes the current list of AUX metrics into a quasi-balanced scorecard format that is consistent across the areas in CSS. Columns in the table are individual lines of business, usually headed by a Team Leader. A second piece of the framework is a sub-section on metrics definitions and further documentation if needed about how they are produced.
The actual values for these metrics are shown in a series of tables in the next section, where we present up to five quarters of data if it is available.
.
Metrics Framework
In the table below, the left column marks off the "balanced scorecard" quadrants in our metrics framework. Adjacent columns list by "line of business" the current roster of existing or potential metrics. The metrics are in various states of development --
- bold face indicates "operational" metrics, with a defined means of production, and a historical record;
- normal face are for "possible" metrics, that aren't well established now but could be with a small development effort using existing systems;
- italic face shows "prospective" metrics, desirable metrics that don't know have a path to becoming established.
New metrics ideas can be introduced at any time. Continuous improvement efforts seek to move prospective metrics to possible, and possible to operational, within the normal operations of the teams involved.
...
Metrics Quadrant
...
Training
...
Accessibility & Usability
...
Documentation
...
Client Satisfaction or
Programmatic Outcomes
...
Satisfaction with Hands-On Training
("smile sheets")
...
Could be developed -- apparent Uptake of Review recommendations as demonstrated in
Change in Site Assessment Scores
(e.g., 60 before, eg., to 91 = explicit and effective intervention,
vs to 62 = natural drift from emerging cultural awareness).
...
To be developed
...
Resource Utilization
... of Team Services
(market penetration)
(utilization of the team
by MIT)
(uptake) (event demographics)
...
To be developed
...
Resource Utilization
... of Tools and Resources
...
Pages Updated Per Topic Category
...
Process Performance
...
Accessibility Reviews, avg duration or rate of completion (how long do they take?)
...
To be developed
...
Finances
...
to be developed
...
no revenue expectations
...
no revenue expectations
Definitions and Means of Production
The AUX Metrics Definitions page holds a table listing metrics we have in some detail, and identifying measures we'd like to build.
Five (or more) Quarters of Measures Presentation
...
| comment |
|---|
h3. Contents The Metrics Framework table below organizes the current list of AUX metrics into a quasi-balanced scorecard format that is consistent across the areas in CSS. Columns in the table are individual lines of business, usually headed by a Team Leader. A second piece of the framework is a sub-section on metrics definitions and further documentation if needed about how they are produced. The actual values for these metrics are shown in a series of tables in the next section, where we present up to five quarters of data if it is available. . h3. Metrics Framework In the table below, the left column marks off the "balanced scorecard" quadrants in our metrics framework. Adjacent columns list by "line of business" the current roster of existing or potential metrics. The metrics are in various states of development \-\- *\- bold face* indicates "operational" metrics, with a defined means of production, and a historical record; \- normal face are for "possible" metrics, that aren't well established now but could be with a small development effort using existing systems; _\- italic face_ shows "prospective" metrics, desirable metrics that don't know have a path to becoming established. New metrics ideas can be introduced at any time. Continuous improvement efforts seek to move prospective metrics to possible, and possible to operational, within the normal operations of the teams involved. || Metrics Quadrant || Training \\ || Accessibility & Usability \\ || Documentation \\ || | Client Satisfaction or \\ Programmatic Outcomes \\ | *Satisfaction with Hands-On Training* \\ ("smile sheets") \\ | _Could be developed_ \-\- _apparent Uptake of Review recommendations as demonstrated in_ \\ _Change in Site Assessment Scores_ \\ _(e.g., 60 before, eg., to 91 = explicit and effective intervention,_ \\ _vs to 62 = natural drift from emerging cultural awareness). _ | _To be developed_ | | Resource Utilization \\ ... of Team Services \\ (market penetration) \\ (utilization of the team \\ by MIT) \\ (uptake) (event demographics) \\ | *Seats Filled, total N* \\ \\ *Training Taken per MIT Organizatioal Area* \\ \\ *Training Taken per Cohort* \\ | *ATIC Lab Student Users, N of* \\ \\ \\ *Accessibility Consults, N of* \\ \\ *Usability Reviews, N of* \\ | _To be developed_ | | Resource Utilization \\ ... of Tools and Resources \\ | *Classes Offered* \\ \\ *Classes Offered per Topic Category* \\ \\ | *Adaptive Device Loans* \\ \\ _Speech recognition software distributions_ \\ | *Pages Updated Per Topic Category* | | Process Performance \\ | *Average Seats Filled per Class* \\ \\ *Training Room Rate of Utilization* \\ | _Accessibility Reviews, avg duration or rate of completion (how long do they take?)_ \\ | _To be developed_ | | Finances \\ | _to be developed_\\ | _no revenue expectations_ | _no revenue expectations_ \\ | h3. Definitions and Means of Production The [AUX Metrics Definitions|https://wikis.mit.edu/confluence/display/CSS/AUX+Metrics+Definitions] page holds a table listing metrics we have in some detail, and identifying measures we'd like to build. h4. h2. Five (or more) Quarters of Measures Presentation ---- {anchor:AUX-metrics} {comment}{viewxls:name=AUX TRAINING metrics.xls|viewgrid=true} {comment} {viewdoc |
...
:name |
...
=AUX training metrics.doc |
...
} {viewdoc |
...
:name |
...
=AUX ATIC metrics.doc |
...
| Panel |
|---|
Team Leaders upload fresh versions of their metrics documents here. |
| Attachments | ||||
|---|---|---|---|---|
|
...
} \\ ---- {panel} Team Leaders upload fresh versions of their metrics documents here. {panel} {attachments:old=true|upload=true} ---- h3. {comment} These macros are experimental attempts to get the metrics, and *charts* that steve winig prefers to have, to actually render on the wiki page. These PDFs generate JIRA exception requests, and MIT's wiki-support has already escalated them to the confluence people. So we wait. One solution to try is to render the Present page of the spreadsheet as a Bitmap in Word, and post that, just for presentation. {viewpdf:name=Training FY2010-2QMetrics-and-Reports (2).pdf} save as PDF from inside Excel {viewpdf:name=Training-graphs.pdf} (PDF printed from Excel 2003 format {viewpdf:name=Training-graphs-again.pdf} (created by PDF printed again to a PDF) {viewxls:name=Copy of Training FY2010-2QMetrics-and-Reports (2).xls|sheet=Present|row=41|col=13} {comment} |