MeasuresBusinessProcesses

As measures roll out from the central groups that use them daily and naturally, and to support the collection of measures as a collective activity, we need to record where the data underlying our measures will come from and who will collect it. Ultimately management will need to sign off on committing time to the effort, or to the creation of more automated means, and to do so across broader swaths of IS&T as time goes on.

Overall Architecture

Sustaining the ISTMeasuresPortfolio "takes a village" of individuals committed to data providers and collectors. The Measures Portfolio uses a distributed architecture based on common technologies to minimized the effort and overhead needed for its upkeep.

Posting Updates

IS&T has a natural cycle of reporting aligned with quarters of the fiscal year. The usual schedule is to publish new summaries on the first Friday of the month following the publishing of the Quarterly Reports. The idea is that all data will already have been collected for use in those reports, and collecting them into the measures portfolio represents no additional work.

• Q1 -- October
  Q2 -- January
  Q3 -- April
  Q4 -- July

IS&T's smallest useful time period for most measures is monthly. The measures portfolio is designed to accept monthly data and summarize it for reporting -- quarterly averages, medians, trends, upper and lower ranges, etc., can all be computed automatically. Reporting monthly imposes a very small incremental burden on the data collectors, ideally based on data collection already in place for monthly management controls like dashboards and scorecards.

Who Collects It All?

Since at least the beginning of fy2005, the IS&T Directorates have included measures in their Quarterly Reports. Theseare tabulated by come combination of staff within the directorate and channeled to the Director, who authors their section of the report. The QR is aggregated by staff in the VP's office.

The staff involved below the Director level are the natural community of interest that we call upon to sustain the Measures Portfolio.

Our goal is to take the Directors out of the direct path of data collection or aggregation. IS&T needs a more standardized system of monthly reports, dashboards, etc. produced as automatically as possible from the core systems. Directors and perhaps managers need to consume this data but have more valuable ways to spend their time than producing the reports.

Archiving Measures Data -- A layered Approach

We rely on a hierarchy of afs directories to support distributed data collection. The home node is the measures locker in afs: //afs/athena.mit.edu/project/measures (which we'll call "./measures" from now on).

A home page in that directory will be the first page of our PublicFacingMeasures presentation. It is not yet live, though a draft mockup lives at http://web.mit.edu/measures/draft-index.html

Each directorate has its own branch in the directory tree. For example,

• ./measures/css
  ./measures/ois
  ./measures/tss
  ./measures/ad
  ./measures/ssit

Each directorate is welcome to store data in their own branch however they see fit. Individuals can update data by hand in a compendium-of-measures.xls file. Automated systems can write log reports into directories in the hierarchy via afs. Data can be contributed in as raw a form as makes sense, decided on an individual basis. These raw data files are rolled up into progressively more refined presentation form by spreadsheets or other tools selected by the directorate, such as the CSS Dashboards, Scorecard or Key Indicators files.

Ultimately, data for selected measures has to be collected for reporting on behalf of IS&T. A spreadsheet in ./measures is configured to update itself with data from the directorate sources, producing a web page presentation with values, time series, trend analysis and graphs. Individual values from this spreadsheet are currently hand-copied into the home page of the measures locker. (With a little technical wizardry, we could probably have the individual values automatically <#include>'d into the home page, eliminating that last manual step.)

This is a layered approach in that directorates can and will collect many more measures than will be reported directly to the public. Some measures will be selected for bringing forward to IS&T as Key Indicators for those lines of business. And only some of those will be selected as public-facing measures.

Updating the Portfolio -- Who Decides and How Often?

The quarterly review cycle provides the natural rhythm for amendments to the Measures Portfolio. IS&T should convene a community of interest with oversight of the Measures Portfolio. Membership will be drawn at first from the Data Collectors column of the table above. They would review the data with the directors, and decide what material deserves new visibility in response to renewed investment by the organization. To do its work, members of this ist-measures team would:

• - meet with their director's management team in the first week of the 3rd month of the quarter to review the measures as reported the previous quarter.
  - then meet as a group in the second week to review requested additions, deletions, etc., and propose the necessary changes to the public-facing-measures and other files.
  - share the proposed changes in the third week with the directors and managers.
  - implement the final version of the reports and tool in the fourth week.
  - begin to capture data in the first week of the new quarter.