What should we measure?
All applications should have the following metrics in common:
Number of Users
- by community (department, MIT affiliation, etc)
- total unique users
- number of user accounts
- number of user sessions/logins
- by time period (peak usage, average usage)
System Utilization
- CPU
- memory
- disk space
- network
Application Specifics (metrics here vary from applications to application)
- wikis (automated)
- DW (automated)
- Roles (automated)
- Jira
- Jabber
- RT
- Web Servers (Apache, etc)
- Web Services (UAWS)
- Thalia (manual)
- Stellar (manual)
- Touchstone unique users by app and logins by app (should merge with DW to report by community)
- Developer Tools
- Kerberos (via RT, bug fixes, commits to tree)
How do we measure them?
Getting information into the Data Warehouse
- in a standard format
- except system utilization
SAMM would provide a continuous view of these stats
- show trends
- how app behaves (learn more about environment)
- blending of stats (users, memory, spikes)
What do we have right now?
Number of Users |
users by community |
total unique users |
user accounts |
logins |
time period |
|---|---|---|---|---|---|
wikis |
DLC |
Y |
Y |
|
Y |
jira |
|
|
|
|
|
jabber |
|
|
|
|
|
request tracker |
|
|
|
|
|
data warehouse |
DLC |
Y |
Y |
|
Y |
roles |
DLC |
Y |
|
|
|
web servers |
|
|
|
|
|
web services |
|
|
|
|
|
thalia |
? |
Y |
Y |
|
|
stellar |
by subject, role |
Y |
Y |
Y |
Y |
touchstone |
|
|
|
|
|
developer tools |
|
|
|
|
|
kerberos |
|
|
|
|
|
System Utilization |
CPU |
memory |
disk space |
network |
|---|---|---|---|---|
wikis |
|
|
|
|
jira |
|
|
|
|
jabber |
|
|
|
|
request tracker |
|
|
|
|
data warehouse |
|
|
|
|
roles |
|
|
|
|
web servers |
|
|
|
|
web services |
|
|
|
|
thalia |
|
|
|
|
stellar |
|
|
~1 terabyte |
|
touchstone |
|
|
|
|
developer tools |
|
|
|
|
kerberos |
|
|
|
|
Application Specifics |
|
|
|
|
|
|---|---|---|---|---|---|
wikis |
# of transactions (edits, comments) |
# of pages |
# of authors |
# of sites |
# of views |
jira |
|
|
|
|
|
jabber |
|
|
|
|
|
request tracker |
|
|
|
|
|
data warehouse |
|
|
|
|
|
roles |
# of transactions (creations, deletions and updates) |
|
|
|
|
web servers |
|
|
|
|
|
web services |
|
|
|
|
|
thalia |
# of images |
# of domains |
storage per domain |
|
|
stellar |
# of courses |
|
|
10,000+ |
|
touchstone |
# of access requests (by application) |
method (username, certs or kerb) |
|
|
? |
developer tools |
|
|
|
|
|
kerberos |
RT bug fixes |
commits to tree |
|
|
|
What requires more work?
- see grid above for whats missing
- how do we allocate resources?
Notes 8/25/2009
What information do you need to run your business?
- how will we use this information?
- make decisions around business process and improvemnts
- help focus and prioritize our work
- what data do you need to get the information?
- do we have access to that data now?
- if not, then how do we get it?
- examples
- #of dead courses in Stellar and wikis (for clean up, storage space and policy to communicate to customers in terms odf service)
- unit cost of a course in Stellar
- successful vs failed logins via Touchstone
- what courses are not in Stellar (for outreach planning)
Links
https://rolesweb.mit.edu/roles_quarterly_stats.html
https://warehouse-web.mit.edu/cgi-bin/warehouse_usage.cgi
https://wikis.mit.edu/confluence/display/WBS/Quarterly+Reports
https://wikis.mit.edu/confluence/display/WBS/Monthly+Usage+Graphs