July through September 2009
April through June 2009
This quarter saw significant changes in personnel with layoffs early
in the quarter. These layoffs have caused significant impact on how
our regular work gets done - in some cases doing more work and in other
cases we have had process improvements. Team morale has been fragile
given staffing changes but it is improving. Promised changes at the
highest level have had impact by keeping fever, uncertainty and doubt
at a reasonably high level. Given the events of the last year, this
appears to be a new normal, unfortunately.
My apologies to any members of DSPS if I have missed any important
activities. We do quite a bit these days. I have to say I am quite
pleased with the overall activity of the team noting how we very much
sit in the middle of things and have to work closely with other teams
to be able to get our work done - we are not an independent entity for
almost all of what we do. We can't get our jobs done unless we collaborate
with others. I thank the team for their continued diligence and professional
behavior in the face of adversity.
AMIT - Application Management and Integration Team
AMIT participates on various projects as a catalyst for technical issues.
Major projects for AMIT involved IS&T Website, Debathena, Wikis and assisting
in various ways with the many projects around ISDA. The IS&T Website has
largely on production from the perspective of AMIT. Working closely with
NIST to provide technology and methodology to meet NIST requirements. Website
has been a challenging project involving significant staff time for 2 members
of the team.
Wikis was upgraded to 273 on April 1. Further work on wikis finally fixed our
performance problems plaguing us for last 2 years. We initiated the process
for upgrading Wikis to Confluence 3.0 bringing it forward 4 major software releases.
This also involved significant infrastructural changes to the application
involving LDAP integration, switching to Tomcat-6, database modifications along with
eliminating lots of custom code while providing additional functionality.
Debathena (formerly known as Athena 10) has gone live in some student clusters.
This is a significant effort culminating 18 months of work by Bob Basch,
Andrew Boardman and Bill Cattey providing significant changes to underlying OS
technology as well as process changes for Athena OS. Largely seen as a behind the
scenes effort - this work will have long-term effect on MIT keeping pace with
the ever-changing world of Linux. Approximately 50% of amb time on Debathena.
AMIT continues to provide various forms of assistance on an as-needed basis for
Events Calendar, DCAD, Stresstester and Web/Application Server technology around
IS&T.
Identity Services (touchstone metrics at bottom of narrative)
We deployed a new WAYF for Touchstone making it cleaner and easier to use. We
continue to provide development and support for Touchstone in close collaboration
with NIST - carefully following NIST operating rules. About 25% of rbasch time
is spent on Debathena. We began working with Alumni to switch to Touchstone and
take on about 100K users into CAMS. This work is proceeding. Working closely
with NIST we have finally deployed a production LDAP service viable for application
use. Apps configured to use the directory but not yet production are: Wikis,
Bamboo, Nexus, Drupal, Moira-Web NG (probably others). NIST is using this new
LDAP for various services as well. rbasch has also been working towards providing
the shibboleth SP 2.1 while keeping up with support requests for additional
touchstone related services. pbh and rbasch have also spent significant time with
MIT Libraries helping them to shib enable several applications along with the
EZProxy infrastructure component used widely. We also deployed a fix to allow
the iPhone to use certificates with Touchstone where iPhones were not working
properly with regular certificate enabled applications.
We completed RolesUI - reworking of the web based Roles interface. A significant
effort by Vijay Konda along with Paul Hill. Both Paul and Vijay also play central
roles in the development of perMIT. Given all the challenges of the last year,
fiscal and delayed talent acquisition, perMIT has been running behind. As a Big
Initiative project it completes June 30, 2009. This project will continue as part
of normal project work into 2010 with the intent of deploying perMIT at MIT. We
made significant progress in perMIT, much more than we expected given the challenges
but we have not delivered on a viable product for MIT or the outside community. We
have succeeded in developing significant interest from the outside community as
many are realizing the efficacy and viability of perMIT.
Developer Tools
David Tanner and Gongqin Li have been working to finish up our developer tools
offerings. By the end of July we will roll out a Maven repository used to maintain
java related components such as jar files and a continuous build server by Atlassian
callled Bamboo. These components will serve developers, along with our SVN code
repository to allow for more effective group development of Java applications supported
by common repositories, consistent code build and testing along with appropriate access
to these components. Putting these components have taken quite some time due to
some historical false starts and multiple directions pursued. This effort will conclude
with a set of tools integrated into our infrastructure and should remain viable for
the next few years. Congratulations to David and Gongqin on the completion of this
effort.
SoftWare Release Team (SWRT)
SWRT began its transition from CSS to ISDA during this quarter. The team
changed at the time of the CSS reorg and was reduced from 8 to 3 members.
At the end of May Atticus announced his departure for June 30 - so we are
now at only 2 positions awaiting EVP approval to fill the vacant position.
The team moved to W92 at the very end of the quarter. During these changes
the SWRT released 6 software updates and responded to 58 support and licensing
requests via the SWRT RT queue. The team has initiated preparations for the
next major releases for Mac OS and Windows - both releases expecting to take
reasonable time and effort to understand, document and deploy. The team put
installers for TSM (mac, win, linux), VirusScan (mac) and X-Win32.
Mobility
You will find attached a very nice set of metrics and a summary of current
development activities for Mobility. We are in the process of transitioning
from use of contractors to in-house developers due to budgeting limitations
moving forward. Amon Horne and David Tanner will be devoting 20% time each
towards Mobile development activity. Getting started with this transition has
been somewhat challenging due to other projects starting and concluding. For
Amon, MITBI is a priority and David Tanner is working to conclude his efforts
on DevTools. We hope to have effectively transitioned dev efforts by September.
Andrew Yu continues to be the lead on the overall effort and he has initiated
iMobileU as an initiative amongst various HE institutions to further the usage
and development of the MIT Mobile platform.
----------------------------------------------------------------
Touchstone Metrics
Core IdP Application Access involving AuthN: 41664
Core IdP Application Access Single sign-on: 40148
Total App Access requests for Core IdPs: 81812 (Q2 was 66703)
Logins by Method:
username/password: 35366
MIT Certificates: 5661
Kerberos: 637
By Applications: Stellar=71518, Wikis=2325, Jira=2286, Thalia=1679
rolesapp=84, developers.mit.edu=68
Remaining apps are non-production or not-yet production or not yet worth mentioning.
CAMS IdP Application Access involving AuthN: 9317
CAMS IdP Application Access Single sign-on: approx 9317 (based on 1:1 relation for core Touchstone, was reported as 65881)
Total App Access requests for Core IdPs: 18634 (Q2 was 218)
By Applications: Stellar= 17105 (was 63712), Wikis=1529
Stellar is so large as it's configuration is broken due to lack of architectural understanding. Thus, we made a good faith estimate that the 1:1 ration for core also exists for CAMS, thus rectifying the skewed Stellar numbers.
Total CAMS accounts is about 3500. In December 31, 2008 we had 65. We are working
to obtain the Alumni community which will add about 100K.
----------------------------------------------------------------
Web Services Statistics
(2008-Q2)
Avg response times (success/exceptions) in milliseconds:
Geo(1202/708), UA(113/61), MITID(4194/1830), Moira(84/136)
MITID has interesting heuristics affecting it's times hence the multi-second
response. This is normal.
2009-Q4
Geo(986/708), UA(4274/678671), MITID(3295/2584), Moira(596/494)
UA response times are out of whack (and will be going forward) due to a
Roles outage on 6/23/2009.
MIT-Mobile_Web_Stats_06-2009.pdf
January through March 2009
No updates posted.
October through December 2008
Application Management and Integration Team (AMIT)
Major efforts this quarter of AMIT focused in several
areas: Configuration Management, Thalia support, IS&T
Website project and config and community development
for developers.mit.edu. AMIT responds to a wide range
of support requests within ISDA and for SAIS and CCS
as well as the larger MIT community.
The Configuration Management project was performed in
concert with OIS Server Ops to better leverage the
Radmind tool previously selected by team members from
Server Ops and ISDA. Server Ops has already been using
Radmind (developed by University of Michigan) for quite
a bit on its server deployment and ISDA is utilizing the tool
for the next layer of middle tier software and application
management where appropriate. This resulted in some changes
to how Server Ops had initially intended to handle things
to better accommodate application needs. Server Ops and
ISDA collaborated well to work thru these issues. While
this effort continues to be a work in progress we are
seeing benefits of server software deployment and config
reduced from about 5 hours to about 17 minutes. ISDA
will still need to go back and reconfigure already
deployed systems in this new fashion to fully realize the
benefits but going forward looks very promising.
Thalia support continues as Thalia reached a production
status. Thalia use of Alfresco continues to be challenging
from the server software management perspective. We spent
time streamlining processes for backups and responding to
various configuration and software deployment requests
from the Thalia development team.
The IS&T website project pushed us to resolve issues around
PHP configuration and support along with Drupal config and
integration with Touchstone. Our experiences with this project
and Drupal/PHP have benefited the creation of the developers.mit.edu
web site and aided HR and DCAD with their own deployments and
integration concerns with Touchstone.
The Systems and Application Monitoring and Metrics effort has not
yet begun due to other work.
Last quarter our people resources were dramatically impacted by
having to perform server restarts on behalf of Alumni for the
Alfresco CMS. We worked with Joe Calzaretta to provide the users
a capability to restart the CMS by themselves and this has made
the users immensely happy and has reduced our effort in support
of Alumni to nearly zero.
AMIT also spent a fair amount of time evaluating our servers, virtual
and real, determining there were a number of systems no longer required
or could be converted to VMs. We eliminated several systems in our
co-lo rack space and utilized the better systems to improve our VM
playground. All this resulted in approximately $40K of savings of
our hardware budget and positioned ourselves to better respond to
requests for playground systems.
ATHENA (9 and 10)
We largely completed the Athena 10 cluster release as a follow-on
the Athena 10 Preview release from last quarter. We are still
targeting end of IAP or February 2009 (or so) for the Athena 10
test cluster and beyond with being ready for Athena 10 quite a bit
before summer 09. Several patch releases were handled for Athena 9
to maintain various end user software and security patches. We
expect Athena 9 updates to slow as we continue towards summer.
Developer Software
Work continues addressing issues of our Web Services infrastructure.
We continue to review previous work in this space to better understand
community needs and refine (fix) issues previously left open like
proper integration of IDE (MyEclipse) with Maven repositories and
how to properly provide a service of a Maven repository. This work
has been presented to the Java Users group. MIT Application Platform
(MAP) has been re-positioned as a Community outreach effort and the
software stacks (presently Java). It is not clear whether or not
MAP will survive as a software stack as community participants have
already diverged from the SASH Server provided. The developers web
site appears to be a promising effort as a community function.
We have looked at various continuous build servers (we currently
have Bamboo and Continuum - we seek to deploy only one build service
and have more thoroughly explored the one's we have along with Hudson
and also explored alternative maven repositories such as Nexus, Archiva
and Artifactory. We will likely settle upon Nexus and have not yet
finished determining which continuous build service to provide.
The Web Services Working Group has been initiated and led by David
Tanner. This WG is intended to be the likely mechanism for guiding
the creation of Web Services going forward and defining reasonable
coding and deployment practices. While initially comprised of ISDA
members, this will change to include other IS&T members next and then,
hopefully merge better with the developer community.
Identity Services
The perMIT project has been stalled. Q1 caused delays for administrative
reasons as we shifted how we addressed the personnel aspects of the
project. Q2 was spent trying to find a full-time developer to meet the
needs of the project and beyond. During this quarter we were unsuccessful
filling this position. The project remains delayed.
Touchstone continued to spin up. Paul Hill gave an ITAG presentation to
over 60 members of the IT community with terrific interest and response
to Touchstone enabling of applications around the Institute. Touchstone
has joined the InCommon Federation which will quickly yield free student
access to Microsoft SDKs for download. With Touchstone/CAMS finally live, we
spent a fair bit of time working with key applications on how to migrate
to using Touchstone as the primary access mechanism and processes for
migrating external accounts into CAMS. We continue to pursue the issues
around Touchstone/CAMS becoming an Identity Provider of Last Resort for
InCommon. MIT will have a seat on the InCommon Technical Advisory Council.
Touchstone statistics are reflected below.
This quarter we managed the development aspects of cleanup work for the
RolesUI with DRS as our customer. We managed the work flow and the
contractor for getting as much of the work outlined as possible and
ensuring the RolesUI web service aligns with our direction for supportable
services. AMIT handled the deployment issues and Dev Software addressed
a number of integration, coding and structural issues with the contractor.
This work is scheduled to completed on time in the worst case, likely
finishing early.
DSPS has been working closely with NIST to develop a next generation
directory service (ldap.mit.edu) preserving the current uses of the
directory while supporting realtime Moira data population in support
of useful data delivery to applications. In example, our Wikis service
has suffered 3 to 4 hour delays for group management via Moira resulting
in frustrations from users. This new directory service will also be
utilized for realtime mail routing, DHCP services and even limited
authentication for LDAP only applications such as the PGP server for
the HIDP project. This service will be replicated in multiple data
centers with multi-master replication. The server software is open source
and free, built on the redhat fedora project having roots in the iplanet
directory server. In the coming quarter we expect to also develop a new
web interface for users to manage their groups and memberships. This
work will be performed in concert with ISDA/CCS.
----------------------------------------------------------------
Touchstone Statistics
Core IdP Application Access involving authN: 49473
Core IdP Application Access "single sign-on": 17230
Total App Access requests for Core IdPs: 66703 (Q1 was 33418)
Core IdP Access by application: (12 service providers)
Stellar = 62144, Wikis = 1502, Jira = 1474
Teamspaces = 1386 (Q1+Q2 = 1710)
Core IdP by AuthN Method:
username/password: 45224
MIT certificates: 3224
Kerberos: 1025
CAMS IdP Application Access involving authN: 99
CAMS IdP Application Access "single sign-on": 119
(Only one authN involved OpenID. These numbers remain low
as Stellar and Wikis have not converted to using Touchstone as
primary access mechanism.)
CAMS IdP Access by Application: (4 service providers)
Teamspaces = 90
-----------------------------------------------------------------
Web Services Statistics
Avg response times (success/exceptions) in milliseconds:
Geo(1202/708), UA(113/61), MITID(4194/1830), Moira(84/136)
MITID has interesting heuristics affecting it's times hence the multi-second
response. This is normal.
July through September 2008
The last two months of this quarter the DSPS group has been
under new leadership. While work has continued largely unabated,
significant activity has taken place to evaluate what work DSPS
does and how it does it. The group has been re-oriented into
largely 3 teams: Identity Services, Developer Services and the
Applications Management and Integration Team (AMIT). This allows
for a better focus of effort amongst those performing the essential
work of DSPS. A fourth team is emerging in support of various
platforms such as Athena and Mobile but the work here is not fully
defined as is the rest of DSPS.
DSPS has delivered to CCS the necessary programming and infrastructure
in support of the TeamSpaces evaluation and demonstration effort.
Several team members have been involved to make this effort happen
and more time spent than anticipated due to various planning and
technical problems evolving. As interim solution for group
management TeamSpaces has been deployed utilizing the Moira Web
Service developed in-house.
Thalia infrastructure support efforts have been re-evaluated and
more properly scoped to ensure delivery of working and supportable
systems to Thalia as it moves forward. This has involved reasonable
effort in re-defining service readiness for this project. Thalia
will now be able to meet its deadlines from an infrastructure perspective.
The MySQL cluster effort as a general service for all applications
utilizing the "clustering" technology has been abandoned as it is
technically and operationally not feasible - the technology is simply
not mature. As we proceed to get various projects back on track previously
impacted by this effort we will revisit a MySQL DB service, not based
on clustering.
DSPS worked closely with the Software Download Licensing System (SDLS)
project to determine operational feasibility and highlight issues
in the development process to get the project back on track. SDLS
and the IS&T website project are both based on the Drupal application
framework and we continue to collaborate on both the projects and
lend our "kung fu" as appropriate. Learning this application environment
in order to meet the needs of these projects will yield benefits
down the road for other projects as well.
We continue to invest significant staff resources in support
of CCS for the Alumni/Alfresco Web Content Management system.
We have taken steps to allow the users to operate the system
independently and securely. We hope this is of service to
the users as well as getting our own time back.
Developer Support has been tasked with
reviewing, researching and making functional a suite of tools in
support of developing software in ISDA and, in turn, IS&T and the
larger MIT community. This is a significant effort as it involves
re-factoring code and previous work to make some "viable services".
We expect project work to be defined and initiated in the coming
quarter.
Our various web services had no unscheduled outages yielding
100% availability.
Athena 10 preview was delivered. This project continues on track
to finish Athena 10 workstation by 1/2009 and cluster support by
summer 2009. Efforts appear promising we will substantially beat
the summer deadline.
The perMIT project has been delayed for administrative (HR) reasons.
As all hurdles have been overcome we are now in the hiring process
and intend to have a developer resource and progress made during the
next quarter. Handling of this project has been moved to Identity
Services. Identity Services is also coordinating the development
effort for correction and completion of the RolesUI project in
collaboration with DRS.
Projects likely spinning up for next quarter: Configuration Management,
System and Application Monitoring and Metrics (SAMM), Touchstone 2.0,
MySQL DB Service.
Touchstone went live as of 9/18/2008. The Identity Services
team is now working with various applications and developers to
properly enable Touchstone as the primary access mechanism for
those applications. As expected, these applications need to
consider user workflow issues as we move towards a more appropriate
single-sign-on and identity managed environment. While Touchstone
has been available for some time to applications it has not been
formally "production" until now. We are collecting statistics and
they are noted, with some explanation. We have yet
to determine how to effectively manage these statistics going forward.
The core Identity Providers representing the MIT community identities
indicate the following Touchstone Access Request counts as:
From July 1 to Sep 30 Touchstone usage has been:
Application Access involving authN: 24205
Application Access "single sign-on": 9213
Total App Access requests via Touchstone: 33418
Touchstone Access by application:
Stellar = 31457, Wikis = 1377, Teamspaces = 324, Jira = 177
By Authentication Method:
username/password: 23210
MIT certificates: 514 (this capability was added on 9/18)
Kerberos: 481
CAMS - The Collaboration Account Management System access counts
are trivial since CAMS was made available on 9/18 as well. We have
had only 63 accesses via CAMS. This will obviously increase as
applications avail themselves of the CAMS capabilities.
April through June 2008
- Touchstone continues to be one of our most important projects. In spite of family emergencies that cost several person weeks of work, development has been largely on schedule. There have been issues around setting up staging and production systems -- slowly but surely these are being resolved. Functional testing is in good shape, stress and load-testing awaits the training and consulting services available with the purchase of the new StressTester tool from Reflective Solutions. We have been intensely engaged with the key customers (esp Stellar, Clearspace) to make sure that they have the functionality that they need to go live in the fall in a phased approach. The full production system will probably not be handed over to OIS to maintain until October or so.
- Web infrastructure and developer support of web service projects has used some extra resources in the last quarter -- a new Geo Codes service provider has been needed, DSPS has helped with issues on the new Roles app. The team has been tasked with integrating Clearspace with MIT infrastructure services and this has impacted work on other projects.
- Athena 10 is on schedule for a fall preview, at the expense of developer tools work.
- The priority of integrating Clearspace with MIT infrastructure has delayed work on Touchstone database configuration and PHP systems for the SDLS, QuickPages, and IS&T website.
- With the departure of the senior developer from the Stellar team, a resource was loaned from DSPS to complete the gradebook work required for the fall.
- IS&T website went before TAP, findings require that the team prove the high availability/slashdot-resistant architecture for Drupal and resubmit this to TAP. This will involve some cost in both time and money that must be spent on this proof of concept before the actual site development can begin. No date can yet be committed for delivery of the IS&T website. However, site map work and functional specification is still on-going, as well as migration planning, as this would be needed regardless of the future CMS platform.
- Alumni Association website still has major outstanding bugs, one of the most severe seems to be fixed in the most recent stable release of Alfresco (2.1 SP 3), so we will begin a process of integrating our modifications with that SP3 and testing it in a staging environment before rolling it out to Alumni.
- Thalia is preparing a final release before going into production. Official production is dependent on "system readiness" configuration with the shared MySQL cluster and more thorough runbook documentation and automated procedures. Clearspace integration has delayed some of the app support team work necessary, but Thalia is still expected to be live by the fall.
- QuickPages is partly backburnered due to higher priorities, but work is still on-going.
- SDLS still has ongoing work, is affected by the higher prioritization of Clearspace by the app support team.