Project plan for ongoing perMIT work, (FY10 and beyond)

Goals:

1.       Complete the work necessary to transition from MIT Roles to perMIT as our enterprise privilege management and access control system.

2.       Continue to add features and functionality to perMIT to make it more useful for MIT, and as a side effect the world at large.

Phase One

Phase One took place during FY09. This included the launch of the project and its primary goal was the creation of an open source version of the existing MIT Roles system. The project started five months late due to a lack of staff resources.  The project has garnered interest from some outside parties. Although the project has not met all of its initial deliverables, the project has reached a point where we can commence on the path of transitioning from MIT Roles to perMIT.

Phase Two

During phase two we expect that MIT Roles will remain our system of record for privilege management. However, we will instantiate perMIT as a shadow system that contains all of the same data as the Roles system. Data updates made to Roles will propagate to perMIT in near realtime. Near the end of this phase some of the systems which currently rely on Roles will transition to using perMIT as the policy decision point, or as the source of the data necessary to make an authorization decision.

Phase Two work items: (FY10) 

1.       Go to TAP with overall plan - (due 9/23/09) - some details, determine when to go to TAP for further review and feedback - https://jira.mit.edu/jira/browse/PERMIT-34

2.       Establish the MIT perMIT server environment.-  (due date for development and staging 9/25/09) - Deployment machine necessary for Development, and Staging. Production will follow at a later date. https://jira.mit.edu/jira/browse/PERMIT-9 , https://jira.mit.edu/jira/browse/PERMIT-10

started 9/15/09 - 3.       Update batch feeds into Roles to simultaneously feed into perMIT. - (due 1/15/10) -  Note that MIT Roles receives various batch feeds of data from the Data Warehouse. This information is used to maintain and operate the privilege management system. This includes, but is not limited to, information about all cost objects and profit centers, and other financial units which become some of the qualifier hierarchies within the system. Other data includes HR Org units, EHS Principal Investigators and room sets.  https://jira.mit.edu/jira/browse/PERMIT-12  

• Complete inventory of all necessary batch feeds. https://jira.mit.edu/jira/browse/PERMIT-11  (DONE)
  • See http://rolesweb.mit.edu/roles_job_doc.html
  • See http://web.mit.edu/repa/www/roles_data_feeds.html
• Modify the feed system so that data is propagated to both Roles and perMIT (start by early October):
  • Data Feed to perMIT: morning_jobs_early  -  https://jira.mit.edu/jira/browse/PERMIT-41
  • Data Feed to perMIT: morning_jobs_late - https://jira.mit.edu/jira/browse/PERMIT-42
  • Data Feed to perMIT: cron_run_exception_notify - https://jira.mit.edu/jira/browse/PERMIT-43
  • perMIT maintnance script: weekend_jobs - https://jira.mit.edu/jira/browse/PERMIT-45
  • Data Feed to perMIT: hourly_jobs - https://jira.mit.edu/jira/browse/PERMIT-46
  • Data Feed to perMIT: cron_roles_cleanup_archive - https://jira.mit.edu/jira/browse/PERMIT-47
• Modify the feed system so that it handles situations where one of the systems is unavailable, and the two systems will remain in sync, once both systems are available once again. Note this is closely related to https://jira.mit.edu/jira/browse/PERMIT-22. If the batch feed has occurred on Roles, but not on perMIT, transactions made within Roles, might fail within perMIT, until the batch feed has occurred on perMIT. E.g Can’t assign a particular ASPEC within perMIT, until the associated qualifier hierarchy exists on perMIT.
• Determine if any of the batch feeds can be converted to realtime incremental feeds. https://jira.mit.edu/jira/browse/PERMIT-13
• Convert batch feeds to incremental feeds as appropriate.

4.   Realtime transaction feeds from Roles to perMIT, to enable perMIT to shadow Roles. - (due 3/15/10) - https://jira.mit.edu/jira/browse/PERMIT-22 

•  As end users create, update, and delete Authorizations and related data within Roles, the transactions will need to be propagated to perMIT in near realtime.
• The goal is to preserve and propagate the transaction, not the resulting data, to the shadow system. This has multiple side effects. First, it will result in perMIT taking nearly the same code paths while a shadow system, which it will have to execute when it becomes the system of record. Second, it forces us to reevaluate our audit logging strategy in both Roles and perMIT. We expect that some changes to the audit system will be implemented as a result of this work. Third, it also provides us with a relatively clean mechanism to leverage when we wish the reverse the system of record and shadow relationship between perMIT and Roles. Finally, it will also lay the ground work for adding a high availability and/or multi-master capability to perMIT in the future.
• Determine if we wish to add more columns to the existing Authorization Transactions table (e.g. which instance/server the transaction to place on, “ignore insert”, “ignore update”, “ignore delete”) - https://jira.mit.edu/jira/browse/PERMIT-53
• Develop code that initiates incremental updates of the perMIT ASPECs based on the Authorization Transaction (AUTH_AUDIT) table.  - https://jira.mit.edu/jira/browse/PERMIT-54
• Design a “Function Maintenance Transaction” table in both Roles and perMIT, along the lines of the AUTH_ADIT table and leverage this for incremental propagation of function maintenance.  - https://jira.mit.edu/jira/browse/PERMIT-55
• Develop code that initiates incremental updates of the perMIT functions based on the new “Function Maintenance Transaction” table . - https://jira.mit.edu/jira/browse/PERMIT-56
• Design a “Category Maintenance Transaction” table in both Roles and perMIT, along the lines of the AUTH_AUDIT table and leverage this for incremental propagation of category and function maintenance.  - https://jira.mit.edu/jira/browse/PERMIT-57
• Develop code that initiates incremental updates of the perMIT functions based on the new “Category Maintenance Transaction” table . -  https://jira.mit.edu/jira/browse/PERMIT-58
• Much of the qualifier maintenance is actually done via batch and bulk updates, however, a small amount of qualifier maintenance is done manually (via the UI). We need to come up with effective strategy for how we will capture the manual transactions and propagate them to the shadow system. This is likely to resemble the approach being used for ASPEC modifications and function maintenance, although it has some added complexities. - https://jira.mit.edu/jira/browse/PERMIT-59

5.   Improve Qualifier data sub-typing, - (due 2/15/10) -  https://jira.mit.edu/jira/browse/PERMIT-35

• Currently ther are stored procedures that contain business logic that evaluates the data format, and uses the result of the evaluation to determine a data type and act accordinging. E.g. Profit center is not a qualifier type. Instead the stored procedures recognize that a qualifier starting with a "P" followed by a legacy 6 digit department code denotes a profit center.
• There are schema changes to be made
  1. Design: https://jira.mit.edu/jira/browse/PERMIT-36 (due 9/15/09)
  2. Implement: https://jira.mit.edu/jira/browse/PERMIT-37 (10/15/09)
• Backwards compatibility issues to be worked out
• A Simple UI to be created, for testing and basic maintenance - https://jira.mit.edu/jira/browse/PERMIT-40 (1/15/10)
• Web service methods to add/revise - https://jira.mit.edu/jira/browse/PERMIT-39 (2/15/10)
• Stored procedures to be updated - https://jira.mit.edu/jira/browse/PERMIT-38 (3/15/10)
• Data to be converted 12.   Begin work with identified applications to use perMIT. – (4/20/10) - Transition some applications to rely on perMIT as their primary data source for authorizations or use the web service to use perMIT as the policy decision point.
• Likely initial candidates will be applications with a relatively stable set of authorizations. Examples include:
• Touchstone/CAMS (web service interface)
• MIT ID DB (uses nightly feeds)
• Library  (implied authorizations)

postponed - 6.       Revisit Kuali Service layer readiness.- (due 9/15/09) -  https://jira.mit.edu/jira/browse/PERMIT-23

• Check in to see if KIM and KS issues have been resolved and determine what we should create as a “standard” web service interface to perMIT

postponed - 7.       Creation of Sample Data – (9/15/09) -  suitable for external site consumption https://jira.mit.edu/jira/browse/PERMIT-28

postponed - 8.       Port existing Roles web service to perMIT - (due 10/1/09) - . (This was one of the uncompleted phase one tasks.) https://jira.mit.edu/jira/browse/PERMIT-21

9.       Add new methods to the Roles Web Service to handle function creation and maintenance. - (due 11/16/09) - https://jira.mit.edu/jira/browse/PERMIT-19

10.       Add federation support - (due 12/1/09) -  https://jira.mit.edu/jira/browse/PERMIT-14   We also need to restructure how we feed identifiers about people into the system. We preserve what we have, and also support federation.

postponed - 11.       Packaging – (12/15/09) for external distribution

There are also other remaining work items that were not completed during phase one:

• Establishing the appropriate open source license for perMIT
• Documentation (ongoing)
• Completion of the stored procedures
• Completion of the CGI programs
• Testing the Roles WS with perMIT

Phase Three 

The major goal of Phase Three is to transition perMIT from the shadow system to the system of record, and Roles will become the shadow system.

• Identify what remaining development tasks need to be completed before we are confident that we can make the transition.
  • The Roles PowerBuilder application must be fully decommissioned.
  • Incremental propagation from perMIT o Roles must be fully tested.
  • The web service that can update Roles data will have to be “re-pointed”  to perMIT.
• Identify all downstream systems that have a tight binding to Oracle because they are making a direct SQL connection to the database. During this phase, these applications will still be able to read the data via a direct SQL connection to Roles. All updates will be done via perMIT.
• Completion of the outbound feeds (all written for SAP)
• Determine if the time is right to re-implement how SAP obtains this data. Should a (new) web service be used instead? Or should SAP get the data from the Data Warehouse? 

Phase Four 

The major goal of Phase Four is to scale back the shadow system and reduce it to a core set of tables that will serve a small number of applications that have a tight binding to accessing “Roles” via an Oracle SQL connection. The primary example is the Data Warehouse.

Closely related to this goal is the need to provide consulting services, and potentially co-development resources, to migrate a set of existing applications from a direct SQL connection to the web service interface. That approach is not expected to be viable for all applications, but we expect that several will be able to make that transition.

This phase is where we will phase out some of the stored procedures that are very MIT specific. This includes the stored procedures that are knowledgeable about the formatting of qualifier data.  

Current unknowns and risks

• Can we get MySQL to perform as well as Oracle for some of our large data sets within Roles?
• Can we get complex rules based authorizations to work? (Implied authorizations.) We believe this is a very low risk.
• There are some financial specific CGI scripts that are very MIT specific and should not be included in the general perMIT distribution. We must continue to balance what we preserve as MIT specific interfaces or reporting, and what we provide in a more generic form.
• Would Postgres be a better choice than MySQL?