Future directions for LDAP services at MIT

 Draft proposal

1.      ISDA proposes to enhance the MIT IS&T LDAP services in the following ways:

1.1.   Evolve LDAP services into a core infrastructure that IS&T provides to the community
1.2.   Engage the developer community to better understand their desired use of LDAP services
1.3.   Formulate an understandable overview of the various LDAP services and services operated by IS&T today and tomorrow

2.      Scope

2.1.   In Scope

2.1.1.      Engagement in discussion of policies regarding authentication and access control to ldap directories
2.1.2.      Maintenance and changes to the ldap schema
2.1.3.      Maintenance and development of new data feeds to the ldap directories
2.1.4.      Developer outreach to developers seeking to use LDAP
2.1.5.      Exposure of data that may be used for authorization decisions
2.1.6.      Staff development

2.2.   Out of Scope

2.2.1.      Changing the groups responsible for running the production ldap servers
2.2.2.      Looking up people via web.mit.edu, mitdir, finger, and related systems.
2.2.3.      Providing an "LDAP authentication" service
2.2.4.      The Active Directory used by WIN.MIT.EDU
2.2.5.      The OID used to support mobile device synchronization to TechTime

3.      Guiding principles

3.1.   IS&T should not offer LDAP as an authentication service to developers
3.2.   The enterprise LDAP service will be primarily read-only, it will not be used to cache application state
3.3.   Looking up people via the MIT home page, finger, and related systems are an application layer. These are not necessarily driven by LDAP, nor part of the LDAP enterprise service.

4.      Driving Factors and Deductions

Approximately 10 years ago IS&T (then IS) started working with LDAP servers to as a backend service supporting other systems and services. Feeds from Moira and the Data Warehouse were developed to populate these systems with data that would remain consistent with other systems on campus. This work was not advertised to the wider developer community nor was it exposed to end users via LDAP. Overtime the user community became more aware of LDAP as a directory service, primarily because of the configuration options available in email clients. Much of the industry marketed LDAP as directory service about people and not as a general purpose directory system that could also be used for systems management. As a result IS&T also created an LDAP service that fulfilled the needs of the user community seeking an institute wide address book for their email clients.

In the past two years a different set of users have started to approach IS&T about the use of LDAP. A growing number of 3^rd party applications, both open source, and proprietary are LDAP aware. These applications tend to be LDAP aware in at least one of three ways:

• They desire to use LDAP as an authentication service or password validator
• They desire to use LDAP as a data storage system for system state about a users account and or preferences
• They desire to use LDAP as group management system that can be referenced for authorization decisions

IS&T should not create an enterprise "LDAP authentication" service (see glossary below). Although at first glance this may appear to be an attractive service to provide, and many sites in the world use LDAP for this purpose, offering such a service would create a much larger surface attack area than we think we are exposing today. The biggest drawback to such a service is that a highly likely outcome is that many applications would appear on campus which would prompt a user for their Kerberos password. Even though it might be sent to the LDAP server in a secure manner, we do not control the system administration and configuration of each of the systems that might be prompting the user for the password. It's likely that at least some of those systems would be incorrectly configured and would be receiving the users' passwords over an unencrypted channel. Another likely scenario is that over time some of those systems would become compromised and would be used to log the users' passwords as they are received by the intermediate system before the password was sent to the LDAP server for verification.

The enterprise ldap service should be primarily a read-only system. It should not be used to store application state, or rapidly changing user data. Some ldap enabled applications do use ldap this way. As an example Exchange introduces nearly 2000 schema changes to Active Directory and it stores information about the last message read, display preferences, last login time, last time an error occurred, and many other pieces of data that may change throughout the day. Such systems may introduce performance problems when tied to an enterprise ldap directory. Offering to accommodate such features at the enterprise level will also increase the burdens associated with making any proposed change. It becomes much more difficult to be confident about the load impact, and if many applications are requiring schema changes it becomes more important to carefully evaluate any such change proposal. Because of these concerns the enterprise ldap service should be primarily read-only, and applications that use ldap as a dynamic data store should be encouraged to use an application specific ldap server that can be optimized for the applications usage.

The enterprise ldap service should not be considered a system of record. Instead it will receive data feeds from the Data Warehouse or directly from systems of record such as Moira and the Roles DB. This will help to ensure data consistency across our systems. Any attempt to create bidirectional feeds both into and out of the ldap system is likely to be a large and expensive project if we wish to maintain a consistent level of quality in our data. It is unlikely that such a cost will have a positive ROI. This also implies that although the ldap service may need to be highly available, the data residing in the ldap system may easily be recreated from existing systems at any time.

There is a need to provide interfaces to our list management and authorization management data in manner which makes it easier for developers to do and also makes it easy to integrate such data into third party systems. We should consider LDAP as one part of our strategy for achieving this. This will require near realtime propagation of such data into LDAP. It also means that we must understand what data from our other management systems needs to be propagated and what types of transformations must be performed. As an example there may be some data from the Roles DB that can be efficiently flattened from the tuple model to an LDAP group. However, there will be other data that should not be flattened in the way because it would cause a cumbersome multitude of unnecessary and unused groups to end up in LDAP.

5.      Inventory of LDAP servers operated by IS&T

5.1.   WIN.MIT.EDU

The WIN.MIT.EDU and all other AD based Windows domains at MIT include an LDAP server which is the basis of Microsoft's Active Directory. The WIN domain receives a realtime feed from Moira using Moira's incremental update service, which is also used to update the AFS PTS servers in realtime. The WIN domain also takes a nightly feed from the Data Warehouse for details about MIT people, this data includes things like a person's office phone number and office location. Other AD Domains on campus do not receive data feeds from Moira or the Data Warehouse, unless the system administrators have done so without general IS&T knowledge or direct support.
The WIN domain's LDAP server requires Kerberos v5 authentication via a GSSAPI bind for any and all access. Application developers are not encouraged to use the WIN directory for their services. However, it has come to our attention that a small number of systems may be doing this. For example, we believe that the parking system may be using queries to AD to determine who can modify parking gate authorizations. (Tom Coveney of DITR should be able to clarify this example.)

5.2.   OID - run by NIST

One OID (Oracle Internet Directory) instance is run by NIST to support PDA synchronization to MIT's TechTime. This is used purely as an LDAP authentication system to support the PDAs and mobile devices. It is not offered as a general LDAP authentication service to the MIT developer community.

The server runs on the same box that hosts the TechTime calendar server. It will not accept connections from other servers or clients. A user's password reaches the calendar server over TLS. The calendar server then passes the username and password to the OID instance. The OID instance then verifies the username and password correctness using the KDC.

5.3.   OID - run by SAIS

SAIS runs two instances of OID. One instance is used in support of the insideMIT portal. It receives feeds from the MIT Data Warehouse for Kerberos ID and other user data, from SAP for various HR data and portal "profiles", and from the MIT Roles database for authorization data. The second instance of OID is used for the MITSIS system. Plans call for the later system to receive feeds from SIS systems regarding student data.

The decision to have two distinct OID instances within SAIS was a result of requirements feedback from OIS.

5.4.   LDAP.MIT.EDU

5.4.1.      Inventory and topology

Ldap.mit.edu is implemented on OpenLDAP version [XXX]. Two servers with the same data reside behind an F5 switch. This configuration provides for fail over and load balancing. The individual servers should not be queried directly except for when testing for consistency or other troubleshooting. The host names of the individual servers are subject to change without notice. The current hostnames are w92-130-ldap-1 and w92-130-ldap-2. They are currently Dell 2850s.

5.4.2.      History

The service was introduced shortly after the introduction of TechTime, about the same time as the NIST OID (for support of TechTime). The primary usage over the past few years has been as an address book for email clients. Since the formation of ISDA there has been a growing interest in using LDAP as a way to look up group memberships to perform authorization decisions.

5.4.3.      Overview of services provided

• Looking up email addresses of people from various email clients including Outlook, Apple Mail, and Thunderbird
• The ability to look up Moira group memberships for Moira lists that are of type visible. Type visible within Moira indictes that non-members may view the membership.
• Used to provide SIP.edu functionality with OpenSER for MIT VoIP pilot and MIT Personal SIP service [?]
• Used by the core MIT Touchstone Shibboleth IdP to generate Shibboleth attribute assertions.

5.4.4         Data sources used by ldap.mit.edu

Ldap.mit.edu recieves data feeds from multiple systems, these include:

• Moira
• Registrar via the Data Warehouse
• HR via the Data Warehouse
• NIC
  

6.      Proposed initiatives and tasks

6.1.   Establish staging servers of ldap.mit.edu so that proposed changes due to maintenance or new efforts can be tested prior to production deployment.
6.1.1.      ISDA and OIS shall have access to these servers
6.2.   Commit code for current data feeds into a known source code repository with read access to at least all of ISDA and OIS
6.3.   Commit OpenLDAP config files to in a known source code repository with read access to at least all of ISDA and OIS
6.4.   Create a run book for the operation of ldap.mit.edu servers so that common operational knowledge is shared by a wider number of staff members
6.5.   Invest time and training in some staff members to ensure that ISDA and OIS have the skills to turn the ldap service into a viable enterprise service offering that can serve the needs of a variety of applications, can introduce new data elements in a timely manner, and can ensure that the system remains stable and does disrupt existing applications that are using the system.
6.6.   Establish code and configuration review procedures for making changes to the staging environment and code repository
6.7.   Create a set of example code for developers wishing to use ldap services on campus
6.8.   Provide whitepapers to explain the proper use, best practices, and limitations of ldap services on campus
6.9.   Recommend some LDAP clients for use by developers seeking to learn more about LDAP
6.10.                    Improve monitoring of the ldap.mit.edu to ensure that it functional and returning correct and consistent results.
6.11.                    Increase the rate of propagation of Moira lists to the ldap service. Increasingly often applications are being developed that want to use ldap for the evaluation of group membership. Users of these systems expect that changes to group memberships should propagate in near real time.
6.12.                    Expose some of the Role DB authorization data as ldap group information.

7.      References

7.1.   insideMIT portal notebook entry regarding SAIS's use of OID http://userdocs.mit.edu/portal-nb/archspecs/techspec/IdMgmt.html
7.2.   Moira DCM, service name LDAP, Target: /usr/local/ldap/data/ldap.out, Script:moira/bin/null.sh
7.3.   IS&T information about using ldap.mit.edu in conjunction with various email clients. http://itinfo.mit.edu/article.php?id=8473
7.4.   ITAG Architectural Guidelines: Identity and Authorization, http://web.mit.edu/itag/guidelines/identity.html
7.5.   Some example code for developers seeking to pull user affiliation information from ldap.mit.edu, http://mit.edu/pbh/web_scripts/ldap/community-p.html
7.6.   SIPB is looking for some to do a cluedump presentation about DNS and LDAP, currently unscheduled, http://cluedumps.mit.edu/~cluedumps/wiki/index.php?title=Main_Page
    

8.      Glossary

8.1.   LDAP authentication - This should not be confused with the LDAP bind operation. LDAP authentication is often used to indicate that an application is passing a username and password over TLS to an LDAP server and the LDAP server then evaluates the username/password pair to determine if the user should be successfully authenticated. It may do this by storing the password in the ldap directory, or it may forward the information on to another back end system such as Kerberos to perform the evaluation.
8.2.   OID (Oracle Internet Directory) - http://www.oracle.com/technology/products/oid/index.html

9. Proposed responsibilities

NIST will be responsible for operating ldap.mit.edu this includes

• system administration of ldap.mit.edu inclduing normal maintenance of the server hardward and applying security patches
• modification of configuration or updates in response to operational situations, e.g. disabling the update of student data in order to prevent unintended disclosure of cell phone numbers due to miscommunications amongst other groups
• notification of changes to configuraiton or updates in repsonse to operational situations in a reasonable period of time
• coordination planning with ISDA for tested and planned changes, e.g. schema changes, or maintenance of feeds
• providing ISDA with staging machines which adequately reflect the operational machines (the assumption is that ISDA will bear the cost of the hardware for the staging machines)

ISDA will be responsible for

• maintenance of the data feeds from Moira
• creation and maintenance of data feeds from Roles DB
• maintainenance of data feeds from DW (if necessary)
• testing feeds
• coordinating and testing schema changes
• creating sample source code for developers interested in integrating applications with ldap.mit.edu services
• creating documentation for developers interested in integrating applications with ldap.mit.edu
• developing improved monitoring tools to ensure that ldap.mit.edu is available and providing consistent responses
• provide 2nd teir support to determine problem solution when a user or developer complains about availability, data consistency, or other data issues