You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 24 Current »

This is a DRAFT.  This is going to be the main document, which can then be turned into slides for a presentation

1 LMOD Introduction

Learning Modules is a replacement for Stellar.  (History Next Generation LMS Evaluation)

LMOD offers:
•    A customizable environment in which faculty tailor the system to include only the modules they require
•    User-focused, intuitive workflow via an easy-to-use, seamless interface
•    Role-based personalization
•    Accessible APIs that allow faculty to leverage module functionality from other platforms

Learning Modules is modular in two directions:  "Horizontally", in that different modules have different functions.  Membership, Gradebook, Materials, Forum, Calendar are all separate systems.  They are integrated, with Membership forming a "hub" and other modules talking to Membership.  The other direction is "Vertically", with the back end consisting of a series of APIs, and the front end consisting of HTML/Javascript which uses the APIs to render the UI.  That means the APIs used by the UI are the same ones exposed to third parties, and there is no "hidden" functionality not exposed by an API.

This presentation will go through general authentication and authorization for Learning Modules,  and then drill down into the APIs for each module.

2 Authentication/Authorization

2.1 Unauthenticated Access

The LMOD APIs support unauthenticated access, although in most cases the information available is very limited. 

2.2 Authenticated Access

There are two ways we support authentication: via Touchstone (Shibboleth), or via application certificate.  Note that some APIs do not (yet) support application certificate; this will be noted below. 

2.2.1 Touchstone Authentication

If the consuming application is a web app and the end user is present when the request to the API is made, then Touchstone authentication is the preferred method.  The user will authenticate to the API (possibly via the application) as themselves, which guarantees that the requesting user is present.  The consuming application therefore does not need to be trusted to impersonate a user; it only needs to be trusted to handle user data while the user is there. 

There are two ways for an intervening app to allow a user to do Touchstone authentication:

2.2.1.1 Touchstone Delegation

One is via Touchstone delegation, in which the user presents credentials to the app, and the app forwards these credentials to the LMOD api.  We have not used this method ourselves but it should work (or can be made to work).  It may require that the Touchstone identity providers be configured to white-list the consuming application.

2.2.1.2 Cross-Origin Requests

The other way to authenticate a user to Touchstone in a web app is using cross-origin AJAX requests.  The web page served by the application tells the user's browser to contact the API directly via AJAX.  This requires that LMOD's servers white-list the consuming application for cross-origin requests.  The AJAX requests need to set the withCredentials flag to true.  Additionally, the application may need to get the user to authenticate to Touchstone and then redirect back to the application.  We support this by providing a login redirector.  Say the app is located at https://google.com.  Then the url https://learning-modules.mit.edu/Shibboleth.sso/Login?target=%2Fcrossdomain%2Floginredirect%3Fhttps%253A%252F%252Fgoogle.com will authenticate the user to learning modules and redirect them back to google.com (note that "https://google.com" has been percent-encoded twice to make it the authentication and cross-domain redirects.  We have used this method inside our student dashboard code.

2.2.2 Application Certificate

If the consuming application is not a web app, or if the user is not present when the request to the API is made (e.g., the app needs to cache information about users ahead of time), then application certificate authentication is more appropriate.  In this case the application authenticates to the API as itself, and can only ask for information on behalf of a user if it is allowed to impersonate that user.  LMOD can white-list certain accounts to impersonate users.  Note that technically the impersonation and the application certificate authentication method can be done separately.  If you get a touchstone account (e.g., a collaboration account) for your application, you can use shibboleth authentication as above and still impersonate whomever you are allowed to.  Also, an application certificate might not have impersonation access, in which case it can still access LMOD APIs as itself, with whatever permissions it has. 

Again, some APIs do not yet support application certificates.

2.3 Roles and Authorization

The membership module maintains "groups" as hierarchical contexts for roles.  So, for example, a user foo@mit.edu may have the role of Instructor in the group "/course/5/sp15/5.678" which corresponds to a course.  There may be subgroups corresponding to sections, and the user foo@mit.edu would have the Instructor role in all these subgroups as well.  (Inheritance of role is dependent on role and group type.)   Each other module maintains its own mapping from role and group to permissions.  There are system-wide roles as well.   Generally the modules will grant sysadmin privileges to any user who has the membership System Administrator role in membership, but again this is module-dependent.

3 External Data Feeds

The membership service talks to warehouse, ESAPIS, and LDAP. Currently we get Academic terms info, new MIT krb account, course catalog info from the warehouse. The data is updated once a day via a nightly cronjob. It talks to LDAP once a day for person info updates such as names, email addresses, phone number, etc.It updates both MIT persons as well as CAMS account. It talks to ESAPIS for roster data (both reg and pre reg), section data, and section assignment data. The update is on-demand and wecache the data for 30 minutes. We are currently working on getting the student advisor info from the ESAPIS.

4 LMOD APIs

4.1 Membership API

The Membership Service serves as the hub of the LMOD wheel. It provides membership information to other LMOD modules as well as some third party systems, but it is not an authentication and authorization module. For example, the gradebook module can query the membership service for instructors of the course, but the gradebook module is responsible for determining what instructors are authorized to do within the gradebook module.

Membership has five types of groups: system, departmental, course, project and  sections. Course groups can be provisioned under departmental groups and section groups can be provisioned under course or project groups. Project group can be associated with a departmental group, but doesn't have to. We also have a set of roles like courseAdmin, departmentalAdmin that can be applied to different group types. Each role is associated with a defined set of permissions like VIEW_GROUP and ADD_MEMBER. The roles and permissions flow down from parent to children. For example, if somebody is added as a deparmentalAdmin to a department, that person will have admin permission to all the courses under that department and all the sections under those courses. Members in system groups have permission in all groups in membership.

In addition to roles and permissions, groups can also have access levels like class, mit, world, staff which also contributes to permission control. Class means the group is only viewable to members in the course. MIT means the group is viewable to anybody with a MIT ID. World means it is open to anybody and staff means it is only open to staff members of the course. A group can also be archived. Once the archived flag is set to true, any modification of the group (except for unarchiving) is disallowed.

•    Within the LMOD LMS:

1.    Provides a simple and user-friendly method to centralize and manage course membership information sourced from MITSIS as well as staff membership

a.  Provisions course groups

b. Integrates with registration data and pre-registration enrollment data

c. manage Instructors/Course Admin/TA/Grader/Guest membership

2.    Provide grouping/section support

a.  Integrates with MITSIS section data and pre-reg section assignment data ( (i.e., recitations, lectures, labs and design section assignments).

b.  Admins have the ability to “section” membership into ad hoc groups.

c.  Option to assign multiple section (types) to any given student.

d.   Ability to apply metadata such as max size for individual sections.

e.   Students get a straightforward interface from which to choose and/or switch sections if course staff allows students to switch sections

3. Provide support to manage membership for system wide groups (System Administrators, helpdesk staff, e-reserve staff, ocw staff etc. ) departmental groups (departmental admins and departmental read-only access), and project groups.

4. Provides student photos

5. provides academic term info

6. provides advisor/advisee info

7. provides person info

8. All API that require a user context support impersonation.

4.1.1 Example

https://learning-modules-test.mit.edu/service/membership/group?uuid=/course/10/sp15/10.992 (get group info for /course/10/sp15/10.992)

https://learning-modules-test.mit.edu/service/membership/group/90885/member?role=Student (get student roster for /course/10/sp15/10.992)

https://learning-modules-test.mit.edu/service/membership/group/90885/groups?groupingScheme=recitation (get sections /course/10/sp15/10.992)

https://learning-modules-test.mit.edu/service/membership/studentphoto/90885/40408?size=small (get photo for student id 40408 in course /course/10/sp15/10.992)

https://learning-modules-test.mit.edu/service/membership/academicterm/2015SP (get term info for 2015SP)

https://learning-modules-test.mit.edu/service/membership/advisees (get advisees for the logged in user)

4.1.2 Intended Use Cases
  1. provision groups (system, departmental, project, and course)
  2. Retrieve group membership or fiter membership by roles (Student, Learner, Instructor, TA, CourseAdmin, etc)
  3. get groups which the current user is a member of
  4. create child groups (sections) based on grouping scheme (lecture, recitation, lab, design, or custom) and apply metadata such as maxSize on sections
  5. retrieve reg and pre-reg student roster from MITSIS
  6. add/delete/edit non-student membership 
  7. assign students to sections or import mitsis section assignment
  8. retrieve person info
  9. retrieve student photo based on privacy agreement
  10. get advisors for current user
  11. get advisees for current user

4.1.3 Pitfalls

We cache the student roster info for 30 minutes. If somebody gets added to class in MITSIS, it may take up to 30 minutes for the update to show up in membership.

https://learning-modules-test.mit.edu/service/membership/doc.html

4.2 Course Guide API

provides a set of courseguide apis which provides a complete listing of all courses at MIT as well as courses in LMOD.The apis are open to public and doesn't require authentication.

4.2.1 Example

https://learning-modules-test.mit.edu/service/membership/courseguide/departments : list all departments

https://learning-modules-test.mit.edu/service/membership/courseguide/department/1/mitsiscourses?termCode=2015SP: list all courses offered by department ID 1 for term 2015SP in MITSIS

https://learning-modules-test.mit.edu/service/membership/courseguide/department/1/courses?termCode=2015SP: list all courses under department id 1 for term 2015SP in membership

https://learning-modules-test.mit.edu/service/membership/courseguide/course?uuid=/course/1/sp15/1.00: view info for /course/1/sp15/1.00

https://learning-modules-test.mit.edu/service/membership/courseguide/course/86982/staff: view staff info for /course/1/sp15/1.00

 

4.2.2 Intended Use Cases

see API examples.

4.2.3 Pitfalls

unknown

https://learning-modules-test.mit.edu/service/membership/doc.html#CourseGuide

4.3 Gradebook/Attendance API

Offers advanced grade management options, integrated student photos, recitation and section support. The latest release of this module integrates the newly designed Assignments component, which integrates on-demand assignment creation and submission capabilities within the Gradebook Module interface. 

Attendance Module: provides a centralized means for tracking and grading student attendance via a customizable calendar.

The Gradebook API is organized around objects including Gradebook, Assignment, Student, Grade.

Gradebook:

Gradebook is the overarching object by which all other objects are contained.

 

Assignments:

There is a hierarchy of assignments within a gradebook.

SimpleAssignment:
The most fundamental is a simple assignment which maps to an actual class assignment. Simple assignments can be simple holders of grades, or they can be homework assignments, and have associated materials stored in the materials service, including student submissions.

GradingSchemes on simple assignments: assignments can be set to us numeric grading schemes, letter grading schemes, or in the case of homework assignment, arbitrary values or phrases.

 

AssignmentCategories & Root Assignment:
SimpleAssignments are grouped by AssignmentCategories. The root assignment in a gradebook is parent to all categories, and is where the student's overall grade is stored. The rules by which grades for categories and the overall grade are calculated are determined in part by AssignmentAggregation settings -- e.g. they can be weighted average, normalized average, or numeric totals.

Assignments as calendar events: when assignments are created or updated, there is background communication with the calendar api, so that this information shows up on user calendars.

 

Students:

Calls to get students get basic student information, and can also get overall grade information, and assignment grade information (one or all assignments, one or all students).

 

Groups:

Gradebook group information, including staff and students, is gotten from the membership service. Most calls can be limited by section, adding /section/{groupId} to url, e.g. /assignments/{gradebookId}/section/{groupId}

Grades:

Grades can be set manually on student-assignment pairs. They are by default derived as you move up the assignment hierarchy (ie, category grades, and overall grade), though these also have the option of being manually set.

The grading scheme associated with the assignment determines the types of values that should be submitted (e.g. numeric, letter).

At the assignment level, grades are associated with submissions on assignments. This means that a "make-up" grade can be a set as a second submission, and will be taken as the grade. Only one submission is marked as default -- the most recent, unless otherwise set.

 

ExcelExport/ExcelImport:

There are a several calls that allow the export of information to excel, and some of which allow the import from excel to update information in gradebook.

 

Timeline:

The timeline calls are used by the you@lmod UI, which is replacing the you@Stellar interface. This provides remidners and notifcations such as assignments being due or posted, restricted by date range provided

 

4.3.1 Examples

To test these calls, you must login via touchstone. Redirection to touchstone will not happen automatically. If using calls via resttest.html, use LOGIN link at top of page.


GRADEBOOK:
Get Gradebook
https://learning-modules-dev.mit.edu/service/gradebook/gradebook?uuid=STELLAR:/project/gb-training&autocreate=true&skipValidityChecks=false

Get/Set Gradebook options
https://learning-modules-dev.mit.edu/service/gradebook/gradebook/options/2586 (GET/PUT)


ASSIGNMENTS:
Get all assignments for a gradebook
https://learning-modules-dev.mit.edu/service/gradebook/assignments/2586?includeMaxPoints=true&includeAvgStats=true&includeGradingStats=true
Note: to get grade data, look in the get students calls; this only provides details about the assignments themselves

Get one assignment
https://learning-modules-dev.mit.edu/service/gradebook/assignment/4522?includeMaxPoints=true&includeAvgStats=true&includeGradingStats=true

Get categories for a gradebook
https://learning-modules-dev.mit.edu/service/gradebook/assignment/4522?includeMaxPoints=true&includeAvgStats=true&includeGradingStats=true


STUDENTS:
Get all students with overall grades as well as grades for all assignments. Special value of 1 is used to represent all assignments. To get information for just a single assignment, pass the assignment id where the 1 is. Leaving off assignment id altogether will get just the basic student information, and no assignment-specific grade info. includeGradeInfo determines whether or not to return the Overall Grade info for the student.
https://learning-modules-dev.mit.edu/service/gradebook/students/2586/1?includePhoto=true&includeGradeInfo=true&includeGradeHistory=true&includeDroppedStudents=true&includeGradingSchemeDetails=false&includeAssignmentMaxPointsAndWeight=false&includeCompositeAssignments=false&includeAllSubmissions=false&convertGradesToParentGradingScheme=false

 

4.3.2 Intended Use Cases

Course assigns grades to assignments.

Course assigns grades to assignments, and also provides homework assignment content, and means for students to make submissions.

PE manages attendance, and determines grades based on attendance thresholds they set.

 

4.3.3 Pitfalls
  • Membership group changes take approximately 15 minutes to propagate to gradebook.  Sometimes the first access to a gradebook in a long time will get "stale" permissions since unused gradebooks are not agressively synced to membership. This may mean that after a permission change the consuming application may need to retry a query before giving up.  The 15 minutes can be shortened if a sysadmin manually kicks off a membership sync for a group.
  • Similarly, membership user changes take approximately 6 hours to propagate to gradebook.  This is largely unnoticeable since one can always query membership for up-to-date person metadata.  The only time this would be an issue in gradebook is if a user has a new system-wide privilege change.  In this case, a new sysadmin may need to wait 6 hours to be able to do sysadmin tasks in gradebook.  The 6 hours can be shortened if a sysadmin manually kicks off a membership sync for a user.

https://learning-modules-dev.mit.edu/service/gradebook/doc.html

Test Page Link: https://learning-modules-dev.mit.edu/service/gradebook/resttest.html

 

4.4 Calendar API

The Calendar module provides a user-friendly interface for managing and displaying custom-created and time-specific events, as well as events listed on the Institute's academic calendar.

The Calendar module exposes the Google Calendar API via our own API underneath the /service/google-apps-proxy/calendar endpoint.  We have our own Google Apps for Education domain corresponding to learning-modules.mit.edu, and we map LMOD accounts onto google user accounts via a name translation scheme.  For example, jcalz@mit.edu in LMOD becomes jcalz-at-mit-dot-edu@learning-modules.mit.edu in Google.  The detail of this are not really that important since the only service we are currently exposing is Calendar.  We also map LMOD group/role pairings onto groups in Google (this may change soon).  When we provision calendars for a group, we assign google groups so that staff roles have "writer" permissions on the calendar, and student/guest roles have "reader" permissions.  Additionally, the gradebook and materials modules add events to a group's calendar corresponding to relevant dates (assignment due / posting dates).  These events have extended properties on them (event.extendedProperties.shared.lmodSystemManaged=true) so that they can be handled programmatically by consumers

4.4.1 Example

https://learning-modules.mit.edu/service/google-apps-proxy/calendar/v3/calendars/mitacademiccalendar@learning-modules.mit.edu/events?singleEvents=true&orderBy=startTime&maxResults=1000

4.4.2 Intended Use Cases
4.4.3 Pitfalls
  • Supports application certificate authentication but not impersonation.  This means that the application certificate account itself needs to have access to the calendar in order to read or write.
  • Supports unauthenticated access by mapping unauthenticated users to guest@learning-modules.mit.edu.  This is a workaround since the Google calendar API itself does not support unauthenticated access.  The guest account has access to all public calendars.  It is considered a misuse of the API to do any writes as the guest account.
  • The dev and test tiers have separate google apps domains, and the data is not copied from production to dev or test. 
  • Users cannot authenticate directly to their google apps account via the google ui, since the password is a strong random value and we do not have touchstone enabled.  We currently only support authentication via the above proxy endpoint.
  • Right now when course access settings are changed in membership (World / MIT-Wide / Class-Only / Staff-Only and Archived status) the calendar needs to be reprovisioned manually to pick up permission changes.  Group membership changes (e.g., person is added to or removed from the group) are picked up automatically, with no appreciable delay.

https://developers.google.com/google-apps/calendar/v3/reference/

4.5 Materials API

Materials module offers course administrators a flexible method of populating, organizing and displaying course materials, as well as controlling access to copyrighted and/or time-sensitive content.

  • Supports access based on class membership, staff-only, MIT-wide and world.
  • Support for bulk delete and updates.
  • Support for html5 pseudo streaming
  • Integration with Calendar: Materials author can post materials events to calendar
  • Integration with Membership: Gets course information, course membership information and person information from Membership APIs.
  • Integration with Gradebook:  For homework, Materials stores assignments, solutions, submissions and submission comments. Materials delegates to GB for permission and authorization. Materials also provides homework timeline view for dashboard module.
4.5.1 Example
  1. List materials for a course: https://learning-modules.mit.edu/service/materials/groups/47994/files
  2. Get materials: https://learning-modules.mit.edu/service/materials/groups/47994/files/476d4ca0-097b-4967-879e-d026d48adb29
  3. Export site: https://learning-modules.mit.edu/service/materials/groups/47994/materials/exports
4.5.2 Intended Use Cases
  1. Upload course material
  2. View materials based on various filters - visibility, copyrighted, archived, staff-only
  3. Organize course materials
  4. Export / import course materials
4.5.3 Pitfalls
  1. Supports application certificate authentication but not impersonation. 
  2. Requires membership API and gradebook API to be available. Does not cache information from these API.
  3. Not yet SCORM compatible
  4. No support "yet" for transcode of audio and video files from one format to another.

https://learning-modules-dev.mit.edu/service/materials/doc.html

4.6 Forum/Announcement API

The Forum Module offers collaboration and discussion capability within the framework of the course.

We have an instance of phpBB in LMOD, which we use for both forum and announcement functionality.  Once a forum is provisioned for a membership group, it grants permissions to the forum based on the role.  Basically staff get read/post/moderate access, students get read/post access, and guests get read access.  Forums are given a name corresponding to the membership uuid.  Each slash represents a child forum.  So "/course/5/sp15/5.678" would be in the forum named "5.678" under a forum named "sp15" under a forum named "5" under a top-level forum named "course".

Initially we used the built-in phpBB UI but we have been adding APIs on top of it.  Currently we have a forum api which only gets/lists forum posts, which can be used for a read-only view into the forums. 

Announcements are "sticky" posts within a forum.  The API is read/write.

4.6.1 Example

https://learning-modules.mit.edu/service/forum/announcement/

4.6.2 Intended Use Cases
  • overview of forum
  • make and view announcements
4.6.3 Pitfalls
  • Does not (yet) support application certificates or impersonation.
  • Does not (yet) support making posts in forums.
  • The dev and test instances are not synchronized with the production data.
    The forum/announcement content is stored as BBCode, a version of a subset of HTML.  That means that if HTML is sent via the API (post_html attribute), some of the content may disappear.  This is probably a good thing in general (e.g., script tags) but it can be surprising.  Where possible, users should send BBCode via the API (post_bbcode attribute), since this will at least be stored as-is.
  • Forum group membership and permissions are synced with membership once an hour.  This is not particularly easy to force manually but it can be done on the phpbb server.    
  • Forums are not currently archived when the membership group is archived.
  • Checking and/or provisioning forums can be done via one-off API calls which are not documented and should be cleaned up before anyone uses them.

Find forum posts: https://learning-modules.mit.edu/service/forum/findPosts/doc/

Announcement API: https://learning-modules.mit.edu/service/forum/announcement/doc/

4.7 Odds and Ends

  • No labels