Lessons learned notes (IS&T session held 02/01/2011)

Lessons learned notes (IS&T session held 01/28/2011)

Project Organization

• Steering Committee
  • - Based on feedback from last years Support Pack Project the Steering Committee was expanded to include more BPO representation.
  • - The general consesus was this was a good idea as in prior years BPO representation was inadequate.
  • - Should create an email list for committee members.
• Core Team
  • - Worked well; representation was adequate.
  • - Testing coordinators made organizing work easier.
  • - QA representation (a point person) on the team proved beneficial to the QA team and the project as a whole.
  • - Transport coordinator continues to play an important role.
• PM - assign an assistant PM as backup and to learn/understand the process

Planning/Schedule

• Adequate for the project
• Validation by BPO should be tested before cutover weekend (some testers where not the normal executers or the transactions being tested)
• Remote cutover testers should test connections to system before cutover weekend
• There was a process defined to allow certain development work to move with the support project (AP 1042s, AP unicode program changes); this worked well for this project.

Testing

 The Goal of testing is to find issues as soon as possible to avoid introducing defects into production.

• Unit Testing
  • Definition of unit testing can be found on the QA wiki as well as this projects wiki
  • Keep the 5 day unit test window - new functionality has been introduced; the time for testing/resolving issues is needed
  • We used an Access db to store a catalog of the transactions we tested by whom and compared to last year
    • concept was good and has value
    • provided visibility to who was testing what as well as the status
    • the execution was rushed (the db was not really "production" ready)
    • the tool was a little klugy 
    • better solution should be explored (within SAP, within SM)
  • Should consider storing unit test cases in QC to promote a single consistant cataloging approach
  • A lot of discussion around what should be unit tested, how it should be tested and how it should be documented
    • QA will follow up on this effort

• SIT/UAT Testing
  • QC - central repository for catalog of test cases
    • Worked well as a catalog for test cases 
    • Great for monitoring execution status
    • Saved preparation time of test "lab" for BAs (cases already in QC and available for use)
  • Test cases for the most part were not ready for QC formatting
    • Cases are/were designed for printing
    • A focused effort updating the cases to be QC formated would be beneficial to the process (requires resources)
    • Validate test cases as we build for QC; vet with the business much earlier (not in Oct/Nov) – of course that requires BPO availability.
  • Automation
    • Was successful (though only a limited number of cases were automated)
    • Desire to expand automation for the next SP project (40%)
    • Need to communicate what was tested through automation, how it was tested and what the results were.
    • BPOs need to reach a comfort level with the automation tool and testing results.

Transports

• Transport management processes executed by the transport coordinator need to be documented.
• Transport freeze
  • IS&T has this processed well defined and understood
  • Could explore another environment to allow development efforts during the freeze (assuming resources are available to do the work during the freeze)
• Critical transport process
  • A consistent approach should be defined for all similar projects (there have been minor differences from project to project)
  • Critical transport process for this project worked as defined.

Communication

• Project wiki
  • Great for dates and status, assuming it is kept up-to-date
  • Was used as reference by all IS&T participants
  • Could be used to report cutover status in addition to status phone line, emails, etc ...
• Status line was used by IS&T staff
• Frequency of meetings
  • Core Team Monthly meetings - adequate and focused
  • Functional BA - daily meetings during testing phases kept teams focused
• Did people feel they knew what was going on and what was expected of them? - Yes
• Other
  • Could utilize WebEx - in light of IS&T moves
    

Cutover

• Communication
  • The Plan itself - well documented, easy to follow, clear as to who was doing what and when.
  • emails- Subscribe to email list of core team during the project/during cutover (watcher list could be added)
  • status phone - was used throughout the weekend by IS&T staff
  • conference line - use WebEx to allow more participants on the call
  • validation steps
• Execution
  • For the most part ran smoothly - went live slightly ahead of schedule
  • Cutover plan contained necessary details for successful execution
• Other
  • Expectation of Sunday attendance by IS&T staff should be communicated earlier in the project to allow staff more time to plan ahead