Page History

Lessons learned notes (SAPbiz session held 02/09/2011)

Project Organization

• Steering Committee
  • Important that Steering Committee not be 99% IS&T.  Having additional Business worked well.

Planning/Schedule

• System Preparation
  • QC setup – For some business users, preparation to use the tool was last minute, should have been set up earlier.  People in VPF are not administrators of their machines and cannot install themselves – so anything which needs to be installed needs to be scheduled ahead of time.

•  
  • Access to sh3 for testing – the SAP GUI was not up to date for all users.  Need to prepare that everyone has up to date version and access.
  • Prepare that remote access is tested before cutover.

Testing

• SIT/UAT Testing
  • QC
    • Disappointed that the Business not able to edit and control changes on their own.
    • Is there a constraint on the number of people involved?  We now have 25 concurrent licenses.  Concerns over if this is enough during testing.
    • Would like expand the use of the QC with the business.
    • Those who could set up on their own found it initially difficult, once in was ok.
    • Many of the test cases need to be reworked – use SAP biz as a way for training, testing, and organization.
    • Automated testing

• Eager to have more scripts automated (HR:chair titles in particular would be great).
• Really liked having QC as a central repository for test scripts, even if printing them out was a bit of a challenge.
• Would be nice if there were a way to view the automated test scripts directly in QC, rather than having to request a printout from Don.
• New test scripts formats – business needs help with new template.  Training should be sooner rather than later.
• Going forward, coordination should be between functional areas, Business Analysts and QA.
• There was confusion about which Benefits test scripts Diane should perform.  Did she actually need to come up with scripts of her own, or was just showing up to key the HR Payroll transactions sufficient?  Diane needed to be included on Leslie’s list of testers.
• Next year we can have Diane do all of the validation testing in production, as opposed to having Riley do one tiny piece in ESS and Diane doing the rest

•  
  •  
    •  
      • No information concerning the content of the automated tests was shared with the business. 
      • Automated tests reviewed by some business groups, not others.
      • Going forward, automating testing should be part of SAP Biz to expand knowledge and usage.

•  
  • Need more people for testing, especially students.  Maybe we can involve the help desk and their students.  Especially important for testing authorizations and different hardware/software – Mac and PC testing.  Would like the community to get involve a little bit.
  • Testing with SAP alias is not effective, especially for certain apps like APR.
  • Run trial balance before the freeze is introduced.
  • Closing testing on Dec 6 – would like to do earlier to leave more time for other testing.  Want to do it end of November, rather than wait.
  • Calendar date changes cause issues with FI testing – need to test within the correct calendar month.
  • HR-Payroll testing:

Communication

• Testing coordinators communicated with business well, responses were quick.
• If WebEx is used in future, make sure people are familiar with process.

Cutover

• Communication
  • Emails appreciated in place of the phone line.  Fine line with emails – some want more, some want less.
• Execution
  • As expected.

Lessons learned notes (Steering Committee session held 02/01/2011)

Lessons learned notes (IS&T session held 01/28/2011)

Project Organization

• Steering Committee
  • - Based on feedback from last years Support Pack Project the Steering Committee was expanded to include more BPO representation.
  • - The general consesus was this was a good idea as in prior years BPO representation was inadequate.
  • - Should create an email list for committee members.
• Core Team
  • - Worked well; representation was adequate.
  • - Testing coordinators made organizing work easier.
  • - QA representation (a point person) on the team proved beneficial to the QA team and the project as a whole.
  • - Transport coordinator continues to play an important role.
• PM - assign an assistant PM as backup and to learn/understand the process

Planning/Schedule

• Adequate for the project
• Validation by BPO should be tested before cutover weekend (some testers where not the normal executers or the transactions being tested)
• Remote cutover testers should test connections to system before cutover weekend
• There was a process defined to allow certain development work to move with the support project (AP 1042s, AP unicode program changes); this worked well for this project.

Testing

 The Goal of testing is to find issues as soon as possible to avoid introducing defects into production.

• Unit Testing
  • Definition of unit testing can be found on the QA wiki as well as this projects wiki
  • Keep the 5 day unit test window - new functionality has been introduced; the time for testing/resolving issues is needed
  • We used an Access db to store a catalog of the transactions we tested by whom and compared to last year
    • concept was good and has value
    • provided visibility to who was testing what as well as the status
    • the execution was rushed (the db was not really "production" ready)
    • the tool was a little klugy 
    • better solution should be explored (within SAP, within SM)
  • Should consider storing unit test cases in QC to promote a single consistant cataloging approach
  • A lot of discussion around what should be unit tested, how it should be tested and how it should be documented
    • QA will follow up on this effort

• SIT/UAT Testing
  • QC - central repository for catalog of test cases
    • Worked well as a catalog for test cases 
    • Great for monitoring execution status
    • Saved preparation time of test "lab" for BAs (cases already in QC and available for use)
  • Test cases for the most part were not ready for QC formatting
    • Cases are/were designed for printing
    • A focused effort updating the cases to be QC formated would be beneficial to the process (requires resources)
    •  

Transports

• • • )
    • Validate test cases as we build for QC; vet with the business much earlier (not in Oct/Nov) – of course that requires BPO availability.
  • Automation
    • Was successful (though only a limited number of cases were automated)
    • Desire to expand automation for the next SP project (40%)
    • Need to communicate what was tested through automation, how it was tested and what the results were.
    • BPOs need to reach a comfort level with the automation tool and testing results.

Transports

• Transport  Transport management processes executed by the transport coordinator need to be documented.
• Transport freeze
  • IS&T has this processed well defined and understood
  • Could explore another environment to allow development efforts during the freeze (assuming resources are available to do the work during the freeze)
• Critical transport process
  • A consistent approach should be defined for all similar projects (there have been minor differences from project to project)
  • Critical transport process for this project worked as defined.

Communication

• Project wiki
  • Great for dates and status, assuming it is kept up-to-date
  • Was used as reference by all IS&T participants
  • Could be used to report cutover status in addition to status phone line, emails, etc ...
• Status line was used by IS&T staff
• Frequency of meetings
  • Core Team Monthly meetings - adequate and focused
  • Functional BA - daily meetings during testing phases kept teams focused
• Did people feel they knew what was going on and what was expected of them? - Yes
• Other
  • Could utilize WebEx - in light of IS&T moves
    

...

• Communication
  • The Plan itself - well documented, easy to follow, clear as to who was doing what and when.
  • emails- Subscribe to email list of core team during the project/during cutover (watcher list could be added)** phone
  • status phone - was used throughout the weekend by IS&T staff
  • conference line - use WebEx to allow more participants on the call
  • validation steps
• Execution
  • For the most part ran smoothly - went live slightly ahead of schedule
  • Cutover plan contained necessary details for successful execution
• Other
  • Communication Expectation of Sunday attendance by IS&T staff should be communicated earlier in the project to allow staff more time to plan ahead
   
  •