View Source

QA Section: Contents

Thalia's QA Schedule

First half of each Thalia sprint is devoted to development. QA uses this time to write test plans and test cases to cover new functionality, and perform testing on fixed bugs and new features as they are released to the staging platform. Bug reports are converted to new test cases, as they indicate an area that needs coverage. The second half of each Thalia sprint is the QA phase. When the candidate build has been put up on the staging platform, QA begins regression testing to confirm that all functionality still works as expected.

Development Phase

As each build is released to QA:

Smoke test to confirm integrity of basic functionality:
- test create, view, update, and delete of all components
Verify and close defect fixes with each new build:
- Close fixed JIRA issues
- Reopen JIRA issues as needed.
- Write additional test cases for new bugs.
Test new functionality:
- Open JIRA issues as needed.
- Write test cases for new functionality.

QA Phase

When final candidate build is released to QA:

Initial smoke test of major functionality.
Regression test - hunting for bugs in previously working areas (use test cases to guide this; log results in the test log).
Re-test bugs fixed during the current sprint - make sure they are still fixed.
Open new JIRA issues as requiredExploratory testing - find creative new ways Thalia can go wrong.

Acceptance of the Release

Does the release candidate work at least as well as as the current public version?
Do the new features work the way they are supposed to?
Are blocker and critical bugs fixed?
Have customer communications regarding the new release date been sent out?
Smoke test of the product after the release

Customer Communications

First notice to customers goes out one week ahead of the expected release.
Second notice to customers goes out 24 hours ahead of expected release
Third notice to customers 24 hours after release - if smoke tests go well.

Explanation of Tests

Usability Testing is periodically performed in consultation with the Usability lab. Does Thalia serve customer needs and abilities?
Sanity Testing determines whether it is reasonable to proceed with further testing. Used particularly for new features that may still be very buggy.
Smoke Testing a minimal test of create, view, update, and delete of all components and is preliminary to further testing. Used for each new release to test, and after each public release.
Regression Testing is used to determine that changes have not caused unintended side effects: do the unmodified parts of the system still work as before?
New Feature Testing to determine if newly added functionality is working, without testing the entire application.
Acceptance Testing to determine if the product is acceptable and ready for release.

Test Case Archive

Test plans change with each version of the product. They also provide documentation about product functionality which would help a new programmer or tech writer ramp up. This makes it as important to keep test plans in a versioned, central location with the code. Thalia's QA plans may be found with the code in SVN at svn+ssh://svn.mit.edu/zest/thalia/qa . You must have access to SVN to access the repository.