We're developing several SOAP-based web services, which are for the most part stateless query-based services which allow for a simple and straightforward test plan. Current test realms are:
Functionality
A range of known good and bad queries will be made; output should be functionally invariant, although if any of the backend data relevant to one of the canned queries changes there will obviously be variance. The goal here is to detect a crash in any of the backend mechanisms or massive database hosage.
Performance
A series of increasingly demanding query sets designed to test for robustness, scalability, and breakdown points.
Conformance
Under the assumption that there will eventually be formal specifications for these services, this will run a series of queries which push at the edges of same and are designed to validate conformance thereto.
Security
In form this is similar to the functionality testing, only we plan a query set which is designed to break or confused downstream input parsers (overflows, SQL injection...) in as many ways as possible. Some low-level protocol manipulation is certainly within scope here, since all of the services in question do their authentication and authorization against a hardcoded certificate list, and there are 2^n ways to misconfigure SSL services in that realm.