Beyond ISEB ISTQB-BCS: User Acceptance Testing (UAT) for Software Systems Part 1
This is the first part of a 2 part article on User Acceptance Testing (UAT) of software systems. It follows my previous articles on Quality Management topics.
Up to this point no-one outside the development and QA teams have test driven your new software system. You've proven that there are no glitches in the software by thoroughly unit and function testing it, you've proven that it is well integrated with integration testing, and you've even proven that it meets the requirements specified for it (or at least the QA groups interpretation of those requirements), but you still haven't passed the ultimate test: do the users like the system? Customer acceptance is critical when building a commercial product and still important when building an in-house system. New software systems are usually introduced to replace existing systems that users have become accustomed to. Changing the way the user does his or her work requires you to overcome the resistance to change that is a common human trait. Overcoming that resistance is made easier when the new system is demonstrably better than the old one. User Acceptance Testing is your opportunity to demonstrate that superiority. User Acceptance Testing is also your chance to eliminate some of the warts from the system before springing it on the general population and demonstrate your responsiveness to their concerns.
User Acceptance Testing is approached differently depending on whether the users of the system are internal to the organization, an external organization who have paid for the project, or individual external customers who buy a copy of the system or a license to use it. User Acceptance Testing is usually used to verify the system with internal users and external customers who sponsor the project. Testing by individual customers is most often referred to as Beta testing and requires a different approach when engaging the testers.
There are 2 goals this testing should accomplish: to demonstrate the system's quality to the user community and to improve the product so that it is more acceptable to users. User Acceptance Testing should never be used to detect programming bugs, integration bugs, or design flaws. These should be detected by the previous testing phases: development testing and Quality Assurance testing. To achieve these goals UAT must be conducted in a production-like environment and the system exercised by at least one representative of every functional group in the user community.
Enlisting Beta Testers
Beta testing of a new product should be part of the marketing campaign for that product. The testers are enticed to take part in the testing program by some form of inducement. A standard part of the inducement should always be that the beta version is replaced by the production version at the end of beta testing, unless the customer chooses to keep their beta version. Other inducements might be a lower price, the opportunity to have input into product design, or a competitive advantage from having access to the beta version. Beyond targeting inducements at the various market segments, the organization developing the system does not have the ability to force customers into becoming beta testers.
Enlisting User Acceptance Testers
The users responsible for conducting User Acceptance Testing will be the responsibility of the customer organization in the case where you've built a system for a single external customer. It is in their best interests to ensure that coverage is complete and they are confident in the system before signing off on the contract. Your responsibility is to ensure that the users identified to you by the customer are provided with all the tools they need to conduct their tests.
You do have more influence over the user community in your own organization than users of external organizations, but will probably need some enticement to enlist users onto the test team. You should discuss your enlistment approach with your project's business sponsor; this is likely to be a person who has authority over the user community and can help you with your enlistment campaign.
From your perspective the inducements are obvious: the system you are producing will replace an antiquated manual system, or offers great new features that the old software system doesn't offer, which will make the users work much easier. UAT will eliminate any problems not detected by development testing or QA testing, rendering the perfect system to production. These advantages are not all that clear to the user community. The users have yet to see proof that your system will deliver the promised improvements and besides, they have been getting along nicely with the existing system. They may also believe that your job is to deliver the perfect system to them and you shouldn't need their help to do so. After all, they don't ask you for help to do their jobs.
The principal objection most users will have to participating in your testing is that they must accomplish the work assigned to them by their bosses plus do your UA testing. The best way of offering an effective inducement to overcome their objection is to change this dynamic so that the tester is not made to suffer as a result of participating in testing. The simplest way of accomplishing this is to take the testers out of their functional organizations and have other resources cover for them. You will definitely need the support of your business sponsor to accomplish this and should make plans for the transfer of the users to the project as part of your Quality Management Plan. This will add cost to the project but will ensure that you have a committed group of user/testers.
Training is always a desirable feature of any new system. Training should be a part of the work of the project; without adequate training the new system will be a hindrance rather than a help. To induce the user community to participate in testing make the training a part of the User Acceptance Testing program. The 2 activities are not all that different. The users should receive training before using the new system and testing can be made part of "hands on" training. Combining these two inducements can make participation in UAT especially attractive to users.
Author: Dave Nielsen