SoftEd wrote a blog post about UAT and how hard it was (here). I gave a longish reply and thought it might be good to re-iterate my thoughts on User Acceptance Testing (UAT) here on the blog.
I think the primary premise of what UAT should be, that we have here in Wellington/New Zealand, is wrong.
Most of our industry does it wrong. Why?
Because testers influence UAT too much!
UAT = User Acceptance Test
To me that means the actual user (not to be confused with stakeholders or managers) must assuage if the software developed is OK for him/her to use.
That means:
- That the application actually works, as proven by previous test stages (so not the actual mandate for UAT)
- UAT coverage is small and mainly UI – But that’s what most of these actual users actually see (Pareto principle).
- They should be doing what they do day to day – Because the corner cases should have been covered by Functional/System Testing and been proven correct (see next point).
- UAT is NOT supposed to find ANY defects/bugs! – The things that don’t work for them should be change requests. I.e. the primary requirement was at fault and not the software.
- Testers should not be involved in UAT – They are not needed, it’s not a testing phase (confusion in the name!). It is a milestone for requirements and their correctness. It should not be around the actual acceptance of the software, that should be a given by that point.
Because testing always gets pushed to the end (in waterfall projects at least) we look for where we can regain lost time. I think over the decades we have pushed our testing regime into UAT, until it became a test phase.
This is subverting the actual reason for this SDLC phase. As stated above, this phase proves Fit for Purpose (FfP). That answers the question “Is the software usable form a day-to-day business perspective”. If that FfP is influenced by open defects that’d be a catastrophe! As this phase is situated right at the end of the project, this is where fixing defects is the most expensive (bar production). If your project is still fighting defects at this point you should start questioning your project timeline. There is likely a severe issue with your SDLC/management/governance.
Things found in UAT should be functional or systematic changes that need to be made to future releases. I would not see them as fixes. In the context of the project the things UAT should find are changes to requirements or added requirements (stuff that has been missed).
The SoftEd post talks about why UAT is the hardest testing. I think that actually misses the point and enforces the bad habit of seeing UAT as a testing phase, with testing methodologies, testing oversight and control.
I stipulate that all that will achieve is…
- do more system testing on a much higher and therefore worthless level (at that point of the project)
- with people who aren’t even testers
- totally miss the point of highlighting the inadequacies of the software needed for actual implementation and use by the customers/users
What has happened seems to be, that we have actually lost (without replacement) a vital stage of the Go Live process. The software process improvement loop. It has been shoehorned into a testing phase. I don’t want to question that there might actually be another test phase needed that does what we do in UAT today but we should then not lose the actual intention of UAT and the phase associated to it.
Author: Oliver Erlewein
