Throughout last year the concept of crowd testing in software development began to grow in popularity with companies like Google, Amazon and eBay.

Advocates point to the speed of it as well as the low cost. The idea of many heads being better than one does have sound logic to it and this approach has definitely worked when it comes to creative ideas, but I do feel that it also creates many problems when it comes to testing, which is a highly skilled and very specific activity.

So here are my top 5 bug bears (or points to think long and hard on) about crowd testing:

Trust and approach

Well-functioning development and testing teams should work together towards the same goal in an atmosphere of trust and mutual support. Without this atmosphere most development projects, particularly agile ones, will fail. It takes time to build trust and employing different random testers through a crowd approach does not allow for this trust to be built.

Communication

Hand in hand with trust goes communication. If you don’t know the testers, communication around expectations, parameters, feedback etc – all very important in successful QA is very hard to get right. For a start there will be many communication layers to get through; developer tells project manager- who tells crowd testing company- who emails testers – meaning many chances for key messages or points to be lost.

Creating a silo

The crowd testing approach is also too siloed for me. Crowd testers are so removed from the actual developers that their feedback does not really reach them. So how can the development team learn from its mistakes, understand them and improve? Without the close and regular input of a QA team, developers risk repeating the same mistakes. Worse still, without that personal, integrated relationship with the testers, perhaps the developers will eventually care less about the quality of their code, as it is a faceless crowd that checks or “judges” their work.

Understanding the context

In my experience, QAs working on a particular piece of software need to have a thorough understanding of the project and its aims, in order to make sensible decisions. While this understanding doesn’t take too long with small projects, some I have worked on can take months to fully understand. With just a couple of days to complete all the testing in, I seriously doubt testers involved in crowd testing are able to devote sufficient time to fully understand the goals and nuances of the software they are testing.

Quality of the software itself

Crowd testing purports to deliver feedback on hundreds of issues in just 2 days. Certainly this is much, faster than what the average QA team can deliver, but the fact that so many issues can be detected raises some worrying questions about the software that went to the crowd testers. How can there be so many issues? This implies that the development and testing process is very poor. Such poor quality software indicates that the developers and project managers working on the software really don’t care about the quality of their work. This means that there could be some real root problems in the company which simply outsourcing the testing to crowd testers is not going to fix.