Crowdtesting: Quicker, cheaper way to test software

BANGALORE, INDIA: Ever wondered how the user centered designs (UCD) and agile development have changed the software testing scenario lately?

Recently, a renowned travel and tour advisor company designed a travel assistance app for the users. They completed the initial round of usability and functionality checks with the testers, but they were still not very confident to go live with the app. Knowing that the app is going to be used by millions of people on a huge number of devices, the main challenge was to verify the app's performance across different devices.

The recent flurry of mobile devices has made it virtually impossible for a single software testing company to ensure the usability of an app across all combination of hardware and software as well as across diverse user base. The alternative testing methods such as crowdtesting are getting more useful in this scenario, which addresses the limitations of the traditional testing methods by bringing the effectiveness of crowdsourcing and cloud platform together.

Crowdtesting is usually done by a freelance community of testers and users from diverse backgrounds, domains and geographies. The process is fast and cost-effective, and more reliable - as the testing is done on diverse and realistic platforms.

Mayank Mittal, director and India head of the Swiss crowdtesting plaform provider, Passbrains, says: "The concept of crowdtesting is relatively new to India, but it is very useful for testing the mobile and web apps. It is mostly used to test things, when you have done with one round of testing in house and want to open up the platform to check the usability and compatibility. However, there are cases where it has been used for to check the viability of ideation and innovation as well.

"The platform is scalable, quick to set up and easy to use, and provides the direct user perspective. Testing methods and parameters can be set according to the project requirements. The test cycles get shortened at least by two to three times."

This testing method works very well for the start-ups. A specialized kind of testing can be done at 30-50 percent cheaper cost structure and you can get it by running a test cycle for just two to three days. Crowd testing can be considered as an ad-hoc testing process, if you are looking to weigh the consumer sentiment regarding any particular product before the release. This complements the in-house testing and works as substitute for outsourced testing too.

Crowdtesting is getting popular among enterprises as it takes them closer to the users. Besides enterprises can get a cloud based platform for testing, a globally distributed workforce to work with.

"The number of mobile devices and applications are growing steadily. Not all the enterprises can simulate a crowd testing type of environment in their organizations. Therefore, it always makes sense considering a partner who can help you build the same. We also provide the platform to the enterprises employees for tehir specific testing needs," says Mittal.

But, how comfortable are the enterprises with this testing environment? Doing crowdsourcing is very easy, it's just like running a contest, but to take it to the enterprise level is quite a challenge. Selecting right people and building a process layer are very important for this.

Mittal adds: "We often face credibility issues, as we source the testers from outside. To allay apprehensions, we create private restricted zone for testing, and use synthetic data to keep the data confidential. The users of the community are also bound by an NDA. There is a defined workflow and best practices for the test management system. Once the project is assigned, the test progress is closely monitored by the concerned projected managers."

The virtual community of user testers is increasing fast as crowdtesting companies keep encouraging people from different domains to join the community.

However, it is not very easy to maintain a large pool of freelance testing resources. As Mayank says: "We have a community of close to 25,000 users inclusive of 3,500 hardcore testers, and subject matter experts. Sometimes, it might not be possible for us to provide projects to each and every members of the community regularly. We engage the community through various thought leadership and knowledge sharing programmes, contests, training, exposure to different domain areas, certifications and rewards, etc., so that they enjoy being part of the community."

Payment methods

There are different type of incentives and rewards based on the projects.

* Payment on the basis of the issues generated - if they get validated testers get paid.

* Payment on the basis of criticality of the issues reported.

* Participation fee to work on specific projects.

* Recurring engagement or hourly payment for the champions.

Chances of getting hired are also high for the testers with brilliant track records.

So, how does the future of crowdtesting look like? The concerns around crowdtesting regarding security, productivity assurance and SLA are still there. But there are areas like banking, telecom, media, education, gaming, social collaboration and Internet-related businesses, where one needs to deal with the users directly, crowdtesting is the best way to get a quick feedback for real-world implementation.