'There is growing maturity and adoption of Open Source tools for testing'

BANGALORE, INDIA: Interaction with Rajesh Sundararajan, Practice Head, Testing services, Marlabs Software.

Excerpts:

CIOL:  Can you explain the concept of cloud based software testing

Rajesh Sundararajan: As we all know, "cloud computing' refers to the Utility based model of consuming shared computing resources as a service over the Internet. Let's see what it means in the context of software testing.

There are 2 primary categories of computing resources that can be accessed over the cloud:

I.Application test environments:

The test environment infrastructure can be provisioned on the cloud with the appropriate configuration for the required timelines rather than have a dedicated infrastructure in-house. The Application under test is then hosted on this cloud infrastructure and tested. This is an example of IaaS (infrastructure as a service) or PaaS (platform as a service).

II.The Testing tool rig: which includes the Testing tool (Automation, Performance etc.) software and/or infrastructure required to host the testing tools - can be provisioned and accessed from the cloud. This can be SaaS (Software as a service), PaaS or IaaS depending on the Test setup. This testing tool can test applications hosted on either cloud or dedicated in-house test environments.

CIOL: What are the benefits of testing and running enterprise software in the cloud?

RS: The Benefits if planned well are significant.

Cost is a primary factor:

I.Large upfront costs of CAPEX like Servers & other Hardware, Testing tools are replaced by OPEX which are spread out over a period of time.

II.With Cloud, there is Value for Money as we pay only for the resources that we consume. The testing life-cycle is often intermittent and is a good candidate for utilizing the cloud. At the end of testing, the cloud based resources can be stopped and restarted when testing resumes to avoid accumulating higher costs.

III.Some activities like Performance testing require large production-like configurations for short cycles. Creating this environment on the cloud is an attractive proposition for IT organizations which otherwise often avoid performance testing due to prohibitive investments.

Time:

Dedicated physical infrastructure requires a lot of time to procure & setup. That is where the cloud provides an option of - quick provision of the required environment, a set configuration which can be recorded and applied at short notice. It is also very easy to scale up the environment and provision additional resources whenever required.

 

There are other interesting possibilities. The global distribution of the cloud enables us to run "distributed" performance tests and generate load from multiple regions across the globe to simulate live usage patterns.

CIOL: What are the risks or downsides?

RS: The cloud comes with its fair share of risks and users need to comprehensively evaluate the relative benefits it offers.

I.First of all, Organizations face a risk in giving control of their assets to an external entity

a)The data and resources are not in your physical control. The responsibility for the data and its safety is with someone else.

b)Business continuity and Disaster recovery are completely in the hands of the Cloud provider.

c)There is a dependency on the Cloud provider for SLAs and other support activities.

But there is another side to it- many organizations especially those lacking in technical expertise and dealing with less critical data may actually be more comfortable with leaving security and infrastructure challenges to the expertise of Amazon EC2, Microsoft azure, Google etc.

Compared to the live production, the test environment is a less risky option when testing the waters with a movement to the Cloud. So, organizations are more open to moving test environments to the cloud

II.A very well know issue is that of security. The cloud brings in additional dimensions which impact security:

a)Multi-tenancy: the cloud is a shared resource and there are risks, known/unknown of data leakage to other users sharing CPU, Memory, and Servers etc.

b)Virtualization-While the vulnerabilities to the underlying physical resources remain, the additional vulnerabilities of the virtualized environment pose an added layer of risk.

c)Data stored on the internet especially in large volumes on the cloud are prone to cyber-attacks, having said that most cloud providers do have stringent security measures in place.

III.The other type of risk is to do with Legal and Industry compliance aspects. There are regulations e.g., HIPAA in the United States and the Data Protection Directive in the European Union which require compliance reporting through the life cycle of data. How that can be ensured when data is hosted by a 3rd party on the cloud has been one of the concerns.

CIOL: How does a company pick a strong test tool?

RS: There are tools for a range of testing activities. For each type of testing, there are many Tool vendors who have come out with products.

Many of the vendors provide tools for end-to-end test activities. Choosing a Tool or Tool platform is very much context driven depending on:

Application technology: Web applications are supported by most tool vendors. Applications based on niche technologies have limited tool options for testing.

Cost of the tools- This is relative and depends on the size of business, application criticality, company policy etc.

Availability of skill set , ease of use, learning curve of the tool.

Extent and quality of support provided by the Tool vendor.Alignment with existing tools and technologies. An organization using UFT from HP may prefer to extend their partnership and select load Runner performance testing tool from HP itself.

2 strong trends in recent years are:

The growing maturity and adoption of Open Source tools for a wide range of testing activities. Though these tools require skilled resources to use, their cost advantages make them a very good option for companies to choose.

Many of the tools are now being delivered in SaaS model over the cloud. This is increasingly becoming a criterion for companies to choose a testing tool for to gain the benefits of Cloud.

CIOL: What is the role of APM (Application Performance Monitoring) in cloud testing?

RS: APM tools find usage in- test and live environments, in-house as well as cloud deployments. Major vendors like - Compuware and CA to name a couple have come out with products to address these use cases.

During performance testing, the APM tools provide a complete and in-depth view of system and application performance including:

Resource utilization at the level of server, application, user, transaction to help in better analysis, performance troubleshooting and capacity planning.

Analyze performance down to the method level, break-down performance bottlenecks and pin-point latencies at each layer of the architecture from the UI to the back-end database and service components.

This advanced analysis is especially helpful in the Cloud context because of the dynamic, complex and multiple components involved in cloud architecture. This analysis is also very helpful in defining auto-scaling strategies for cloud hosting.

CIOL: What is an SLA (Service - level agreement) and why is it important?

RS: Any professional service undertaking between two parties should have suitable SLAs to promote the interests of the buyer and provide incentives for the service provider. It defines the standards for the service, sets expectations and is like a warranty for the customer.

In case of cloud services, the cloud provider's quality of service directly impacts the Business of the customer and therefore, becomes extremely important.

Common SLAs which would be of interest are:

Availability- Network, cluster, Infrastructure

Performance

Security

Data compliance

If you look at SLAs stated by Cloud providers, most of the focus is on Availability only. The buyers get Service credits for the provider's deviations from availability SLAs. But, it is necessary to read through the fine print and clarify how Availability SLAs are actually defined and how the other service parameters critical to the business are guaranteed.