Advertisment

Storage virtualization to manage data explosion

author-image
CIOL Bureau
Updated On
New Update

Advertisment

Vivekanand Venugopal

BANGALORE, INDIA: The data storage industry is one of the most dynamic sectors in information technology today. Storage technology has undergone rapid transformation as one innovation after another has pushed storage solutions forward. At the same time, the viability of new storage technologies is repeatedly affirmed by the rapid adoption of networked storage by virtually every large enterprise and institution. Businesses, governments, and institutions today depend on information, and information in its unrefined form as data ultimately resides somewhere on storage media. Applying new technologies to safeguard this essential data, facilitate its access, and simplify its management has readily understandable value.

Since the early 1990s, storage innovation has produced a steady stream of new technology solutions, including fibre channel, network-attached storage (NAS), server clustering, serverless backup, high-availability dual-pathing, point-in-time data copy (snapshots), shared tape access, storage over distance, iSCSI, CIM (common information model)-based management of storage assets and transports, and now, storage virtualization. Each of these successive waves of technical advance has been accompanied by disruption to previous practices, vendor contention, over-hyping of what the new solution could actually do, and confusion among customers. However, each step in technical development eventually settles on some useful application, and all the marketing dust finally settles back into place.

Advertisment

Virtualisation has been used for many years in the technology industry to mask complexity, enable new functionality and drive improvements in performance, connectivity, capacity and availability. Because it is a technique rather than a specific technology, applied to areas as diverse as servers, storage, applications, desktops and networks, it is often poorly understood. Fundamentally, any type of virtualisation aims to abstract software from hardware, making the former independent of the latter and shielding it from the complexity of underlying hardware resources. At a very basic level, it is a mechanism that makes many things appear as one or one thing to look like many. For example in servers, virtualisation is used to make one physical device appear as multiple logical servers.

Storage virtualization is the logical abstraction of physical storage systems and thus, when well implemented, hides the complexity of physical storage devices and their specific requirements from management view. It is the amalgamation of multiple network storage devices into what appears to be a single storage unit. The Storage Networking Industry Association (SNIA) defines storage virtualisation as “the act of abstracting, hiding, or isolating the internal function of a storage system or service from applications, compute servers or general network resources for the purpose of enabling application and network independent management of storage or data.”

 
Advertisment

Storage virtualization is often used in SAN (storage area network), a high-speed sub-network of shared storage devices, and makes tasks such as archiving, back-up, and recovery easier and faster and has tremendous potential for simplifying storage administration and reducing costs for managing diverse storage assets. According to IDC, virtualization includes the capability of connecting servers to logical volumes that are flexibly connected to actual physical volumes. Virtualization helps reallocate a heterogeneous collection of storage resources without concern for low-level details like block size and automate storage management functions. On the one hand, it is designed to make many storage systems look like one, in order to simplify the management of storage and data resources in the face of accelerating storage demand and functional complexity. On the other, some applications of storage virtualisation provide partitioning so that one storage system can appear as many, in order to provide quality of service and security for applications that need to be kept separate.

According to IDC, storage virtualisation in all its forms is emerging as a critical enterprise priority, with a recent survey finding that 49 per cent of organisations are currently evaluating storage virtualisation solutions with 34 per cent having already deployed it.

Business Challenges

Organisations right around the world are facing a data crisis. Data volumes are spiraling out of control across every enterprise as the amount of information and content we create grows exponentially. We are ever-increasingly dependent on information technology in virtually every facet of our life and this is creating a tidal wave of information that somehow must be stored and managed properly.

Advertisment

Today, data storage within most organisations is increasing by about 60 cent each year, with many experiencing growth in excess of 100 per cent. In addition to traditional data-intensive functions such as relationship databases, online transactions and business applications, IT managers have to cope with a massive influx of data from sources such as video, audio and other types of multimedia content, Radio Frequency Identification (RFID) and other wireless data capture technologies and data stored for compliance purposes such as phone calls or transaction logs.

Traditionally, many organisations have tried to deal with growing data volumes by buying more disks. This seems like the simplest of all solutions – when you reach capacity, just buy some more disks. Commodity storage is continually getting cheaper and it is by far the easiest and quickest solution for the enterprise to adopt amidst increasing storage demand.

However, the increasing challenge for organisations is huge and unmanageable storage infrastructure. This challenge is combined with very low utilisation of these systems, often running at only 25-30%. In some cases, storage growth is outstripping data growth by a large degree, which means that organisations are increasing their capital and operational expenditure in areas such as IT staff, data centre management and even storage infrastructure space.

Advertisment
 

Another challenge is that while data volumes continue to hit the roof, IT managers still need to meet demands such as supporting new and existing applications and users so that the business remains competitive, managing risk and ensuring business continuity and maintaining compliance with government regulations and specific industry standards.

There is a strong argument that organisations need to stop buying more capacity and to look at ways they can consolidate their existing estate, increasing utilisation while improving data replication and migration and reducing costs. Storage virtualisation is an increasingly popular way for organisations to address these challenges. It allows organisations to gain control over their infrastructure, extend the life of their existing assets and achieve cost-effective business continuity and data protection.

Advertisment

Storage Virtualisation Benefits

The benefits associated with virtualized storage are due to two improvements in storage management. First, management tasks are simplified and streamlined by underlying software automation, which enables fewer storage managers to oversee larger pools of storage. Secondly, storage resources can be better utilized due to improved management. Without virtualization and pooling, storage managers over-provision storage resources to make sure that they were sufficient. Virtualized storage enables better utilization of total resources and eases the task of migrating data among different storage assets in a multi-tiered storage system. These benefits contribute to a reduced total cost of ownership. Put simply, the economic drivers are fairly straightforward: reduce costs without sacrificing data integrity or performance.

The amount of data companies store is skyrocketing. The growth of unformatted data is fast, email being a prototypical example. Businesses, however, cannot increase their numbers of administrators in proportion to the amount of data. This creates a need to change to an architecture that makes it possible to increase the amount of data managed per operations technician. As factors such as these prompt changes to the storage environment, there are a growing number of situations where building a SAN alone is not sufficient in terms of efficiency and system flexibility, and a virtualized solution is needed. Virtualized storage addresses typical enterprise concerns in the face of growing business challenges like manageability, scalability and availability--- reduces downtime due to failures or configuration changes.

An important objective for storage virtualization is to overcome vendor interoperability issues. Storage array manufacturers comply with the appropriate SCSI and Fibre Channel standards for basic connectivity to their products. Each, however, also implements proprietary value-added utilities and features to differentiate their offerings to the market and these, in turn, pose interoperability problems for customers with heterogeneous storage environments. Storage virtualization products can be used to provide data replication across vendor lines and replicate data from higher-end storage arrays with much cheaper disk assets thus addressing both interoperability and economic issues.

Advertisment

In addition to streamlining storage management tasks, the right storage virtualization solution can make management of multi-tiered storage systems practical as uniform, non-disruptive procedures replace point-to-point, disruptive procedures. Without virtualization, moving data sets from one tier of storage to another is challenging. It is a disruptive process, demanding some amount of scheduled outage, plus the risk of unscheduled outage if something goes wrong. Some degree of virtualization is necessary to make multi-tiered storage systems practical and functional.

 

One of the most widely used approaches is to use the intelligent storage controller as the virtualisation engine. Instead of deploying extra hardware layers in the form of management servers, application blades, new switches and separate management software, organisations can take advantage of virtualisation capabilities that already exist in the storage controller.

By installing an intelligent storage controller in front of their infrastructure, companies can not only aggregate existing storage systems but can also virtualise the services provided to host applications such as data protection, replication, authorisation and monitoring.

This approach offers numerous advantages such as simplified management, increased utilisation of storage resources, seamless migration across tiers of storage, lowered interoperability barriers and better integration of common functionality.

Business benefits

Storage virtualisation simplifies today’s diverse and complex network storage systems and offers a wide range of benefits. Organisations can simplify the management of their infrastructure, consolidating many physical storage systems from different vendors into one pool of storage, masking the complexity of the underlying physical structure and greatly increasing utilisation.

Storage virtualisation also brings about significant cost reductions and efficiencies, by reducing the need for additional software applications and licences, by reducing the need for additional hardware investment (which in turn means lower power, cooling and space costs) and also by reducing labour costs and resources required to manage spiraling data volumes. Typically, administrators can manage from three to 10 times more storage capacity once virtualisation is implemented.

Storage virtualisation also allows organisations to consolidate and utilise existing storage assets, extending their shelf life so they continue to deliver value. Storage virtualisation also allows organisations to consolidate their management and storage services, using a single standard interface to manage storage, archive and backup functions.

Organisations can also achieve far greater business agility, aligning their storage infrastructure more closely to application requirements and reacting quickly to changing service-level requirements. Storage virtualisation also allows for seamless migration of data between systems, meaning that organisations do not have to take systems down when performing upgrades or replicating data across heterogeneous systems.

Finally, the environmental impact of technology and the looming power crisis has made energy efficiency a huge priority for many organisations. Storage virtualisation and associated technologies such as thin provisioning allow organisations to consolidate systems and increase utilisation, significantly cutting the power required to both operate and cool their data centres. This has the dual advantage of improving technology’s impact on the environment and reducing energy costs.

The author is Director, Product and Solutions, APAC, Hitachi Data Systems.