/ciol/media/agency_attachments/c0E28gS06GM3VmrXNw5G.png)
Follow UsAnil Chopra
Data is the lifeblood of any organization. It is, therefore, important that this blood is kept clean and free of all impurities, so that it has a smooth flow and is available as and when required. Sound like something that a healthy human body should do? Well....isn't your IT infrastructure similar and complex? It has many different applications that are constantly generating so much data and working on it. It has a network that must ensure smooth flow of data across all applications, no matter where they are located. It has various storage devices to store data, so that data is always available whenever you need it. If this storage infrastructure is not managed properly, it could have serious implications for your business. What you need, therefore, is a good storage management strategy not only to manage the data properly, but also to ensure business continuity. Storage management should, therefore, be top priority for any IT manager today. We arrived at this conclusion based on views of 58% of the respondents of our
survey, who said that storage management was far more important than any other IT infrastructure management area. Another 25% said that it was equally or more important than other IT infrastructure management streams.
Key challenges
How difficult or challenging is the task of managing storage? For more than 50% of our respondents, it was manageable, but another 38% found it to be either difficult or very difficult.
We can divide the specific challenges themselves faced by the IT managers them into issues related to data backup, user related problems, data consolidation, availability and performance. Plus, issues related to managing data spread across a large geography.
Issues related to backup include reliability of storage media, insufficient backup windows and speed of restoration should the need arise to do so. There were also worries related to rapid changes in storage technologies. As tape has been around for a long time, the associated challenges of backing up data to tape are fairly well known. There's no single solution to the challenges, but there are a few guidelines that need to be kept in mind. For instance, when going for a tape solution, you need to ensure that there's a pre-defined roadmap for it so that five years down the line, your existing solution will still work with whatever is available at that time.
| Research Methodology
While researching this story, we interacted with 25 users and CIOs across large enterprises to understand their strategies for managing storage. We had futher in-depth sessions with a few others who have been . There are some very clear strategic trends that emerge from this survey, which have been presented in this story. The interpretations of what the CIOs have told us are ours and they are in no way responsible for the same. |
|
| How do you manage the ever growing volume of data in your organization? |
Likewise, you need to ensure that the vendor will be able to support your existing backup solution in future as well. It shouldn't happen that there are no drives available to read the extremely critical data that you've backed up to tapes today. On the data archival front, there were worries about retaining data for compliance, identifying and distinguishing the right data that needs to be archived, and finally retrieving it in case of an investigation. Moreover, the challenge is not in data retrieval per say, but the speed and accuracy at which it's done. Here again, policy plays a more important role here than technology. You need to ensure that whatever you're backing up is checked for accuracy. You need to perhaps test your backups at regular intervals to ensure that you don't get a heart attack when you really need to restore the data.
| What's your biggest hurdle in ensuring data compliance? |
Another key challenge was managing data that's spread across a large geography. There can be many issues to tackle in such cases. The data could be residing on multiple platforms and being used by different appls. You might be adding additional branch offices at a very rapid pace, for which additional storage capacity needs to be provisioned for, etc. The amount of storage usage and application performance due to growing data is the other issue faced by CIOs. As data increases continuously, it's a challenge to ensure that the applications that use them are able to do so with good response time.
One of the key reasons for so many storage challenges is that the there's no end to the growing amount of data. It just keeps growing and growing. The opinion on how easy was it to manage this growth was almost equally split between being manageable to difficult to manage. 55% of the respondents felt that this was manageable while another 41% felt that it was difficult. But by and large, there were two ways in which a majority of the respondents were managing the growing volumes. One was through storage consolidation, and the other was to optimize storage utilization by following data management policies. That is perhaps the best combination to follow today. There might be instances where adding more storage capacity can help, but that is not the best method beyond a certain point.
Major and minor pain points
Owing to its vastness, there are lots of management tasks that need to be prioritized and taken care of. The task that seemed to keep the CIOs busiest was adhering to statutory requirements for data retention and archival. This was the issue that CIOs spend most of their time on. This is not surprising as more than 60% of the respondents said that their organization is required to store data for legal requirements.
The biggest hurdle to staying compliant appears to be consolidating multiple islands of data. Some 52% respondents stated this as their biggest challenge. As 30% of the respondents to our survey were from BFSI, they had to follow various guidelines like the banking regulation act, prevention of Money Laundering Act as well as RBL and SEBI guidelines. Some CIOs also said that their companies followed US based and international guidelines such as Basel II, Serbanes Oxeley, etc. More than 50% of the respondents felt that the Indian IT laws on data storage for compliance were actually sufficient for their industry. However, an alarming 36% said that the Indian laws were insufficient. The key issue that emerged here was a lack of clarity on the IT laws for data compliance. We were also told that there were contradictions of data requirement under various laws. There weren't any specific laws defined for data storage, and whatever landmarks were set were insufficient. Therefore a consolidation of data laws themselves is required, along with a clearly laid out policy. The next most time consuming task was managing data security, with 40% of the respondents saying that most of their time is spent taking care of this. Another 32% of the respondents said that they spent moderate time on this issue. On the other hand, email archival, regardless of its high importance in storage management, isn't really a pain area for CIOs. A majority of the respondents said that they spend the least time handling email archival woes. The other area that our respondents didn't spend much time was migrating applications and data from one server to another. While migration is definitely not to be taken lightly, it also isn't something that's done frequently.
Existing vs future deployments
More than 60% of the respondents already had centralized their storage management, and 23% wanted to do it in the near future. Email archival is also being taken seriously with more than 50% respondents already using it, and another 23% of the respondents wanting to deploy it in near future. Same situation prevails for disaster recovery and business continuity planning solutions, with more than 50% current ownership and another 23% wanting to deploy it in the near future.
There were more fiber based SAN implementations than IPSAN based, but the future holds more promise for IP based SANs. Our survey revealed that 27% of the respondents already had IP based SANs, while another 23% said that they plan to deploy it in the near future. On the other side, 36% of the respondents already had fiber based SANs, but only 9% more displayed interest in deploying the same in the near future.
The area of maximum interest amongst the CIO community was ILM (Information Lifecycle Management). Though only 9% of the respondents had already deployed it, another 32% wanted to deploy it in the near future. The next major area of interest was disaster recovery and business continuity planning implementation. Interestingly, 52% respondents had already deployed disaster recovery and BCP, while another 28% said they plan to do so in near future.
Content management, document management, continuous data protection and e-mail archival were the other three promising areas for the future. There were 24% respondents each for these four areas, who plan to deploy them in the near future. Content addressable storage or CAS and snapshotting for all their hype aren't very popular yet. There weren't too many takers for these two technologies. In fact, there weren't too many current ownerships for CAS. Storage virtualization for all its hype is still not a priority for most. In fact, the number who already had it was higher than the number of CIOs who planned to deploy it in the near future.
Manpower and vendor management
A good 44% of the respondents to our survey said they had dedicated manpower for managing storage. There were another 36% who didn't have any dedicated staff for the job, and relatively few had outsourced the same.
Managing such manpower is not easy and the key reason for this seems to be the rapid advances in storage technology. Every other day, something new and exciting comes up in storage, promising great benefits to your business. If you deploy the same, your manpower must have the skills to manage it. Providing training on the latest technologies therefore emerged as the key concern. The next biggest concern is finding and retaining dedicated storage professionals. As technologies change rapidly, storage professionals would want to learn the same and if the organization isn't able to provide the necessary training, they would look for better opportunities. The results on vendor related issues were fairly interesting. Around 36% of the respondents were more than satisfied by their primary storage vendors. However, another 44% who were moderately satisfied. That is, they didn't have any major issues with their vendors, but weren't exactly overwhelmed by their support. So when we asked for specific issues related to vendor management, hardly any responses came out. There were stray cases of inadequate onsite support or the technology of data storage being provided. There's no end to challenges in storage management, as this is one stream that's under constant development with new technologies evolving every day. This is understandable considering that the growth in volumes of data in an organization isn't going to stop or slow down. Organizations will therefore continuously require new and better technologies to manage them. Therefore, you need to constantly evolve your storage management strategies to manage this data and the infrastructure that holds it in place.
| View Point: Anand Bhatnagar, Vice President, The Bank of Rajasthan Limited | View Point: Ashish Dandekar, AVP Enterprise Management, NSE.IT Limited | |
| Q: You'd mentioned that you plan to deploy ILM in the near future. What are the benefits you are likely to get by deploying it? If we can't define the life cycle of data, then it will be a never-ending process and data pruning will have to wait for dead end. While presently what we and mostly everyone in the banking industry has adopted is needs based management. While data is mostly backed up to magnetic media, sometimes either due to an application upgrade or non-availability of the specific platform to access that data, the archival (historical data) remains a thing of mandatory requirement only. Hence while accessing the desired data when really needed and also at the same time pruning and destroying the "not so useful" data will save the space and management effort to a great extent. Another dimension is falling of stored data in wrong hands will also be safeguarded, when there are policies and systems in place for the same. Q: You'd mentioned that your biggest There is an ever-increasing need of historical data in any organization for analytical purpose. But as the volume of the data increases, efficiency moves in reverse order. So on one side, there is always pressure to retain data that's at least eight to 10 years old, on the other side whatever indexes you have, backup time, data extraction becomes more time consuming That was the reason I mentioned that the biggest challenge is keeping the balance between response time and data availability. Currently we are managing the same by maintaining a parallel system for keeping an archival reporting system, which is accessed by users more frequently. This way we reduce access to such data by users and data also remains in the system for exigencies and analytical uses. Q: How do you ensure high data availability for your applications? Q: What are some of the challenges you face in doing storage consolidation? How are you addressing them? Another dimension to doing data consolidation is for making a common database for multiple islands of databases in the organization. Though the ideal solution for the same is data warehousing and implementing business intelligence tools on the same, but we did not have enough budgets for the same to implement it immediately. Therefore we found a middle path and skipped the data-warehousing step for the time being. We developed separate ETL (Extraction Transformation and Loading) scripts for extracting data from different databases and got the cube developed using SQL 2005 through one of our technology partners. Then by using reporting tools, we generated analysis from this cube and thus the data was utilized from |
Q: What are some of the best practices to follow for doing storage consolidation? You need to examine the growth of your data by checking all your historical data. You also need to look at your current storage requirements and projections/analysis of requirements for next 3-4 years. Plus, you need to identify the real business need for doing consolidation. Check whether your organization is subject to Indian or foreign compliance norms. For storage consolidation, you need to do forward thinking and visualize your organization's IT strategy for next 3-4 years, keeping in mind the cost of technology obsolescence. You need to plan & strategize for business continuity, and don't mix it with the implementation time. That needs to be accounted for separately. Q: Suggest a few guidelines for optimizing storage capacity. Q: What are some of the best practices to follow in storage management? Sensitive data can be encrypted. The media should be stored in non-condensing but controlled humidity conditions under constant temperature (OEM stipulates temperatures between 16 to 32 degrees C) for better life expectancy. When placing purchase orders, ensure better services by a formal Service Level arrangement/ agreement for ensuring better uptime and problem resolution. Q: What should you look for when choosing a disaster recovery partner? |