Advertisment

Harnessing the information overload

author-image
CIOL Bureau
Updated On
New Update

The amount of information produced in the world increases by 30-40% every year. Trends show technology evolution leads to information explosion - from Kilobytes to Petabytes. As the amount of available data grows, the problem of managing the information becomes more difficult, which can lead to information overload or information fatigue.

Advertisment

Let us look in to the Telecom service industry in India. On an average, 5.5 million cellular lines were added every month, taking the mobile subscriber base to 157 million from 91 million in 2005-2006. (Source: Annual survey is conducted by Voice & Data)

For example, for a large wireless service provider with around 5 million subscribers, every subscriber makes 20 calls per day on an overage. Each call generates call detail record (CDR), which is used for calculating the bill and subsequent collection process. Each CDR is money to the telecom service provider. Around 100 millions of raw CDRs generated per day, gets transformed before reaching billing system and needs to be maintained for 3 months online and later archived. Moreover, the same data (refined and transformed) is used for various applications like billing, customer care and business intelligence - which translates the total information in various forms in to Terabytes over the period.

Managing such huge amount of data is a big challenge. This involves securing, managing and archiving. This increases management cost and storage acquisition cost. Apart from achieving operational goals from the data, new government regulations and guidelines are a key driving force in how and why data is being retained, as they are now requiring organizations to retain and control information for very long periods of time.

Advertisment

An analysis of data will most likely reveal, that 25% is accessed and maybe updated on a very frequent basis by most of the users. As the age of the data increases, its access frequency diminishes to almost negligible, if any. Only few and very few users are accessing, all of the other data. Thus, data can be described as being; active, less active, historical or ready to be archived. With so much data being held, data, during its lifetime, the data will be moved to different physical locations. Also, the business user's requirement of information on demand, force the information available online.

In traditional approach, the data is stored in huge storage boxes, and once the information is used, straightaway it is stored in the secondary storage media like Tapes. With flat file systems, this sounds simple. In RDBMS environment, it requires downtime for both archival and retrieval. User has to wait for the information to be restored.

Information lifecycle management

Advertisment

So, managing the information is one of the key requirements in modern day IT enabled business. What is the solution for managing this information overload? Information Life Cycle Management (ILM) - the new building block in the IT management strategy. ILM is a combination of data management capabilities of the database server with appropriate storage technology.

Oracle Database 11g, with it's strong data management capabilities offer an ideal platform for fine grained, application independent ILM. Oracle Database 11g enhance supports for information lifecycle management (ILM), enabling organizations to ensure regulatory compliance, optimize storage, streamline business processes, and reduce cost. Oracle Database 11g Partitioning based ILM can greatly enhance the manageability, performance, and availability of almost any database application.

Oracle Database 11g ILM capabilities help the organizations to classify the information according to the age, deploy it on various tiers of storages, thus balancing the cost vs. performance. This greatly improves the availability of the information to all the applications and all types of users. With the technique, DBA can allocate the current data in SAN, historic data in low-cost storage or NAS and archival data on Tape or any solid-state media.

Advertisment

With Virtual Private Database feature, built-in Oracle Database 11g, the accessibility of the data can be controlled to select users based on the age of the data. With Oracle Secured Backup in 11g, the backup and restoration tasks can be managed with data encryption functionality.

The benefits of using Oracle Database 11g as ILM platform are

1 Application Transparency; data classification is transparent

2 Fine-grained; managing data at individual row level

3 Low-Cost; uses low cost storage to reduce costs

4 Enforceable Compliance Policies; define and enforce policies

Advertisment

The Oracle ILM Assistant is a GUI based tool for managing your ILM environment for Oracle Database 11g. It provides the ability to create lifecycle definitions, which are assigned to tables in the database. Then based on the lifecycle policy, the ILM Assistant advises when it is time to move, archive or delete data. It will also illustrate the storage requirements and cost savings associated with moving the data. Other capabilities include the ability to show how to partition a table based on a lifecycle definition and to simulate the events on a table if it were partitioned.

To assist with managing compliance issues, the ILM Assistant will show all VPD and FGA policies that have been defined. It can also display and query all the audit records and generate and compare digital signatures.

Director Technology Sales Consulting, Oracle India