Importance of Data for Business Success

BANGALORE, INDIA: All the data that has accumulated in your organisation over the years can give you more insights than you would have ever thought, only if you deploy data mining techniques

What if your data could tell you automatically how the time of the day, combined with the weather, affects the footfall in your shops? What if your data could help you visualize correlations between raw material availability and time to market with hardly any effort on your end? That, and much more, is what data mining can help you do.

What data mining is NOT

It is a popular misconception that data mining is semantically equivalent to business analytics/business intelligence. Data mining is concerned with the discovery (which is typically accidental and hence should be surprising to the end user in ideal conditions) of unknown patterns/hidden trends in your data. It is a means to an end, not the returns gained by itself.

Where is data mining in daily lives?

Google Trends is a good example of data mining. So are TV viewership estimates which co-relate a particular program's viewer's age and location with purchasing power. The actual data mining process always happens behind the scenes. In fact, not all data mining tools even have a GUI, but they do a good job at mining the data and if properly used, are worth every Rupee spent. They affect our daily lives.

Why should I consider data mining for my organization's data?

Data mining brings to you many opportunities that have the potential to help you gain high returns. For instance, customers typically buy a set of goods together during few sporadic intervals of time in a year, such as during festivals. Data mining can be a valuable asset in deciding the margin that you keep. You can find out when a loyal customer turns otherwise, what purchase made him to do so, and so on. This is valuable by all means for any SME. Similar benefits are applicable when you are dealing with data from your supplier chain.

How can I get started using data mining?

For an SME, the first step to start reaping benefits from data mining is to consistently record data in a pre-defined format. For a start, most of your needs will be satisfied by even common Office applications such as Excel and Access. You may already be analyzing the data contained in these applications. But when you enter the data that is necessary (as well as sufficient: no more, no less) to discover hidden patterns, you will start seeing the results.

When recording data, you should record information that can help you gain the added advantage. For instance, when you enter a new customer, try to extract information about how the customer was introduced to your SME (via another customer by word of mouth/advertisement, etc.) and include that data in your records. This will help the data mining tool easily draw relations and help you find out influential customers. It won't take too long before you are able to gauge the returns. When you enter data for a new batch of raw material having arrived, enter information about the stock in the inventory and the demand at that point of time. It will be beneficial to draw relations and that is what data mining will help you do.

What about Big Data?

Many SMEs have already started exploring the capabilities of Big Data because they too, like enterprises, have been facing issues with the volume of data having accumulated over time. The traditional RDBMS tools used by SMEs are at a competitive disadvantage when it comes to handling large volumes of data AND extracting actionable information from the same. This is despite the fact that there is confusion as to what exactly constitutes Big Data. Many vendors such as SAP and Oracle provide Big Data solutions for SMEs.

What if I do not have the budget to procure licenses for commercial data mining products?

There are plenty of open source tools available for data mining which you can obtain free of charge. This has two important implications: the first one being no expenditure on product procurement and the second being that open source solutions typically adapt to the latest technology faster than others. In certain cases, this might give you the edge over enterprise competitors who may have deployed commercial solutions.

In fact, if you have the right technical expertise available in-house, you can even fine-tune the algorithms used in the software to match your specific needs. Most such tools are found to be cross-platform. However, as with most open source tools, the trade-off that you make is that the cost saved on obtaining the software might get spent on training your employees and data mining software does bring with it a certain learning curve.

When I am evaluating data mining products, what should I look for?

In no particular order of preference, the following factors must be evaluated:

1. Data preparation capabilities: As stated above, the tool's job becomes much easier if data is in a ready-to-use format. This involves several steps such as selection of the relevant dataset, filtering data and cleaning invalid entries, etc. However, a good data mining solution should be able to get this done by itself with minimal user input.

2. Input formats supported: From a MySQL database to a CSV (comma-separated-values) file to an Excel worksheet, the data mining product should ideally be able to digest information from the maximum number of formats without compromising on quality of data retained. You might only be using one of these formats in your day-to-day business but it always helps to have more options open.

3. Configuration options: The very nature of doing the data mining process requires one to be a control freak. As with the ever-present trade-off between ease-of-use/convenience and security, there are trade-offs made in different algorithms used in data-mining, including the most basic ones which use simple averaging. Such algorithms deal with classification , clustering, forming associations, modeling, etc.

At the end of the day, you are trying to come up with statistics on a best-effort basis. There can be no formally defined perfect solution to the same. You will need to get a solution that is able to cut the coat according to the cloth. When you choose a certain algorithm, each parameter should be allowed to be controlled ideally. In some cases you will notice that even the order in which you feed data to the algorithm directly affects the end results.

4. Visualization capabilities: It is imperative that different types of tables, reports, charts, graphs, etc. can be generated and that you are able to understand all of them and they are actionable.

5. Scalability: Your data mining tool should have proven ability to handle large data sets without any adverse effect on performance. This helps in the long term. Certain open source solutions are seen to falter here compared to their commercial counterparts, which are usually stable if not high performing.

6. TCO: Because of the learning curve that most data mining tools come with, you need to consider not only the licensing costs (if any), but the support/training costs over a period of time.

You might want to come up with your own set of documentation for internal use for your specific needs because any good data mining tool comes with support for dozens of algorithms. Not only do you need all of these (especially if you are an SME), but you cannot also buy packs of algorithms relevant to you separately. You will need to obtain the complete package, and avoid being overwhelmed or confused by the choices.

Source: www.pcquest.www.ciol.com