Smart Data can take over Big Data

Anand Sam

Increasing volume and detail of information captured by enterprises combined with the surge in digital, social and Internet platforms usage will fuel exponential growth in data. According to leading data experts, 90 percent of the data that exists in the world today was created in the last 2 years.

Big Data which can analyze large datasets of structured and unstructured information such as videos, pictures, blogs, logs etc. that cannot be analyzed by using conventional analytical techniques is perceived to bring in revolutionary changes in decision making process of organizations and individuals.

However, the misconception about Big Data is amassing more and more data; however the crux is how we use that data intelligently. So it is not the data size itself that matters but how organizations can smartly use the extensive data gathered from multiple sources to gain insights about consumers' preferences, behaviour and use that data for developing business strategies.

This has lead to emergence of methods and techniques called "Smart Data" that help organizations look at information efficiently and more meaningful. Smart Data employs methods, which help analyze information that is relevant to the business requirement.

Organizations need to intelligently use the data and integrate it in such a way that it is relevant to their business. Complex algorithms can help understand the patterns in your data and determine how relevant it is to your business. These algorithms will help organizations sieve through large volumes of data to provide smart insights faster and efficiently.

At Blue Star Infotech we are working on a framework that will help data scientists do their analysis quicker and faster. The framework uses machine learning techniques which can predict the analytical needs of the analyst based on past behaviour, micro and macro factors to come out with smart data which is highly relevant and immediately available.

Smart data is an approach that is seeing lot of adoption; organizations are increasingly looking at intelligent ways of leveraging the data that is available. Let us look at how smart data can improve the ways organizations can leverage data.

Today an analyst in a trading firm would probably be spending 2-3 hours crunching data on oil and gas industry by running complex models on big data sets to generate insights which are used by the traders to make valuable decisions.

With a smart data framework in place, relevant information required for the trading analyst can be predicted and made available in minutes, thereby significantly reducing the time to insight delivery. The information available 2 hours ahead would give the traders significant edge over competition.

It is not about "Big" size anymore, it is all about how you "Smart" size your data.

There are many examples of how businesses have innovated by smartly leveraging data. An interesting example would be how Blue Star Analytics team helped a leading Rugby club in Australia improve their chances of winning matches. Complex analytical algorithms were implemented to unearth patterns by analyzing past match performances, player skills & health statistics, match video footages, venue and weather conditions. The insights generated were effectively used to come up with winning strategies. That year the club went on to reach the grand final for National Rugby League after almost a decade and went on to win the premiership trophy.

So it is not only about analyzing these large volume and variety of information, but organizations have to quickly, efficiently and accurately mine relevant information and make it available for meaningful decisions.

(The author is vice president, BI Services, Blue Star Infotech)