Big data calls for more data scientists

BANGALORE, INDIA: In the past, we had enough number of information technology and software engineers to drive the IT revolution in the country. And today, as the industry is undergoing a change with the onset of cloud and big data, we need a new breed of engineers skilled around these two technologies to drive the next stage of IT revolution in the country.

Zinnov in a study points out that India will require over 1,00,000 people skilled in cloud computing technology, by 2015. However, do we or will we have that many people to cater to the technology by 2015, is something that industry and academia needs to look into.

A similar skill set deficit will be felt in the space of big data as well.

Manoj Chugh, regional president, Global Accounts, APJ, EMC Corporation, in an interview with Deepa Damodaran of CIOL, explains how data is growing in terms of volume, velocity and type and also why traditional data analysts and the old-world IT educated engineers will not be able to cater to the big data needs.

CIOL: What is driving Big data and why is it important?

Manoj: Studies say that in the next ten years the world will have to deal with far more data than today's. IDC says that in 2011 the data created and shared around the world was about 1.8 exabytes and over the next ten years this is going to grow to 35 exabytes.

The growth rates in India are significantly higher than the worldwide rates. We expect data to grow 60 times in the next 10 years.

Not only the total chunk of data that is being created is increasing, but also the velocity at which these data is being created is also increasing due to high social media usage. This is a huge opportunity for businesses.

Businesses are today looking at means to drive insights from this large chunk of data on social media and other external sources, in order to know what their customers are thinking about them. Such  insights could help them take decisions for the future.

CIOL: So wouldn't the old-world data analysts be able to analyze this set of data?

Manoj: Traditional analysts are used to only look at data warehouses. They use visualisation tools to help you understand what happened in the past.

However, today the data, which is being created at a high speed and large chunk from external sources, needs to be analysed in real-time.

Today data is not just about structured rows and columns, but largely unstructured like videos. And, unlike in the past, data sets have become multi-media data sets today.

The world is moving very fast and you cannot wait for six months for some guy in a white coat to look at your dataware and tell you this is what you should have done, because by then it might be too late.

The ability to look at real time analytics and driving insight from multi-media dataset is the need of the hour.

So data scientists are required. They will play a pivotal role in helping drive competitive advantage for organisations going forward.

CIOL: Didn't we have data scientists in the past?

Manoj: Data scientists have existed before, but only in niche areas such as financial and oil and gas sectors.

Today, big data is becoming main stream and the need to merge external and internal data sources in order to drive business insights will become very critical for businesses to remain competitive.

So data scientists are the new breed of professionals who will come to mainstream businesses. They will be skilled in maths and statistics, computing technology, social sciences, behavioural studies and business.

Over the last many decades there have been a huge focus around maths in India. I believe India holds the opportunity of being the hub for data scientists.

Whether we seize the opportunity is a different matter. I hope we do not lose this opportunity.