Friday, March 07, 2014

Smart Data


Everyone wants a piece of ‘big data’ but most people are not really sure how to go about it. It is generally thought of as a large investment involving big companies who can help set up the infrastructure to collect all the data they will ever touch, generate or consume. The thought is that this investment will enable them to visualize the data in various fancy ways or give them ability to slice and dice the data as they could imagine. These are all great capabilities and will be useful for an organization who is mature in such data based decision system, but most companies are not even looking at the data they already have which could be very useful in providing them with insights before they start to venture into the World Wild Web of outside data.

For SMBs and even for some of the large companies, just collecting and using the in-house data itself can help them with better understanding and taking various decisions. For example, a retailer can make actionable decision in various parts of a business lifecycle; buyers making the right buying choices based on the data on pricing, demographics, seasonality and competition; merchandiser can reach the customer at the right price point with the right promotions  as well as bring in new customers and provide up-sell and cross-sell; fulfillment can be an exact science as supposed failed promises and disappointed customers; and technology infrastructure can plan accordingly for promotions and seasons.
One of the challenges for the SMBs is that they generally do not operate their own technology which is maintained and hosted by other companies. And in many cases, they do not even have access to their own data. For them, the first step would be to bring their own data under their control so they can start to assess it and drive value from it.

Some of the forward-thinking companies are changing this culture and moving from gut-feel to fact-based when it comes to decision making. A clear example is the product pricing on Amazon.com where the prices can change within minutes and there is no manual person involved. It probably took many years and few mistakes before the merchandisers were comfortable with a system making such decision, but nonetheless this is the only model which would have scaled for Amazon without leaving any penny on the table. So the companies which collect this data and apply various predictive and optimization modeling algorithms to it are going to get a lot better and efficient than the ones who don’t.


The term ‘Big Data’ could be intimidating as it seems to suggest that everything about it is big; commitment, investment, and so is the value. That’s why I prefer to think of it as ‘Smart Data’. In the end, it’s just data and all of a sudden it’s sexy again.