What’s the whole purpose of any data driven organization?
To make the data easily accessible or deliver information to users within. Simple, right?
Not really.
In an attempt to achieve this simple goal, most of the organizations end up complicating the journey, thereby reaping very less outcome for all the investments made on the Data and Analytics teams and technologies.
Let’s look at the major composition of any data teams – Data Engineers and Data Analysts.
Majority of the Data Engineering work involves building efficient data pipelines to pull data from multiple operational and external data sources, handling the transformations and any optional aggregations to enable the consumption of the data. Data is fundamentally messy and agreed transformations are quite intricate indeed. If you ask any Data Engineer he will tell you, most of the challenges or the code complexity comes from the need to handle the transformations.
Data Engineering is an area that is increasingly getting automated with many cloud based low-code platforms. Solutions to solve the Volume and Velocity aspects of datasets are highly commoditized as well.
Data Analytics is also getting simpler by the day. A plethora of self-service analytics tools that can talk to diverse datasets and APIs are in the market that enable the production of analytic output. Add to that, the Auto ML solutions that is offered by most of the major cloud providers like Azure, Google and Amazon. It would be a crime to write code for say, a Linear Regression problem, from scratch. And, you also have platforms like H2O that try to simplify ML and AI workloads for enterprise grade business problems.
Data Engineering is getting simplified. Analytics toolsets are plenty. Data Science is no longer limited to PhDs. Which means, simpler and better technology should have resulted in better outcome. Right?
Unfortunately not. Then, where lies the issue? Why are organizations still struggling with their Data and Analytics ROIs? As this article in HBR analyzes, why are companies failing in their efforts to become data-driven?
Who runs your Data team?
It all starts from the team composition.
Most of the data teams are run by technology engineers who tend to focus more on engineering than the actual business need and the real users of data. Their priority turns out to be more often on a robust technology architecture over the real purpose. This inherent bias towards technology often result in them building a solution that is so technically bloated, when the real need is to get simple answers on data.
Not a surprise that many analytics projects turn out to be huge monoliths of technology and a majority of the time is spent in fighting bureaucracy over choosing one technology over another. This tool over that one. In all that melee, the teams tend to forget why they even need the data in the first place.
It is not a bad idea to have the Data teams to be run by business people who are supposed to benefit from the data, working in tandem with technologists. This argument holds good even more now with the way data engineering is getting simplified with low-code platforms.
The primal data analytics skill for today.
The data engineering challenge of today’s digital organization is not in the Capture and Store of data. Rather it is in the ‘Analyze’ or the ‘Utility’ value of data.
As most of the data storage, data integration and analytics problems get commoditized, enterprises will have more data than ever before, capturing every byte that they have very less idea on how to use them. How do we effectively use the data for the right use cases to derive value?
We all know Data Analytics is a multi-disciplinary sport and to be a real expert you need a combination of data munging, domain knowledge, statistics and programming. Now, add to that some data soft skills.
What could be some of the data soft skills?
These are skills that help you thrive in a data rich era. Ability to map a problem space to the right data attributes, to ask with the right set of questions, collaborate, build a storyline, narrate a story and most importantly map the answers to an outcome.
Or, simply put, Learn to Ask.
Asking the right questions doesn’t sound as complicated, till you really start asking. It also relates to solving the right problem, having the right data attributes, deriving the right insights and stitching the answers together into a right story that helps solve a problem or make a decision or take an action.
