May 4, 2019 • Henry Le
5 Key Success Factors in Building an In-House Analytics Innovation Factory
Want to deliver more business value for your utility? One sure fire way to do that is by creating an in-house analytics innovation factory. Sounds great, right? Utilities have a lot of data, and lots of business challenges that could be solved by putting it to good use. But how, exactly, do you go about it?
First, let's define what an analytics innovation factory is. It is an organization with the capability to churn out analytics solutions for use cases that add direct near-term business value and has the ability to continuously adapt and improve based on feedback, changes in data, and business condition.
An analytics innovation factory can add value to the business quickly, but building out this capability requires a carefully thought-out plan, and constant monitoring and updating of execution. This blog will lay out the 5 key success factors, and some practical advice on how to approach them.
Key success factor #1: It starts with the mindset
In my experience working with many Utilities organization across North America, the first question that I usually get asked was about buying a new platform or what kind of new skillsets required.
However, new capability starts with mindset. One of my favorite movies is The Matrix, not because the characters can do many cool tricks, but because of how they do it: first is to become. New capabilities starts within, with a decision and mindset that this is something we are going to do, and we are going to excel at. Same goes with data analytics capability.
Key success factor #2: It's about people
This is harder than it seems, and it's not about hiring some data engineers or data scientists and calling it a day.
For analytics to draw meaningful insights, we need the knowledge, experience and subject matter expertise of everyone in the organization. And once the insights are drawn, decisions need to be made, and changes to existing process need to be implemented. All these have far-reaching impacts beyond the data engineers and data scientists. To build an effective in-house analytics innovation factory, organizations must find ways to engage existing resources, empower and enhance their capability through training and augment needed skillsets with strategic hiring and partnerships.
There needs to be a roadmap detailing how the organization’s resources and skillsets look today, what's missing, how gaps can be filled today and how growth will be managed over the next 3-5 years.
Key success factor #3: Data
In doing analytics, similar to doing analysis to support important decisions, it's natural to desire more data, better data to make quality decisions.
Today we have more data than ever, growing in multiple ways—in Gartner's terms: volume, velocity, veracity.
However, at the same time, it is very common for analytics projects to have incomplete data, quality gaps in data, or missing some important data altogether.
We should resist the desire to acquire as much data as possible; sometimes adding more data sets does not help the analysis, but only serves to make it more complex to integrate and maintain.
If important data sets are not available, assumptions and solutions can be worked out with the business. Some important questions to ask are: how can we make up for the missing data today; are there stop-gap measures we want to put in place; and are we willing to collect and maintain this data in the future? While we now have machine learning available to us, decisions like these in data strategy demonstrate exactly why the role of the human worker is more interesting and important than ever.
Key success factor #4: Platform & Technology
Now we can talk about tools and technologies. Analytics continues to experience an explosion of tools and technologies: big data, unstructured data, AI, machine learning, deep learning, cloud computing, serverless paradigm, etc. all are hot topics these days. It's easy to get lost chasing the next shiniest object.
However, it's certain that we don't need all the above technologies to achieve business value.
A good foundation for your analytics innovation factory should have the following components:
- Ability to ingest data from internal and external sources that you have include ETL, real-time replication and streaming
- Ability to orchestrate and manage the flow of data
- Ability to store and process big data (include unstructured) in batch
- Ability to perform real-time or near real-time stream processing
- A data warehouse (Editorial comment: In case you heard otherwise, the data warehouse is not dead. It's actually very important for high-quality data analysis, application integration, more user-friendly to users who do not have deep data engineering and data science skills, and to support dashboard and reporting)
- A "sandbox" area for data scientists, data and business analysts to freely perform their analysis
- A mechanism to deploy and run analytics integrated with applications (as a service)
Key success factor #5: Delivery capability
Last but not least, to successfully deploy an analytics innovation factory, organizations need to employ a more agile approach to delivery. Meaningful analysis needs to be done in a matter of days, not weeks or months. Use cases built need to add value in a matter of weeks not months or years. Quick wins are critical to demonstrate value and keep the factory doors open. How to do it is in the art of breaking down the work in value-added chunks. For example, instead of 3 months of machine learning model builds, can the first few weeks yield some analysis that can help the business to understand and be more prepared for how the model will impact day-to-day business and how the rollout of changes can be better managed?
It does not need to be a full-blown implementation of scrum (an agile project management methodology with the goal of delivering new software capability every 2-4 weeks), as your organization may not be ready for it yet. We need to start with knowing the organization's objectives and goals, have a plan, but be flexible in changing as new information unfold. Use your organization's goals and objectives as the North Star, and check against it often to ensure the team does not stray too far.
Developing an in-house analytics innovation factory can be an excellent way to deliver value to the business quickly. And in my experience, the benefits increase over time as utilities’ understanding and abilities grow. It's by all means not a replacement of how to deliver all analytics projects. Sometimes, long, waterfall projects are still the way to go, especially when there's dependency with critical application roll out (like a CIS implementation).
But in today’s business environment, developing the mindset, capabilities and infrastructure to deliver quick wins is something that can make a difference in every utility. If you’d like to talk about how to build an innovation factory in your organization, email me at email@example.com, or contact us at firstname.lastname@example.org. Happy innovating!