Navigating the MLOps Maturity Model
The first significant step in your ML journey is recognizing which stage of maturity you are currently in. Once you have determined your starting point, the next step is figuring out how to advance to the next level. Here, we guide you on how to move up along the ML Op Shop Maturity Model (illustrated below).
The 5 stages in this model are named as follows:
Stage 1: Early Analytics
Stage 2: Mature Analytics
Stage 3: Early Machine Learning
Stage 4: Mature Machine Learning
Stage 5: Transcendence
If you have not yet mapped yourself on to one of the 5 stages, then take a moment to assess where you live by reading more about the MLOps Maturity Model here. Or, if you need additional help, our Fractional Chief Data Officer service offering may be of interest.
Entering Early Stage Analytics
Why Begin The ML Maturity Journey?
You don’t want to be left behind. The Third Phase of Big Data, which began in the early 2010s, is enabling Data Scientists and Machine Learning Engineers to proliferate impact across almost every industry. If your competitors are not already utilizing data science for business impact, they soon will be.
Beginning is low-lift. By navigating through our staged approach, you can keep gas in the tank by generating immediate business impact and lay foundations for long term success.
Who do I need to start my journey?
Analytics Champion: To establish a successful Early Stage Analytics program, an Analytics Champion is required. A lot of energy will be needed to get the project on the road. This person should be a self-starter with a mix of domain experience and a passion for extracting patterns from data.
It is important that they understand the business and are able to communicate their insights to other stakeholders in a meaningful way. Alternatively you can pair someone with intimate business knowledge with an Analytic Champion to help keep their work tethered to where impact can be generated. Their goal is to prove the value that an advanced analytic program can have on the organization.
What tools do I need?
At this early stage, prioritize simplicity. There are very few if any advanced analytics professionals and sharing results in an accessible manner is preferred. Use tools like Excel or PowerBI which have a relatively intuitive user interface and can generate key insights quickly generating a buzz and momentum around the work you are doing.
You may have been collecting lots of data in the hopes of one day leveraging it for business value. If so, you may have more data than you know what to do with, so spend some time assessing your data and consolidating down to the datasets that can provide the most immediate impact. Also make sure you prioritize what to work on with heavy input from the business domain experts and their business stakeholders.
When should I enter this phase?
You have been collecting data for some time now, or are just beginning to. There has been an interest across the business in leveraging data to make smarter decisions. Now is the perfect time to start exploring the data you have and how it can be used to generate business impact.
Maturating your Analytics
Why should you mature your analytical foundations?
Progress is a double-edged sword. As you add more horse-power to your analytics engine, it becomes increasingly more difficult to drive by feel and keeping in-between the lanes is becoming a challenge.
With growing impact, the anticipation of leveraging advanced methods like machine learning is growing. To realize this your tools and processes need to evolve.
Who do I need to mature?
BI Professionals: Best suited to build dashboards and reports at scale.
Data Engineers: Required to establish professional ETL flows and make fast and reliable Data Lakes and Marts available for the BI Professionals.
Analytics Manager: As the program grows, a dedicated analytics manager may be needed, but be mindful that a skilled General Manager can oversee smaller analytic functions with the aid of quality Product Management.
Product Management: As the portfolio grows, a product manager can help ensure the team remains laser focused on business value.
Project Management: As the team grows, a project manager can add value in organizing the team’s logistics.
Smaller teams can do more with less. Management overhead only adds values in larger teams. When teams are spending more than 30% of their time on logistics and consider the Product Manager the role with the highest potential tie-in to business impact.
What?
Tools like Excel start to become less ideal at this stage. Reproducibility becomes more important and so dashboards become internal products. Data marts help feed in to self-serve analytics. Real time analytics add extra value. Your goal here is to establish a set of tools that will set a foundation for a future ML practice. That means data is managed closely, processes are automated and potential for human error is removed wherever it can be. Ad-Hoc practices will stifle growth here.
When?
The early work you have done has built hype around the potential of ML to bring real business impact. Everyone is wondering how it can benefit them. There is talk about predictive and even prescriptive analytics or using ML to help build new features. Up until this point, it has been a lean machine focused on driving short-term ROI. Now it’s time to set the stage for scalability. Your workflows are becoming too important to be left to hackers.
Entering Early Machine Learning
Why should you evolve to early machine learning?
A lot of impact is generated in your current stage of maturity, but you may find your decisions are being made reactively. Evolution will allow you to be more proactive.
Adding a proactive element, you are supercharging your impact.
Who is needed to enter early ML?
Data Scientist: It is important at this early stage of Machine Learning that any Data Scientists brought in have a strong understanding of the domain (or like early stage analytics, pair them with an SME). You could also consider upskilling one of your BI Professionals or Data Engineers who already know the business well. The Data Scientist at this stage will build early use cases. They should also know and follow best practices of software development or it will slow you down in the future.
Team: To successfully navigate this next stage, your entire team will need to up their knowledge of machine learning to be most effective. There should be a common language they all share to ensure the focus is on business impact and not novelty.
What?
The early use-cases in many ML teams will likely include models that help predict certain outcomes or detect patterns that would be invisible to the human eye. You may want to detect outliers or anomalous patterns in your data or make better decisions through pattern recognition. Previously, you may have used basic statistical analysis to do these things but Machine Learning approaches allow you to use techniques that can spot trends in the data that humans would find almost impossible. You will start to incorporate programming languages like Python or R and use collaboration tools like Git and cloud based IDEs.
When?
You are getting to a point of diminishing returns on the work that is being done through dashboards alone. You want to start using advanced techniques not just to report on data but to take automated and highly intelligent action on the data you are already storing at a large scale.
Entering a Mature Machine Learning Practice
Why should you mature your machine learning foundations?
As the number of projects grows, lack of evolution will leave you stuck in the mud. Your models will be stuck in an ever increasing traffic jam, with fewer and fewer making it to their destination (production) on time.
By maturing your processes and infrastructure, you will increase the speed of delivery, improve success rate and free up time to work on adding extra value.
Who is needed to evolve?
Machine Learning Professionals: Crucial at this stage are Machine Learning Professionals who are much stronger in the development process than experimental data scientists. Specifically
ML Ops Specialists help operationalize the ML practice taking the best practices from DevOps and applying them. There should be an increased involvement of stakeholders from IT to ensure the program can scale well.
ML Engineers are Data Scientists with much stronger development skills. Your Data Scientists should evolve to this state, writing clean, scalable and reproducible code.
What?
Techniques like CI/CD should be implemented to improve the integrity of the development workflow. An ML platform helps with co-ordinating the different stages of a model across a multi-disciplinary team. Models and data should be version controlled and systems should be in place to autonomously monitor model performance.
When?
A growing portfolio of models is starting to impact your ability to move at speed. Models are taking too long to move into production and large amounts of time are spent monitoring the health of models and environments. Increasing amounts of time spent on these activities mean less attention being placed on the business impact of the models. You want to mature your ML Operations when you need to leverage higher velocity and ensure models make it to production more often.
Machine Learning Proliferation
Why do I need to go beyond perfecting my own ML team?
As other ML teams start to pop up in different teams in your organization there grows a risk of silos developing and double work, with teams unwittingly competing where they should be collaborating.
A center of excellence responsible for sharing and standardizing knowledge ensures the entire organization has a route to success.
Who is needed to proliferate ML impact?
This should be a collaboration between leaders across the organization to disseminate knowledge and best practices. Identifying leaders in the Machine Learning space and other business functions who can advocate for this stage is vital.
What?
Start by creating a forum where the top minds in ML within the organization can align. Ensure there is buy in from the leaders in the org. You have successfully negotiated the first 4 stages of ML Maturity within your team. Now, success looks like helping all parts of the organization navigate that evolution successfully and hopefully at a higher velocity than what you experienced.
When?
Your practice is up and running successfully. You notice other parts of the business have teams who are on different stages of the journey or perhaps are showing interest in getting started. By generating a decentralized space to collaborate, you can help the entire org drive business impact with machine learning.