Lean AI Software Development — Shannon Alliance

Introduction

Lean software development principles are popular for software teams building minimal viable products that can test product and market assumptions fast.

But, in my experience working with data teams over the past decade, it’s rare that I come across an AI team fully embracing lean development workflows. Sure, most teams pay at least nominal homage to lean principles (by managing their workloads with Scrum or Kanban), but I’m always surprised at how many Data “leaders” affirm that lean development principles are impossible for them to implement. The most common objection I come across is some variation of “my workloads are too experimental and unpredictable to standardize”. But this is nonsense, people said the same thing about building cars before Henry Ford came along!

I’ve shipped over a dozen AI software solutions over the past decade, and learned from many mistakes along the way. Just because AI development is more experimental and non-deterministic than traditional software development, it doesn’t mean lean principles don’t apply. It’s amazing how big of an impact you can have in your team with just a few lean interventions. Now I use the “5D” framework for getting an AI idea to MVP in a lean way.

The 5 D's of lean AI-SaaS development, shown above, is a battle tested formula for ensuring you only start breaking ground on an idea once you know it's worth a bet! It’s broken down into these 5 phases:

Define your idea
Design a Rough Solution
Derisk with a Minimum Viable Model (MVM)
Deliberate on whether it's still worth pursuing
Develop a Minimum Viable Product (MVP)

In this article I’ll walk you through the repeatable “5D” process for getting an AI idea to MVP.

Phase 1: Define your idea

It can be tempting to start building as soon as you get an idea, but in my experience this approach almost always fails—most ideas should be thrown out, and any remaining ideas will need refinement. A good AI product solves a problem for a willing customer so before you start building, it’s important to do as much work up front as possible to validate your product has a market.

In this stage of AI software development you define the problem and customer and capture any evidence that shows your idea does/doesn’t:

Solve a real problem
For a real customer (if you’re building an internal tool for other people in your organization, your intended users are your customers)
That would be willing to pay for its solution

You will answer those questions through the development by building out the following artifacts:

An Idea Description Document clearly defines what your product is, what problem it aims to solve, and how it uses AI. This should be the first document you create as it sets the foundation for the others.
Your Customer Persona Profiles contains detailed customer personas documenting their needs, preferences, and behaviors and how your product fits into their context. Once you have a clear description of your idea, identify who your ideal customers are and capture all you learn about them in this document.
An Assumption Mapping Chart includes assumptions about your customers (the market), your product's usability, and its technical feasibility. With an understanding of your product and potential customers, map out your assumptions.
A Market Analysis Report involves looking at potential size, competition, and market trends relevant to your AI product. Armed with a clear idea and assumptions, analyze the market.
An Evidence Log is critical for validating or invalidating your assumptions and refining your product idea and market understanding. This should be developed concurrently with the above steps. As you gather new information from customer interviews, market research, and other sources, continuously update this log.

Iterate on these documents as you gather more information. The process is cyclical and adaptive, allowing you to refine your understanding and approach based on real-world feedback and data. Coming out of this stage you should have a good understanding of what type of a solution would meet your customer’s needs and you should be ready to either start designing a rough solution or throw your idea out. Note: don’t be discouraged if you come out of this phase realizing your product doesn’t solve a real problem for a real paying customer—this should be viewed as a win—if your product is destined to fail, it’s better that it fails during this phase rather than after you have sunk countless hours into its development.

Phase 2: Design A Rough Solution

Once you have your idea defined you should start creating a High-Level Design Document to describe what a viable solution to your customers’ problem would look like. This Design Document should include:

Customer requirements, i.e. what set of features would convert potential customers into paying customers
the overarching architecture of your AI solution (detailing the integration of AI elements with other system components)
potential risks specific to the AI aspects of the project, including technical, market, and operational challenges.

This design document should be evidence based—the Assumption Mapping Chart, Market Research Report and Evidence Log from Phase 1 should be considered your source of product truth, and as such you should continually reference and update them as your idea gains more touch points to reality during this design phase.

You should emerge from this phase with a clear understanding of what your software needs to look like, how the AI component of your software needs to perform, and any other potential pitfalls likely to stand in your way.

Phase 3: Derisk with a Minimum Viable Model (MVM)

You won’t always know up-front if AI will be able to solve your problem—maybe you won’t have enough high-quality data, maybe there won’t be enough signal in the data, maybe you can’t achieve the latency or accuracy requirements, maybe it will take too long to build the solution, etc. AI related software projects can come with a lot of non-deterministic technical risk, as such it’s important to derisk the AI component before you commit to building out a full MVP solution. We call this derisking process minimum viable model development.

When developing an MVM you answer the questions:

“Can AI solve this problem?”
“Can AI solve the problem in an economically viable way?”

Typically you should not spend more than two weeks trying to solve these two questions—it’s important to keep things time-bound so that you don’t overcommit resources.

When answering question one, “Can AI solve this problem?”, you typically want to leverage the best-in-class models (if the top of the line models can’t solve your problem, it’s unlikely that cheaper/more efficient models can), and it usually helps to use low-code tools to move faster. At first, you should “eyeball” whether or not you are solving the problem, however, once you think you are getting close to having a solution that solves your problem you should start designing and performing formal acceptability tests. Once you have a feasible solution, you can start answering the second question: “Can AI solve the problem in an economically viable way?”.

Oftentimes it’s more difficult to get an economically viable solution than a working solution. There are many ways to optimize costs. You can save by scaling back from the best in class models to leaner models. You can see how your models perform with less data. You can shift from managed services to more bespoke solutions (however if you decide to go a bespoke route, don’t emphasize getting a production ready solution in this phase, first just focus on getting something that works first). As you optimize your solution, just make sure you continue to ensure it still meets its technical requirements.

Coming away from this phase you should have derisked the AI component of your MVP.

Phase 4: Deliberate on whether this idea is still worth pursuing

After validating the technical viability and market potential of your idea, it's time to pause and consider if this is the most effective use of your resources. Once you get good at AI software development, you’ll start running into commercially viable applications of AI everywhere you look, and so you need to prioritize what you work on.

The outcome of Phase 4 is an aggregation of the groundwork laid in the earlier phases, stitching together insights from all previous phases in order comprehensive project proposal that includes:

A Development Roadmap that serves as a step-by-step guide for transitioning your AI concept into a market-ready MVP, detailing each stage of development, testing, feedback integration, and final adjustments.
A Project Readiness Score that carefully considers the

Level of Effort: An estimated calculation of the time, resources, and personnel required for the project.
Risk Assessment: Categorizing risks and employing a probability and impact matrix to evaluate each risk's likelihood and potential effect on the project.
Market ROI: Anticipating market ROI through analysis of market size, growth rate, competitive landscape, and profit margins.
Strategic Fit: Evaluating how well the project aligns with the organization's long-term strategic objectives.

These elements combine to form a robust decision-making framework, ensuring that resource allocation aligns with both technological potential and strategic business goals.

The proposal document ensures that you only commit your resources to projects that promise not just technological advancement, but also real-world impact and financial viability. After you complete a proposal, add it to your proposal backlog – a collection of potential projects waiting for the green light. You should regularly review this backlog, and move proposals up or down the priority list based on ongoing evaluations against market trends and internal priorities.

Phase 5: Develop a Minimum Viable Product (MVP)

Now is the time to put your comprehensive plan into action. This phase is about transforming your idea, which has been meticulously shaped and defined in previous phases, into a tangible and functional MVP. A critical distinction between AI software development and traditional software development, is that you’ll need an ML Ops specialist for the effective productionizing of your MVM.

ML Ops, short for Machine Learning Operations, is a set of practices that combines Machine Learning, DevOps (Development and Operations), and Data Engineering to streamline and optimize the lifecycle of machine learning and data science projects. It's akin to DevOps but tailored specifically for machine learning. The key aspects of ML Ops are essential to your MVP if AI is its killer feature, and these include careful consideration around AI-specific:

Automation and Scalability
Version Control and Management
Continuous Integration and Deployment (CI/CD)
Monitoring and Maintenance
Collaboration and Communication
Compliance and Governance

As you progress in building out the MVP, adhering to a lean development approach is crucial. This involves consistently prioritizing value over a broad feature set, focusing on the essential components of your product, and staying agile to pivot or iterate based on real-world feedback and performance data. The ultimate objective is to develop an MVP that not only validates the viability of your AI solution but is also flexible enough for rapid adaptation and enhancement in response to user needs and market dynamics.

Lean AI Development: Transform Ideas into MVMs into MVPs