How to define an AI use case using Gap analysis and Machine Learning Canvas

Irina Kolesnikova
June 21st, 2022

machine learning canvas

The latest hype around AI hides that this is a complex but, if executed properly, rewarding way to enhance business. The journey toward beneficial AI implementation starts with a business problem. However, since machines speak their own language, business leaders naturally find it a challenge to describe this problem in a way that is clear enough for the AI.

To transform a business case into an AI use case, business leaders will need the help of an AI team, who judge if and how to proceed with the problem enhancement; the problem owner who knows the most about the issue; and technical specialists who understand how the problem described by the problem owner can be interacted with in the technical world.

Having experience setting up machine learning solutions for more than 100 projects in 20 countries worldwide, MindTitan shares this guide on formulating an AI use case based on existing business challenges and how to quickly analyze which ML solution will be the most convenient to bring the most business benefits.

Table of contents

Part 1. Choose the right use cases using the AI gap analysis

  1. Define the use case
  2. Define the business processes gap
  3. Determine your data gap
  4. Assess an application gap
  5. Revise an infrastructure gap
  6. Estimate the use cases
  7. Define the first steps

Part 2. Specify the use case in detail using a machine learning canvas

  1. Value proposition block
  2. Predictions block
  3. Learning block
  4. Evaluation and monitoring block

Conclusion

Before you continue reading, we highly recommend downloading Gap analysis and Machine learning canvas templates.
 

machine learning canvas templates

Part 1. Choose the right use cases using the AI gap analysis

Discovering the right process to be enhanced with AI may be an unusual task for business people. A handy tool for this goal is the AI gap analysis. Having all the possible cases written down briefly with the most important pieces of information will soon help you put a plan together. It’s the first step to make sure you connect what AI can do to your organization’s objectives as well as assess its feasibility. Collect and describe ideas with the AI gap analysis: the main goal is to find out and sum up the info concerning current processes, future processes, how to close the gap between these two states, and setting minimal limits on the tools and procedures to be used.

Below you will find the seven steps of the gap analysis to determine the best use case for AI implementation.

  • Points 1 and 2 should be the task for business people as the first round to define the right process to enhance.
  • Points 3-6 should be the results of discussions between business and technical people to describe the process to enhance in more detail.
  • Point 7 should be a result of collaboration between all parties to prioritize use cases, think about the next steps, and create a master plan.

To be more efficient, all steps should be supervised by AI experts.

Place all answers into the table to make it easier to grasp.

1. Define the use case

The first step is to define a pain point of the business process you want to enhance with AI by asking the right questions and transforming the business case into an AI use case.

define the use case

The value proposition of the use case will help you later too, at the stage of the machine learning canvas stage.

Answer the following questions:

1. What is the value proposition of the use case?

Gather all the relevant information about the issue. You can think of it as the What+Why+Who+How:

  • What are we trying to do?
  • Why is it important?
  • Who are the users?
  • How can success be measured?

Example:
Company X wants to improve the auditing process. It is important because it will reduce the number of mistakes and speed up the process. The users are auditors inside the company. The success can be measured by comparing the number of checked cases and the percentage of mistakes

2. What is AI’s role in solving this challenge?

This helps to reveal if the business case you are analyzing is specific and realistic. At this stage, it is important to understand what exact change will improve the process: e.g., “improve the auditing process” is not suitable, but “helping to flag suspicious cases for auditors” promotes thinking and is more realistic.

3. Who is the business owner?

Adding the name and position to the table will help people look for the right person to ask a question.

2. Define the business processes gap

process gap

This step is important to understand the goal of the AI implementation and to determine whether the improvement target is significant enough to consider changes in existing processes.

Answer the following questions:

1. What does the process look like at the moment?

2. What should the process look like in the future?

The more detailed description it is, the better: this will help to communicate the task to the AI team.

Example:
At company X auditors are checking all the cases manually: it takes a lot of time and the human factor causes mistakes.

After AI implementation, if AI is highly confident about cases, those will be processed by AI automatically, cases with medium confidence will be reviewed by auditors, low confidence cases are not likely to be relevant for auditors to review and thus are only reviewed at random. The process should be at least 25% faster, and it should produce 10% fewer mistakes.

3. Determine your data gap

Determine your data gap

Data is essential for machine learning, as the machine runs on data. The need for massive amounts of data to train the machine is imperative. However, both the quantity and the quality of data are crucial to getting the desired results.

Answer the following questions:

1. What data do we have at the moment?

2. What additional data should we integrate/collect?

Specify the answers with the following:

Data sources:

  • Which raw data sources can we use? (for example, company X has all the cases recorded for 2 years.)
  • What new data sources should we include?
  • How should the current data collection process be changed if any process exists?

Features:

  • Input representations to extract from raw data sources. To a certain extent, all data is “temporal”. A customer’s account balance fluctuates over time, but their age or even their country changes as well. When you’re using that sort of information, you need to specify at which point in time to extract the value.
data engineering

Collecting data:

  • How do we get new data to learn from (inputs and outputs)?

Inputs are the objects you want to make predictions on: a house, an email, a customer. These inputs need a computer representation with numerical, categorical, or textual values.

The “feature” values must be chosen in a way that allows the characterization of inputs in such a manner that the outputs (house value, email importance, customer fragility) could be determined from the features.

Afterward, you may use these answers for further analysis and description with the help of a machine learning canvas.

4. Assess an application gap

asses the gap

Revise the interactions of end-users or other systems that bring data at the moment and in the future. Consider what integrations with existing applications might be necessary.

  • What kind of software is being used for this at the moment? For example, software that collects/records the data at the moment through the interactions of end-users.
  • What kind of software needs to be developed/implemented into use for the future state? For example, a data labelling application or integrations with another system that could use the AI predictions.

5. Revise an infrastructure gap

infrastructure gap

This point reveals if the infrastructure needs any updates for AI implementation.

  • What is the current supporting infrastructure? For example, all the data is stored in AWS S3.
  • What updates need to be implemented in the infrastructure for the future state? For example, a GPU upgrade might be needed to train an AI dealing with a computer vision task.

6. Estimate the use cases

estimations

The AI gap analysis prepares the ground for further steps, helping to choose the most relevant and promising use case. After filling the gap analysis table, analyze each use case with all parties. Taking into account the columns on the left, place the resulting answers in the columns now and put AI use cases in order, starting from the most business valuable one, thus it will be easy to compare all the rest parameters.

  • Business value if solved (High, Medium, Low)
  • Difficulty from an AI perspective (High, Medium, Low)
  • Rough time and budget estimation
  • Dependencies between different use cases
  • Technologies required

7. Define the first steps

It’s time to define the first steps with the project or projects you chose for implementation. It could be the proof of concept, data access, data collection solutions, data/business analysis, etc.

Part 2. Specify the use case in detail using a machine learning canvas

After finishing the gap analysis, you have the project (or projects) that is the most suitable for your business goals and worth implementing (no point in trying to solve a problem with AI if it can be solved more easily), define them in the machine learning canvas in even more detail. The best version of the machine learning canvas results from collaboration with an outsourced machine learning partner.

use case

The machine learning canvas lets you lay down your vision for your ML system and communicate it with your team.

This is a tool that can be updated during the project and can help when the team needs to refresh their heads about the project, and specifically is very useful if new members join the ML project.

Note that you already may use the answers you received from the gap analysis.

 

1. Value proposition block

It’s convenient to start with the value proposition block in the center dedicated to the benefits of the machine learning implementation for the exact system. You can take the What+Why+Who from the gap analysis you already made for this use case. Then there’s the How, split into two parts: learning and making predictions.

2. Predictions block

The part on the left-hand side is dedicated to predictions, based on the models that we’ll learn from the data. Filling the prediction block of the machine learning canvas will require the help of the AI team. It’s made of the following blocks:

  • Machine learning task: Which type (e.g. classification, regression…), what is the input, and what is the output to predict (along with possible values)? This could be taken from the gap analysis as well.
  • Decisions: How are predictions used to make decisions that provide the proposed value to the end­ user?
  • Making predictions: When do we make predictions on new inputs, and how long do we have for that?
  • Offline evaluation: Which methods and metrics can we use to evaluate the way predictions are going to be made and used, prior to deployment?

This part should be done together with AI and machine learning specialists who will guide you.

 

3. Learning block

The part on the right-hand side is dedicated to learning from data. It’s made of the following blocks:

  • Data sources: Which raw data sources can we use?
  • Collecting data: How do we get new data to learn from (inputs and outputs)?
  • Features: Input representations to extract from raw data sources.
  • Building models: When do we create/update models with new training data and how long do we have for that?

The first three points could be taken from the gap analysis, but be specifically careful with the data block, as its purity and suitability are the foundation of machine learning success.

Questions to pay attention to:

  1. Can data be easily accessed once the project starts?
  2. Are people available who really know the data?
  3. Are labelers available if required?
  4. Is a test data set available for evaluating the resulting AI solution?

The top of the machine learning canvas provides a background view, and the bottom goes into the peculiarities of the system. The upper left and right blocks relate to domain integration: how predictions are used and how data is collected in the domain of application. The lower left and right blocks relate to the “predictive engine” and its constraints, in terms of latency and throughput, for making predictions and updating models.

4. Evaluation and monitoring block

Finally, the last part of the machine learning canvas is dedicated to measuring how well the system works, on the domain side (“Live Evaluation and Monitoring”). This is where you’ll specify methods and metrics to evaluate the system after deployment, and to quantify value creation.

Conclusion

An AI gap analysis and an ML canvas are neat and helpful tools to transform a business problem into an AI use case to communicate and plan between different parties.

The AI gap analysis reveals the big picture of different use cases, helping to build a long-term AI strategy.
Specific descriptions for each project from the ML canvas help to communicate and transfer the ideas and vision of the AI use case between teams and team members. As you fill it in, you’ll be able to identify key constraints of your ML system, which have an impact on the technology to choose.

In the next step, with AI use cases defined, you can create an execution plan for the AI project.

To get the best results, remember: it is important to coordinate and discuss the answers with a good AI team.

ml canvas

Go back