We routinely underestimate the long-term impact of new technologies and overestimate their short-term impact. The law, which came to be known as the “Amara Law” after the late researcher and scientist Roy Amara, is now being applied to many organizations using artificial intelligence and machine learning (AI/ML) technology. increase. From self-driving cars to image recognition to advanced robotics, AI/ML had sci-fi-like capabilities not so long ago, but more will come in the future. At the same time, even forward-thinking companies can struggle to get the expected returns from their AI/ML investments. Expectations may have been high, but by following multiple guidelines, the client was able to maximize the business value of his AI/ML initiative. We share them in this AI/ML blog post mini-series.
No magic: recognize uncertainty
The pressure to deliver solutions faster and at lower cost while ensuring compliance, security, and reliability has put many IT departments in a bind. So it’s no wonder they are looking for new solutions to break down old barriers. And it’s no surprise.
Combining the latest technologies with new ways of working can transform IT from a cost center to a powerhouse of innovation. But the latest technology is too often touted as a panacea that can fix all the shortcomings of its predecessors. So while modern IT solutions can overcome previous limitations, it is important to remember that neither AI/ML nor any other technology has algorithms that “improve customer satisfaction”. AI/ML solutions model human thinking by allowing computers to make decisions (“inference”) based on past positive or negative outcomes. As with human thinking, results remain educated guesses. Accuracy can be increased by building more sophisticated models, but the business value of that accuracy must be weighed against the cost of building and training the model.
Pre-built business models that treat AI/ML initiatives as if they were deterministic software programs risk exaggerating expectations. You can’t expect an out-of-the-box AI/ML model to perfectly match your business problem and make good decisions out of the box. The value a solution delivers depends largely on how well it balances investment and return in the face of uncertainty. For example, if you want to use a new data source for ML (or any other analytical process), you face a high degree of uncertainty both in terms of required investment and potential benefits. Suppose you want your ML product to use the daily inventory levels of warehouses around the world to determine which items to promote. Whether the data quality is good enough for automated processing, how much it costs to reconcile data from different warehouses, whether daily inventory levels are relevant to your business processes, how this data will be used I don’t know if I can use it to improve my business. Top or final KPI.
Many companies have been frustrated with the “put all your data in one data lake and find value later” approach. The addition of “artificial intelligence” will not change that. Instead, reduce uncertainty step by step and learn which investments impact company value. Building on cloud analytics and AI/ML platforms is a great way to limit the initial investment when learning starts. Along the way, use his three mechanisms to continuously improve and refine yourself.
1. Recognize the value of right decisions and the cost of wrong decisions
Suppose you want to use ML to improve sales forecasting for consumer products. This is a common use case that we help many clients with. Such a system can produce several positive business outcomes, such as increased net sales due to stock-out avoidance, increased customer satisfaction due to faster delivery times, and cost savings due to increased operational efficiency. A perfect ML model works equally well on all three, but no ML model is perfect.
ML models are not transparent. They are wrong on both counts, no matter how well trained they are. The cost of these errors is determined by your organization and can be asymmetric. For example, overstocking seasonal products can be as damaging to your business as understocking seasonal products. Some customers are willing to wait five days for a high-demand item if they have a fixed delivery date. Considering the cost of goods sold and storage costs, it may be more cost-effective for businesses to stock goods in a central warehouse and pay a premium for delivery, rather than having specialized or regional warehouses. there is. As with all of his IT endeavors, a clear understanding of business economics is critical to improving bottom line.
Identifying potential costly mistakes early in an AI/ML project balances expectations and enables early adjustments. It also helps avoid creating ML models that are accurate to cents but sacrifice pounds. These errors can take many forms, so let’s look at some possible scenarios for retailer sales forecasts.
The cost of these mistakes depends on the dimension of business value you are looking at. It’s tempting to fine-tune AI/ML models to avoid all the costly mistakes, but doing so involves integrating entirely new data sources or spinning off models designed for a particular subset of items. Costs will increase as the need may arise. So look beyond the actual model to avoid costly mistakes. For example, forecasts can be tied more closely to pricing strategies, campaign calendars, immediate customer feedback, or route optimization.
Sounds complicated? That’s the point. Realistic cost and benefit estimates are not one-dimensional, one-off calculations. This is true for both traditional analytics and AI/ML efforts.
2. Measure and improve over time
The path to business value is not linear and requires an ongoing process of mitigating uncertainty by gathering additional information from actionable AI/ML models. By continuing to conscientiously measure, learn, and recalibrate the model along each dimension of value, the likelihood of making good decisions increases over time. For this reason, Michelle Lee, vice president of Amazon Machine Learning Solutions Labs, warns:
This does not mean that you should recalculate your business value with every sprint or small experiment. Instead, the AI/ML development lifecycle offers some natural milestones to summarize trends.
Estimate upper and lower ROI early on based on use cases and industry benchmarks. This can be achieved with minimal effort of prioritizing use cases to come up with ideas.
Proof of concept:
Conduct a proof of concept using your own data, IT landscape, and business processes to estimate your ROI. It requires a small investment, but can be used to identify the most promising use cases that merit a larger investment.
Minimum Viable Product (MVP):
Build products with minimal viability to validate ROI in real business scenarios. This moderate investment helps create a full-featured product business model.
Fully functional product:
Form a stable cross-functional team that takes ownership of AI/ML products. Just because the operation is complete doesn’t mean the work is done. The team constantly measures the costs and benefits of solutions to improve them and adapt to changing conditions.
3. Know when to switch
By iteratively calculating the business value of your AI/ML solution, you can adapt your solution to your insights. Companies often calculate detailed business scenarios in advance, but those scenarios are often based on unrealistic and inaccurate assumptions. When these assumptions later correspond to reality, the company changes project results to fit the original business case. However, the only thing it accomplishes is taking away business value. Be prepared to readjust and change course, such as adjusting model types, acquiring additional (perhaps external) data sources, or adjusting business processes. In some cases, you may need to switch to another use case entirely. Recall the forecast example above.
Instead of trying to predict future demand more accurately, moving to a live view of all inventory (whether in production, storage, or in transit) allows customers to You will be able to respond to demand quickly and efficiently. Similarly, leverage can be used to influence demand by dynamically adjusting prices and campaigns rather than treating them as open-ended inputs. This not only reduces the forecast error rate, but also increases full price sales. It might be worth testing to see if search terms and customer comments on your site have the power to predict short-term demand.
No one can predict what the most important steps will be for your ML model and your business. that’s ok. ML solutions are not copy-paste exercises based on the successes of other organizations. These are tools for maximizing the value you get from your assets or for overcoming certain constraints. Using new technology means constant recalibration
A deeper understanding of AI/ML techniques can help you achieve greater business value. However, much more is possible with a comprehensive understanding of how business dynamics interact with new technology capabilities.
AI/ML is therefore not just a technology project. Please remember the following important points:AI/ML models make mistakes. To properly assess the value to your business, you must be able to weigh the impact of making a mistake against the benefits of making the right decision. Real business impact is based on the complex dynamics of all KPIs and is difficult to pinpoint before you start. Instead, gradually reduce uncertainty through experimentation and continuous learning.
You should be willing to make changes based on what you learn, even if that means abandoning ideas that were once good. Be prepared to incorporate the solution into real business processes early on.
These steps go hand in hand with making your business data-driven, whether it employs AI/ML technologies or traditional analytics. Each of the above calculations may change over time depending on business and customer demand. Integrating this development into a sustainable AI/ML software lifecycle will be the topic of our next post.