In our last post, we looked at how predictive models are used in automated decisions. A key take away from that post is that a prediction is not a decision. Rather predictive models provide us with key insights based on historical data so we can make more informed decisions.
For example, a predictive model can identify customers that are likely to churn, transactions that are suspicious, and offerings and ads that are likely to have the most appeal. But, based on these predictions, we still need to decide the best response or course of action. A decision combines one or more predictions with business knowledge and expertise to define the appropriate actions.
From Predictions to Decisions
Determining how to take action based on predictions is not trivial . Most likely, there are multiple business options for the actions an organization could take based on a prediction. Consider for example charges that are identified as potentially fraudulent, a card issuer could report the case to the fraud team for further investigation, shut down the card to prohibit further charges, or text the cardholder to verify the charge.
Or in the case of customers who have been identified as having a high probability of switching to a competitor, a company may decide to contact them with a special incentive or renewal offer. But the company still needs to decide exactly how many customers will receive the offer and how much to offer. The company could target a flat percentage of customers, or could focus only on those with the highest projected CLTV (customer lifetime value). Targeting too many customers with too large off an offer might be too expensive to be worthwhile. The possible actions an organization can take based on predictions have different costs and benefits that need to be evaluated to determine the optimal decision. This is where decision simulation is applicable. Decision simulations help you identify the best decision strategy from amongst a set of alternatives.
Measure Your Decision Quality with KPIs and Metrics
The “best” decision strategy means the one that most closely meets your organization’s objectives. By defining KPIs and metrics that measure the quality of the decision in relation to these objectives, we have a basis to compare alternative decision approaches. Ideally these KPIs were identified early on, when you first decided to automate the decision.
Decision KPIs give us a clear understanding of how decision performance is related to business performance. They provide the basis for evaluating decision alternatives. To compare alternatives, you can run simulations using historical data. Using simulations you can compare one decision strategy to another, or you can compare how a given strategy performs on each of your customer segments as represented in your data.
Returning to the above customer churn example, we may decide we want to target customers who have an 80% or greater probability of churn based on our predictive model. One option would be to offer them a special 25% discount to attempt to re-engage and keep them as a customer. We can run a simulation against our historical data to learn how many customers fall into this bucket. From there, we can evaluate how much the discount offer would likely cost us. We can run multiple simulations using different thresholds, offers, and combinations until we find the best decision approach to deploy.
Decision Simulation Helps You Evaluate Alternative Decision Strategies
Decision simulations help us evaluate alternative decision strategies to narrow to the best approach. Modern decision management technologies, like SMARTS, make it easy to set up and run these simulations, even on very large data samples. Of course, the ultimate quality of the selected decision approach is related to its success once deployed- how many customers do we manage to retain and at what cost?
Once we deploy a decision we can monitor and track the KPIs but we have no way of knowing whether customers who did not accept our offer would have instead accepted a different offer. Or whether customers who did accept would also have accepted a 20% rather than a 25% discount. To answer these questions we need to use Champion / Challenger experiments. We’ll cover how Champion / Challenger works with decision management in an upcoming post.
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager
