In our last two blog posts in this series we discussed decision engine performance and how performance is impacted by deployment architecture choices. In addition to those considerations, you should also focus on business decision performance, the topic of this post.
Central to SMARTS’s approach to decision design is the idea that you need to have a strong focus on the expected business performance of your decision. The business performance of the decision is measured by multiple KPIs defined by the different business stake holders and characterize how the decision is contributing to the business.
Decision Analytics for Simulations
SMARTS provides you with fully integrated decision analytics, including aggregates, reports and dashboards that you can configure to track those KPIs. As you are implementing and optimizing the decision logic, you can run simulations to assess the impact your change have on the decision, and take appropriate action. This allows you to ensure that the business performance of your decision is actually what you want before you deploy it in production.
Real-time Decision Metrics
SMARTS also provides you streaming decision analytics, allowing you to monitor the same KPIs on the live decisions as they are deployed, and to specify alerts that trigger if those KPIs deviate from limits you can set. This gives you the peace of mind that you are always kept up to date on how well the deployed decision is behaving and that you can take early action to update it should the situation need it.
Champion/Challenger Experiments
There are also cases where it is not possible to necessarily know in advance the impact of a change. You may be exploring with new decision options you had never attempted before. SMARTS allows you to deploy your decision in an experimental mode – where part of your invocations will be routed to the new “experimental” version, and the rest to the proven one, and where you will be monitoring the relative performance to identify whether your “experimental” version is doing better than the proven one. In many financial services areas, this is called Champion-Challenger, in marketing or design, this is called A/B testing. With this approach, you can gradually and safely introduce decision optimizations that lead to better and better business performance.
In summary, when considering performance of decision management systems it is critical to consider the topic from a business perspective as well as a technical perspective. We hope this series has helped clarify performance related issues pertaining to decision management.
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager
