Our Best Practices Series has focused, so far, on authoring and lifecycle management aspects of managing decisions. This post will start introducing what you should consider when promoting your decision applications to Production.
Make sure you always use release management for your decision
Carole-Ann has already covered why you should always package your decisions in releases when you have reached important milestones in the lifecycle of your decisions: see Best practices: Use Release Management. This is so important that I will repeat her key points here stressing its importance in the production phase.
You want to be 100% certain that you have in production is exactly what you tested, and that it will not change by side effect. This happens more frequently than you would think: a user may decide to test variations of the decision logic in what she or he thinks is a sandbox and that may in fact be the production environment.
You also want to have complete traceability, and at any point in time, total visibility on what the state of the decision logic was for any decision rendered you may need to review.
Everything they contributes to the decision logic should be part of the release: flows, rules, predictive and lookup models, etc. If your decision logic also includes assets the decision management system does not manage, you open the door to potential execution and traceability issues. We, of course, recommend managing your decision logic fully within the decision management system.
Only use Decision Management Systems that allow you to manage releases, and always deploy decisions that are part of a release.
Make sure the decision application fits your technical environments and requirements
Now that you have the decision you will use in production in the form of a release, you still have a number of considerations to take into account.
It must fit into the overall architecture
Typically, you will encounter one or more of the following situations
• The decision application is provided as a SaaS and invoked through REST or similar protocols (loose coupling)
• The environment is message or event driven (loose coupling)
• It relies mostly on micro-services, using an orchestration tool and a loose coupling invocation mechanism.
• It requires tight coupling between one (or more) application components at the programmatic API level
Your decision application will need to simply fit within these architectural choices with a very low architectural impact.
One additional thing to be careful about is that organizations and applications evolve. We’ve seen many customers deploy the same decision application in multiple such environments, typically interactive and batch. You need to be able to do multi-environment deployments a low cost.
It must account for availability and scalability requirements
In a loosely coupled environments, your decision application service or micro-service with need to cope with your high availability and scalability requirements. In general, this means configuring micro-services in such a way that:
• There is no single point of failure
○ replicate your repositories
○ have more than one instance available for invocation transparently
• Scaling up and down is easy
Ideally, the Decision Management System product you use has support for this directly out of the box.
It must account for security requirements
Your decision application may need to be protected. This includes
• protection against unwanted access of the decision application in production (MIM attacks, etc.)
• protection against unwanted access to the artifacts used by the decision application in production (typically repository access)
Make sure the decision applications are deployed the most appropriate way given the technical environment and the corresponding requirements. Ideally you have strong support from your Decision Management System for achieving this.
Leverage the invocation mechanisms that make sense for your use case
You will need to figure out how your code invokes the decision application once in production. Typically, you may invoke the decision application
• separately for each “transaction” (interactive)
• for a group of “transactions” (batch)
• for stream of “transactions” (streaming or batch)
Choosing the right invocation mechanism for your case can have a significant impact on the performance of your decision application.
Manage the update of your decision application in production according to the requirements of the business
One key value of Decision Management Systems is that with them business analysts can implement, test and optimize the decision logic directly.
Ideally, this expands into the deployment of decision updates to the production. As the business analysts have updated, tested and optimized the decision, they will frequently request that it be deployed “immediately”.
Traditional products require going through IT phases, code conversion, code generation and uploads. With them, you deal with delays and the potential for new problems. Modern systems such as SMARTS do provide support for this kind of deployment.
There are some key aspects to take into account when dealing with old and new versions of the decision logic:
• updating should be a one-click atomic operation, and a one-API call atomic operation
• updating should be safe (if the newer one fails to work satisfactorily, it should not enter production or should be easily rolled back)
• the system should allow you to run old and new versions of the decision concurrently
In all cases, this remains an area where you want to strike the right balance between the business requirements and the IT constraints.
For example, it is possible that all changes are batched in one deployment a day because they are coordinated with other IT-centric system changes.
Make sure that you can update the decisions in Production in the most diligent way to satisfy the business requirement.
Track the business performance of your decision in production
Once you have your process to put decisions in the form of releases in production following the guidelines above, you still need to monitor its business performance.
Products like SMARTS let you characterize, analyze and optimize the business performance of the decision before it is put in production. It will important that you continue with the same analysis once the decision is in production. Conditions may change. Your decisions, while effective when they were first deployed, may no longer be as effective after the changes. By tracking the business performances of the decisions in production you can identify this situation early, analyze the reasons and adjust the decision.
In a later installment on this series, we’ll tackle how to approach the issue of decision execution performance as opposed to decision business performance.
How to Keep and Delight Fickle Customers in a Competitive Insurance and Fintech Markets
Are your customers fickle? How well do you anticipate their needs, proactively offer packages at a competitive prices, react to regulatory and competitive changes before they leave you?
Today’s banks, insurance and financials firms operate in a fast moving, highly competitive and rapidly changing market. Disruption is everywhere and the customers have choices they can make in an instant from their smart phones. Losing a customer to a more nimble competitor can be as quick as a cup of coffee with a few finger swipes at a Starbucks patio.
Particularly in the insurance market, customer interactions are precious and few. An insurance company rep needs to not only delight their customer when an opportunity arises, but also upsell them by offering them a personalized product or service tailored to their need virtually instantly.
Doing the same thing as before is a certain way to lose business
Nimble competitors now use the latest AI and analytics technology to rapidly discover and deploy intelligent decision systems which can instantly predict customer needs and customize the offering and pricing relevant for the customer at the right time.
To achieve and sustain such flexibility, a financial organization needs to modernize its underlying systems. Best companies build a living decision intelligence into their systems, ready to be updated on a moments notice.
If a competitor offers a better deal, customer has an life event or data analysts discover a new pattern for risk or fraud, core systems need to be updated virtually instantly. By having an intelligent, AI-driven central decision management system as the heart of your core system, anyone in your organization can have the latest intelligence at their fingertips. Intelligent systems will help verify customer eligibility, provide custom product or a bundle offering at a competitive price, speed up and automate claim adjudication and automate loan origination across all sales and support channels.
The heart of this solution is a modern, AI-driven decision management and rule engines platforms that use the latest AI and analytics techniques, have sophisticated cloud offerings providing unparalleled flexibility and speed. Best systems are no longer just for the IT – they allow most business analysts to view, discover, test and deploy updated business logic in a real time.
A modern organization needs the latest decision analytics tools
These tools will allow you to discover new patterns from the historical data using machine learning, connect and cross correlate multiple sources of data and incorporate predictive models from company’s data analysts. Updating and deploying new logic is now as easy as publishing a web page and does not require changing the application itself, just the underlying business logic.
Sparkling Logic SMARTS AI Decision Management is the third and the newest generation of the decision management and rules engine offering using cloud, AI, decision analytics, predictive models and machine learning. We currently process millions of rules and billions of records for the most progressive companies. Find out how we succeeded in creating the ultimate sophisticated set of decision management and decision analytics tools that every modern financial institution should have in their competitive tool chest.
Automated decisions are at the heart of your processes and systems. These operational decisions provide the foundation for your digital business initiatives and are critical to the success and profitability of your business. Learn how SMARTS Decision Manager lets you define agile, targeted, and optimal decisions and deploy them to highly available, efficient, and scalable decision services.
Get the Whitepaper
Decision Management has been a discipline for Business Analysts for decades now. Data Scientists have been historically avid users of Analytic Workbenches. The divide between these two crowds has been crossed by sending predictive model specifications across, from the latter group to the former. These specifications could be in the form of paper, stating the formula to implement, or in electronic format that could be seamlessly imported. This is why PMML (Predictive Model Markup Language) has proven to be a useful standard in our industry.
The fact is that the divide that was artificially created between these two groups is not as deep as we originally thought. There have been reasons to cross the divide, and both groups have seen significant benefits in doing so.
In this post, I will highlight a couple of use cases that illustrate my point.
In the early days, we were very focused on knowledge. Figuring out how to extract, capture and model knowledge biased our approach to business rules the same way that we can get obsessed with nails once we have a hammer in hand.
I am not saying that knowledge isn’t important or valuable of course.
The point I want to make is that knowledge in the abstract isn’t as valuable as it could be with data. Data is the life blood of decision management. I came to realize that a few years ago, once I finally took a step back to rethink where we were at in this industry. It was ironic that we did not see that working for an analytics company back then.
Last week we jointly hosted a webinar with our consulting and implementation partner, Mariner. Shash Hedge, BI Solution Specialist from Mariner, described operational BI, its challenges, and some traditional and recent implementation approaches. He concluded with a few cases studies of operational BI projects that were missing an important piece — the ability to make decisions based on the operational insight provided by the system.
Operational BI systems provide critical insight on business operations and enable your front-line workers to make more informed decisions. But as Shash highlighted, insight delivered in the right format, to the right people, at the right time is often not enough, you need to make decisions based on that insight in order to take action…
I lead the second half of the webinar, introducing decision management and describing how it complements operational BI. Watch the recorded webinar to learn more.
The recording is a bit rough when the video gets to my part; it sounds like I am presenting from another country! We’re planning another joint webinar in May where we will cover the topic in more depth and demonstrate how these two technologies complement each other. Stay tuned for dates and registration information. I’m sure we’ll get the sound issues resolved next time!
I joined the Churchill Club this morning for an exciting breakfast on Machine Learning. In May 2013, Steve Jurvetson of DFJ said on the Churchill Club stage that he believes machine learning will be one of the most important tech trends over the next 3-5 years for innovation and economic growth. I was eager to hear what Peter Norvig and the other guys would say about that.
What might be surprising is that none of them painted an ‘unfathomable’ picture of the future. It was all about more power, faster modeling, more data…
I can’t say that they shared a vision… I wonder if we have all been dreaming in our young years, watching Star Trek, and super-computers fueled our imagination. Super smart machine able to assist the crew, and eventually perform medicine or look for their ‘humanity’, is the vision. We are all working hard at figuring out ways we can make it real, ways we can build technology that achieve the ideals we grew up dreaming about.
It has been a rocky road for Artificial Intelligence, but in the past few years, Watson, the self driving car and other wonders have made us believe that machine learning could actually live up to our expectations, and more.
Automating decisions has its own Return on Investment (ROI), but it is only the very beginning of a Decision Management transformation. The end goal should be to improve your decisions. Having the underlying decision logic out of the system code, gives you opportunities to analyze, understand and experiment; which was not really possible before.
It used to take time to ensure that your decision logic did what it was supposed to. Business Rules had to be implemented as code, then compiled, then tested in QA, then deployed, then eventually executed for real and the produced Production reports would tell you if there is a problem.
The sooner you actually see the effect of those business rules on your production data, the sooner your can correct the course of action, or feel safe about pushing those rules into Production.
What you can do… is to operationalize the gathering of data from your Production systems, and get them into your Decision Management systems to see how your business rules will be applied.
Measure early, measure often
Testing is good, and allows you to reduce the typos and logic errors in your business rules. You can see the raw impact of your decision logic before it gets out the door. What will make a key difference on your bottom-line though is how this new or update decision logic will behave in aggregate.
What you can do… is to measure Key Performance Indicators (KPIs) in your systems. KPIs are aggregated metrics that measure your business performance, your success. For example, you might care about the distribution of Approve, Decline and Refer decisions. but that datapoint alone is not sufficient: you want to make sure you decline the bad risk, keep the good risk, while at the same time do not undercut your revenue. In the Fraud case study I presented with ebay, we had a different set of key metrics that were critical to the fraud expert: Catch rate and Hit rate. Whatever those KPIs might be for your business, make sure you define them carefully, and that you measure continuously the progress you make towards them.
Look for more
There is what you see in the KPI reports, and there is what you don’t see… Why don’t you get an extra help from the super processing power of some analytics? they will likely not know better than you, but they can uncover some patterns in your historical data that you can refine and operationalize.
What you can do… is to crunch your data using analytic algorithms that help you ‘mine your business rules’. Once you obtain the data-driven rules, massage them
There might be more than one way to improve your bottom-line. If you are implementing compliance rules, then you may not have that many options to experiment on; but if you are looking to improve your profitability you might have to try things for real into multiple segments to see for yourself which one is most effective.
What you can do… is to start by setting up your simulation environment to run those business rules ‘comparisons’ at large-scale. It will give you a more realistic idea of your KPIs based on a larger sample. The next step is to start experimenting live. Marketers have done A-B testing for a long time. In the Decision Management space, we call that experimental design or champion-challenger.
You are the business expert, right? But how well do you know the contribution of your rules? The decisions we make in life do not always pan out exactly how we expected them to, sometimes for the better and sometimes not… It is the same for your business decisions. They generally work the way you expect but there could be surprises.
You could measure Key Performance Indicators (KPIs) in your systems and look at the reports on a regular basis. we recommend that of course.
What you could do too that is even more powerful, a greater learning experience… is to take the time to anticipate where you expect those KPIs to land and where you would like to take them. With clear objectives in mind, you will be more attuned to the outcome and, as a result, more effective in affecting those KPIs.
If you are interested in this topic and would like to hear practical illustrations of these techniques, please join us for a webinar on 9/19 or 10/3 on “Decisions by the Numbers”.
In the decade (or two) I have spent in Decision Management, and Artificial Intelligence at large, I have seen first-hand the war raging between knowledge engineers and data scientists. Each defending its approach to supporting ultimately better decisions. So what is more valuable? Insight from data? Or knowledge from the expert?
Mike Loukides wrote a fantastic article called “The unreasonable necessity of subject experts“ on the O’Reilly radar, that illustrates this point very well and provides a clear picture as to why and how we would want both.
Data knows stuff that experts don’t
In the world of uncertainty that surrounds us, experts can’t compete with the sophisticated algorithms we have refined over the years. Their computational capabilities goes way above and beyond the ability of the human brain. Algorithms can crunch data in relatively little time and uncover correlations that did not suspect.
Adding to Mike’s numerous example, the typical diaper shopping use case comes to mind. Retail transaction analysis uncovered that buyers of diapers at night were very likely to buy beer as well. The rationale is that husbands help the new mom with shopping, when diapers run low at the most inconvenient time of the day: inevitably at night. The new dad wandering in the grocery store at night ends up getting “his” own supplies: beer.
Mike warns against the pitfalls of data preparation. A hidden bias can surface in a big way in data samples, whether it over-emphasizes some trends or cleans up traces of unwanted behavior. If your data is not clean and unbiased, value of the data insight becomes doubtful. Skilled data scientists work hard to remove as much bias as they can from the data sample they work on, uncovering valuable correlations.
Data knows too much?
When algorithms find expected correlations, like Mike’s example of pregnant women being interested in baby products, analytics can validate intuition and confirm fact we knew.
When algorithms find unexpected correlations, things become interesting! With insight that is “not so obvious”, you are at an advantage to market more targeted messages. Marketing campaigns can yield much better results than “shooting darts in the dark”.
Mike raises an important set of issues: Can we trust the correlation? How to interpret the correlation?
Mike’s article includes many more examples. There are tons of football statistics that we smile about during the Super Bowl. Business Insider posted some even more incredible examples such as:
- People who dislike licorice are more likely to understand HTML
- People who like scooped ice cream are more likely to enjoy roller coasters than those that prefer soft serve ice cream
- People who have never ridden a motorcycle are less likely to be multilingual
- People who can’t type without looking at the keyboard are more likely to prefer thin-crust pizza to deep-dish
There may be some interesting tidbit of insight in there that you could leverage. but unless you *understand* the correlation, you may be misled by your data and make some premature conclusions.
Expert shines at understanding
Mike makes a compelling argument that the role of the expert is to interpret the data insight and sort through the red herrings.
This illustrates very well what we have seen in the Decision Management industry with the increased interplay between the “factual” insight and the “logic” that leverages that insight. Capturing expert-driven business rules is a good thing. Extracting data insight is a good thing. But the real value is in combining them. I think the interplay is much more intimate than purely throwing the insight on the other side of the fence. You need to ask the right questions as you are building your decisioning logic, and use the available data samples to infer, validate or refine your assumptions.
As Mike concludes, the value resides in the conversation that is raised by experts on top of data. Being able to bring those to light, and enable further conversations, is how we will be able to understand and improve our systems.