In this post, I briefly introduce SMARTS Real-Time Decision Analytics capability to manage the quality and performance of operational decisions from development, to testing, to production.
Decision performanceH. James Harrington, one of the pioneers of decision performance measurement, once said, “Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” This statement is also true for decision performance.
Measuring decision performance is essential in any industry where a small improvement in a single decision can make a big difference, especially in risk-driven industries such as banking, insurance, and healthcare. Improving decisions in these sectors means continuously adjusting policies, rules, prices, etc. to keep them consistent with business strategy and compliant with regulations.
Decision performance management in SMARTSSMARTS helps organizations make their operational decisions explicit, so that they can be tested and simulated before implementation —- thereby reducing errors and bias. To this end, we added a real-time decision analytics capability to the core decision management platform.
Currently used in financial and insurance services, it helps both business analysts and business users to define dashboards, assess alternative decision strategies, and measure the quality of performance at all stages of the lifecycle of decision management — all with the same interface without switching from one tool to another.
Development. From the start, SMARTS focuses the decision automation effort on tangible business objectives, measured by Key Performance Indicators (KPIs). Analysts and users can define multiple KPIs through graphical interactions and simple, yet powerful formulas. As they capture their decision logic, simply dragging and dropping any attribute into the dashboard pane automatically creates reports. They can customize these distributions, aggregations, and/or rule metrics, as well as the charts to view the results in the dashboard.
Testing and validation. During the testing phase, analysts and users have access to SMARTS’ built-in map-reduce-based simulation environment to measure these metrics against large samples of data. Doing so, they can estimate the KPIs for impact analysis before the actual deployment. And all of this testing work does not require IT to code these metrics, because they are transparently translated by SMARTS.
Execution. By defining a time window for these metrics, business analysts can deploy them seamlessly against production traffic. Real-time decision analytics charts display the measurements and trigger notifications and alerts when certain thresholds are crossed or certain patterns are detected. Notifications can be pushed by email, or generate a ticket in a corporate management system. Also, real-time monitoring allows organizations to react quickly when conditions suddenly change. For example, under-performing strategies can be eliminated and replaced when running a Champion/Challenger experiment.
Uses casesInsurance underwriting. Using insurance underwriting as an example, a risk analyst can look at the applicants that were approved by the rules in production and compare them to the applicants that would be approved using the rules under development. Analyzing the differences between the two sets of results drive the discovery of which rules are missing or need to be adjusted to produce better results or mitigate certain risks.
For example, he or she might discover that 25% of the differences in approval status are due to differences in risk level. This insight leads the risk analyst to focus on adding and/or modifying your risk related rules. Repeating this analyze-improve cycle reduces the time to consider and test different rules until he or she gets the best tradeoff between results and risks.
Fraud detection. An other example from a real customer case is flash fraud where decisions had to be changed and new ones rolled out in real time. In this case, the real-time decision analysis capability of SMARTS was essential so that the customer could spot deviation trends from normal situation directly in the dashboard and overcome the flood in the same user interface, all in real time.
Without this built-in capability, the time lag between the identification of fraud and the implementation of corrective actions would have been long, resulting in significant financial losses. In fact, with SMARTS Real-Time Decision Analytics, the fraud management for this client has gone from 15 days to 1 day.
Marketing campaign. The two above examples are taken from financial services but SMARTS real-time decision analytics helps in any context where decision performance could be immediately affected by a change in data, models, or business rules, such as in loan origination, product pricing, or marketing promotion.
In the latter case, SMARTS can help optimize promotion in real-time. Let’s say you construct a series of rules for a marketing couponing using SMARTS Champion/Challenger capability. Based on rules you determine, certain customers will get a discount. Some get 15% off (the current offering — the champion), while others get 20% (a test offering — the challenger). And you wonder if the extra 5% discount leads to more coupons used and more sales generated. With SMARTS real-time decision analytics environment, you find out the answer as the day progresses. By testing alternatives, you converge to the best coupon strategy with real data and on the fly.
ConclusionAs part of the decision lifecycle, business analysts obviously start by authoring their decision logic. As they progress, testing rapidly comes to the forefront. To this end, SMARTS integrates predictive data analytics with real-time decision analytics, enabling business analysts and business users to define dashboards and seamlessly associate metrics with the execution environment — using the same tool, the same interface, and just point and click.
- SMARTS comes with built-in decision analytics — no additional or third-party tool is required
- You can define metrics on decision results so you can measure and understand how each decision contributes to your organization’s business objectives
- Decision metrics enable you to assess alternative decision strategies to see which should be kept and which rejected
- SMARTS add-on for real-time decision analytics lets you monitor the decisions being made and make adjustments on the fly
- SMARTS’ real-time decision analytics helps in any context where decision performance could be immediately affected by a change in data, models, or business rules
AboutSparkling Logic is a decision management company founded in the San Francisco Bay Area to accelerate how companies leverage data, machine learning, and business rules to automate and improve the quality of enterprise-level decisions.
Sparkling Logic SMARTS is an end-to-end, low-code no-code decision management platform that spans the entire business decision lifecycle, from data import to decision modeling to application production.
Carlos Serrano is Co-Founder, Chief Technology Officer at Sparkling Logic. You can reach him at email@example.com.
Low-code no-code is not a new concept to Sparkling Logic. From the beginning, the founders wanted to deliver a powerful yet simple product, so that a business analyst could start with data and build decision logic with built-in predictive data analytics and execution decision analytics.
Version after version, they have achieved this vision through SMARTS, an end-to-end decision management platform that features low-code no-code for business analysts and business users to develop and manage decision logic through point-and-click operations.
For business analysts, SMARTS offers a low-code development environment in which users can express decision logic through a point-and-click user interface to connect data, experiment with decisions, monitor execution without switching between different tools to get the job done. Depending on the nature of the decision logic at hand and user preferences, business analysts can choose on the fly the most appropriate representation to capture or update their decision logic. The resulting decision logic is seamlessly deployed as a decision service without IT intervention.
To push the simplification even further, Sparkling Logic founders turned to their customers for inspiration on their needs and developed three complementary technologies:
- RedPen, a patented point-and-click technology that accelerates rule authoring without a need to know rule syntax or involve IT to author the rules
- BluePen, another patented point-and-click technology to quickly create or leverage a data model and put it into production without involving data scientists or IT
- A dynamic questionnaire to produce intelligent front-ends that reduces the number of unnecessary or redundant questions
In addition to low-code development capability for business analysts, SMARTS also elevates the decision logic to a simple web form-based interface for untrained business users. They can configure their decision strategies, test the updated decision logic, and promote the vetted changes to the next staging environment — without learning rules syntax.
These business apps offer a business abstraction for most tasks available in SMARTS related to configuration, testing and simulation, invocation and deployment management, as well as administration.
For example, credit risk specialists can configure loans, credit cards, and other banking products, and pricing specialists can control pricing tables, through a custom business app specific to their industry. The no-code business app enables business users to cope with environment changes whether they are related to internal policies, competition pressure, or industry regulation.
Furthermore, SMARTS tasks can also be automated through orchestration scripts. Business users can trigger these scripts through the click of a button, or schedule them to be performed automatically and seamlessly.
Sparkling Logic is a decision management company founded in the Bay Area to accelerate how companies leverage internal and external data and models to automate and improve the quality of enterprise-level decisions.
Sparkling Logic SMARTS is an end-to-end, low-code no-code decision management platform that spans the entire business decision lifecycle, from data import to decision modeling to application production.
Regardless of its market, in order to thrive, a business must manage its daily operations through well-defined, executed, and controlled processes and decisions. The top performers automate both with business process management (BPM) and business decision management (BDM).
Generally speaking, BPM is the set of technologies that automate processes of an organization, that is to say the stages through which the organization passes. Often, the stages include data entry, transformation, and enrichment. Then data activation, exploitation, and reporting. On the other hand, BDM is the set of technologies that automate the multitude of operational decisions of an organization. The number often varies from hundreds to thousands, and sometimes millions of times a day.
As is always the case in technology, there is no clear distinction between BPM and BDM. Suppliers of both technologies operate in both markets, and companies combine them to maximize the benefit of each technology. The purpose of this article is to draw a line between the two technologies through an example from subscription-based businesses such as Amazon Prime, Netflix, and WSJ, and a second example from life, property, and causality insurances.
Processes and decisions
Subscription-based businesses such as those listed above take the form of a billing relationship with a starting subscription event, a continuous servicing or insuring process, and an eventually un-subscription event.
Figue 1: Simplified lifecycle of subscription to a media
Figure 1 presents a high-level view of the processes and decisions that take place behind the websites of Amazon Prime, Netflix, and WSJ. Figure 2 does the same for the life, property, and causality insurances. When you take a close look to the diagrams, you may wonder what a process is and what is a decision. In fact, both are taking place within these two examples.
Figure 2: Simplified lifecycle of subscription to an insurance
The key difference
The processes are stable enough in the sense that they do not change, because they are independent of each situation and indeed of each data. Whether it is a media or an insurance, the company always starts by prospect acquisition, then customer activation, and then customer management.
On the other hand, the decisions change too often, depending on the situation captured through data. Take the subscription to a streaming media (Figure 1). Ending the contract can be a tricky decision. Should the media end the contract the date of the subscriber’s call to cancel? Or should it try to propose a discount to keep the subscriber? Should the media recall subscribers who have not paid their last bill? A week or two weeks after the due date? These are the typical decisions that separate the media from each other. Each media has its own specific way to investigate customers when the relationship has ended.
Now, take the subscription to a car insurance (Figure 2). The insurance, like any other insurance competing for the same policies, searches to augment its business. But at the same time, it wants to reduce its risks. So decisions are more complex than in the case for a media. The subscriber may not be eligible for different reasons and to decide, third-party data may be needed. Also, the subscriber may present different risks depending on the age, car, and accident history. So the insurance has to take another series of decisions: Compute a risk level, then a price that hedges the risk.
In short, the processes are what make a company belonging to an industry, the decisions are what make the company unique in this industry.
BPM, BDM, and standardization
Under the pressure of competition on the one hand and regulation on the other, many service companies have realized the importance of separating decisions from processes. Until recently, they were nested. Leading this trend, the Object Management Group (OMG) has published two recommendations (BPMN for the former, DMN for the latter), thus accelerating the emergence of BPM and BDM as two different yet complementary technologies.
In practice, BPM technologies are appropriate when the problem is centered on a document that must be co-signed by different stakeholders. For example, a loan contract that passes from the Sales to Finance, then to the Legal department, then back to the Sales, and finally to the client.
On the other hand, BDM technologies are more appropriate when the problem is centered on operationalizing decisions. For instance, evaluating the eligibility for a loan, accepting or rejecting the application, and so on.
- Regardless of its market, in order to thrive, a business must manage its daily operations through well-defined, executed, and controlled processes and decisions
- Processes are what make a company belonging to an industry, decisions are what make it unique in this industry
- BPM is the set of technologies that automate processes, that is to say the stages through which the organization passes. BDM is the set of technologies that automate the daily decisions
- In practice, BPM technologies are appropriate when the problem is centered on a document that must be co-signed by different stakeholders. BDM technologies are more appropriate when the problem is centered on operationalizing decisions
The statements in this article belong solely to the author. The article was not reviewed nor endorsed by any company or organization mentioned. You can send your comments to the author at firstname.lastname@example.org.
You think that authoring business rules is difficult? If you attend a decision management show, you will likely hear many business analysts share their frustration while struggling with their first rules project. For instance, I remember vividly ever-lasting conversations aimed at defining what a rule is. As a consequence, I made it my personal mission to address this challenge. Business analysts should be able to log into their decision management tool and feel confident.
Clearly, authoring decision logic is not rocket science, but the tooling you use plays an important role. In other words, if that tool looks like a software development environment, it will only feel intuitive to software developers, which, as a business person, you might consider rocket science! Overall, look for advanced capabilities that support what you need, as a business analyst:
- Get started writing Business Rules intuitively in the context of data
- Adapt your view depending on the task at hand, switching from text to a decision table, tree or graph
- Combine business rules with very large MS Excel spreadsheets for efficient lookups
- Design a business app, for those that need to access the decision logic in their own terms
Intuitive authoring in the context of data
As a business analyst, your business is your comfort zone. For example, an insurance business analyst will speak about applications, eligibility criteria, claims, and more. Asking that person to think in terms of “IF” and “THEN” and rules cannot feel natural out of the box. The solution is to bridge those two worlds by overlaying decision logic in the context of data. That is to say that you can click on Joe’s age to indicate that he does not meet the policy’s age requirement. Et voila!
Change representation at any time
Having to decide what representation is the most appropriate can be paralyzing. Often, requirements are bound to evolve over time. As a result, what seemed like a good candidate for a decision table may end up filled with scattered data. Assuming your tool locks you into a single representation after that initial choice, the cost of a bad choice becomes unbearable. A dynamic representation removes that dilemma. Just get started, and switch representation if and when you need a different way to interact. The nature of the rules may very well dictate a natural representation. For the convenience of editing, it may make more sense to change that view temporarily. As a compliance officer, you may prefer to look at a graph that documents all the paths that lead to an adverse action.
As a business analyst, I bet you have to deal with quite a few spreadsheets. Once, I met a lady that managed around 10,000 spreadsheets for an insurance company! While many spreadsheets contain requirements that need to turn into business rules, many do not. In this lady’s case, a great number of these spreadsheets contained rating tables, which were regularly updated in that format. When it is the case, you want to keep those very large MS Excel spreadsheets as spreadsheets. We call them lookup models, for which you only supply the formula to read the data, aka which columns you need to match and how, and which columns you need to return. In the end, when you need efficient lookups, you need efficient lookups. Not every requirement needs to turn into a rule.
Business users may need business apps, not business rules authoring
As intuitive as a decision management tool can be, it may never meet the needs of a true business person. The bells and whistles that business analysts need, may be overwhelming for the financier or underwriter that needs access to the decision logic. In short, that person needs the decision logic abstracted into a business app. As the business analyst, you have full control of the capabilities you expose. Should you expose some thresholds? Or expose all rejection criteria? In some cases, you may not need to expose any of the decision logic, but rather give him or her the ability to run a simulation, and the authority to promote the decision to Production. Once again, business rules are an important part of the decision management project, but not everything has to do with authoring ‘just business rules’.
I will actually present a webinar in January, and would love to have you join me.
Rating engine, pricing engine, compensation calculation, fee calculation, claims calculation, and many more types of projects aim at calculating a fee or cost. They typically share some complexity on the fine prints.
Implementing a rating engine has relied on different technologies over the years. The debate continues: can business rules be used for a rating engine? My answer is yes and no. I strongly believe that we need the full power of a decision management system. Let me explain.
When the rating engine logic is pretty straight-forward, business rules will do a good job. In the pay-as-you-go demo, we could summarize the pricing strategy in a few business rules. In real-life projects, compensation calculations and benefit calculations have also been implemented in a modest number of rules. As a rule of thumb (no pun intended), I would say that if you can express your rating logic as a formula, with or without a few exceptions here and there, you are in a good shape using business rules only.
Business rule engine is not enough
The limitation, though, is when these rate tables grow significantly. Some of these spreadsheets include tens of thousands of rows! This combinatorial explosion makes sense when you think of having rates specific to:
- 50 states,
- maybe up to 42,000 zip codes,
- 2 or 3 genders (some states can’t legally discriminate over gender, so you may deal with a gender-neutral pricing in addition to male and female)
- several risk score bins,
- many, many, many age options…
Writing all of these rules by hand may be a significant task. Rule import can help, of course, but keep in mind that it is likely a one-time solution. Updating and maintaining these rates over time will become a painful task, less painful than coding of course, but still painful. When actuaries come up with an updated rate table, finding the right line items could be a tricky exercise. My experience tells me that rare business experts provide a color-coded spreadsheet, highlighting only the changes. You certainly do not want to be in the business of comparing line by line the old and new rates.
However, the great advantage of business rules is their extreme flexibility. Assuming that the rating tables per se are taken care of, which I will cover on my next point, overrides for specific states, or product options, remains trivial by complementing the rating engine with business rules. Often, I see projects in which these overrides happen while or after retrieving rates. There is literally no end to the flexibility you can implement as business rules once rates are retrieved.
My first takeaway: I do like using business rules for fine-tuning rates, dealing with exceptions.
Rating Engines are mostly lookups
For rates that are dependent on this (sometimes) enormous number of lines to pick from, I prefer using lookup models. Lookup models are spreadsheets that can be used to return the matching columns. Let’s say you import a spreadsheet with state, age, gender, and score range. The lookup model will retrieve the rate that matches all 4 columns. You are not limited to one column to return though. In addition to the rate, the spreadsheet could return volume discount or any additional information.
The main advantage of using lookup models, in my opinion, is that there is no manual translation from spreadsheet to executable lookup model. The implementation effort is in the interface. Do you pass the actual gender for example, or do you translate its value to ‘neutral’ for those states that prevent you from using gender? Do you return a single rate, or is it possible to return multiple rates?
While rate tables can be as large as your business experts might dream, lookup models offer a great advantage in performance. They are indexed, and can return rates very quickly, somewhat independently of size.
My second takeaway: I do like using lookup models for retrieving rates.
Addressing evolving rates
As I mentioned before, the majority of the pain is typically in the maintenance of the rate tables in the rating engine. Using lookup models, this pain disappears. Updating rates is as easy as uploading a spreadsheet to the system. Overall, the 2 key aspects that I love about these mechanics deal with time-travel, seen from 2 different perspectives.
(1) The convenience of version tracking
Although rates change over time, you may need to resurrect these past rates. Many reasons come to mind. You might need to justify what your rates were at that time. Or, in urgency, you might need to backtrack bad rates that made their way into production.
Thanks to the underlying versioning system, and release management, these past rates are just a click away. Time-travel to the January 2020 release of your rating engine to resurrect pre-pandemic rates, or just to run simulations. You can also use the version history to promote that specific version of the spreadsheet to become current again. You will never lose any iteration of that spreadsheet that went into production. Peace of mind!
(2) The flexibility of time-sensitive rates
The introduction of new rates into your rating table is likely to follow a specific schedule. It happens that rates are just adjusted directly. But, more likely, they will start on a specific date, like January 1st. Rather than scheduling a job that will update the rates at midnight, it makes more sense to upload these rates upfront, and specify their effective dates.
Like business rules, rate tables can follow effective dates that apply to the entire sheet. Using the full power of business rules, you can activate these 2021 rates on January 1st for a group of states, and at a later date for another group of states.
I particularly appreciate that you have full control over the clock. Do you switch on January 1st at midnight at your headquarters? Should you adjust for the local time zone in each state? Do you consider invocation time? Or do you apply the proper rates based on delivery date? Once again, the sky the limit. I have yet to see a scenario that could not be implemented using a decision management system.
My third takeaway: I do like using lookup models for managing rates over time.
In conclusion, I highly recommend you consider a decision management system when implementing a rating engine. Business rules may have been limited in the past. However, decision management systems can certainly take you there now!
Per definition, Dynamic Questionnaires generate user interfaces that collect data. Their main characteristics include the following:
- Primarily, they apply reflexive logic for dependencies between questions
- Additionally, they enforce validation rules to ensure quality of data input
- Finally, they collaborate seamlessly with back-end decision logic
As one expects, dynamic questionnaires have been leveraged primarily for business applications. However, we also leverage them quite often in other use cases that may surprise you.
#1 Dynamic Questionnaires for Business Applications
In the Dynamic Loan Evaluation demo, we presented an example of business application that uses dynamic questionnaires. In no time, a business form turns into an online questionnaire that collects the borrower’s information, and renders the underwriting decision.
While this academic example is limited in complexity, we have seen real-life questionnaires for finance, insurance, and healthcare, that contain hundreds of questions.
Let’s say that you report past violations, we may ask, for each violation, if that was a ticket or an accident. If an accident, we may ask if it was at fault, and if there were fatalities. These nested questions, also called reflexive logic, justify the use of dynamic questionnaire technology.
#2 Dynamic Questionnaires for Implementing / Testing Multi-Step Processes
In order to navigate from page to page, a sort of business process engine handles these transitions. This engine applies decision services when appropriate. All-in-all, it provides all that long-running transactions need. When questionnaires are pretty long, applicants may get interrupted mid-way through the paperwork. You certainly do not want to force your users to start from the top then. Whatever they have filled in already can be retrieved, and the applicant can continue filling the form at any point in time.
From time to time, we encounter a scenario in which a separate business process handles the overall orchestration. While there is no issue at all with integrating a separate business process, that business process is not always ready from the get-go. When the business process is not yet available, or when it cannot be easily tested, I use dynamic questionnaires for testing. That gives me a convenient environment to navigate the series of steps needed, and select the profile for the data that needs to be pulled.
#3 Dynamic Questionnaires for Generating Business Rules
Last but not least, I use dynamic questionnaires for capturing business rules. Most decision projects author and maintain business rules in our SMARTS environment. Yet, we also encounter another scenario in which multiple rules configurations follow the exact same structure, activating or deactivating rules, and possibly setting up values.
In this example, I have several insurance products available. For each product, I can configure eligibility rules using point-and-click in the questionnaire. As product owner, the questionnaire constrains my options to only valid options. Note that I could also leverage business rules on the back-end to ensure that my configuration is valid!
We can design the questionnaire to reflect product specification sheets that you may use as a PDF for example.
I hope that these use cases will give you some ideas on how to leverage dynamic questionnaires.
With the world on a partial lockdown due to COVID 19, we had to be creative. DecisionCAMP 2020 takes place virtually this year, through Zoom presentations and Slack interactions. The show invited me to present ahead of the event.
Watch my DecisionCAMP 2020 presentation now
I decided to tackle one of the most common rules designs. Though I hope that you will implement it in SMARTS, it is technology-agnostic. As such, you could adopt it regardless of the decision management system that you use.
A decision management system obviously makes decisions. These decisions can boil down to a yes or no answer. In many circumstances, the decision includes several sub-components, in addition to that primary decision. For this design pattern, however, I only focus on the primary decision. Note that you could use the same design applied to any sub-decision as well. This is a limitation of the presentation, not one of the design.
In an underwriting system, for example, the final approval derives from many different data-points. The system looks at self-disclosures regarding the driver(s) and the vehicle(s), but also third-party data sources like DMV reports. If the rules make that decision as they go through the available data, there is a risk of an inadvertent decision override. Hence the need for a design pattern that collects these point decisions, or intermediate decisions, and makes the final decision in the end. In this presentation, I illustrate how do it in a simple and effective manner.
In our last two blog posts in this series we discussed decision engine performance and how performance is impacted by deployment architecture choices. In addition to those considerations, you should also focus on business decision performance, the topic of this post.
Central to SMARTS’s approach to decision design is the idea that you need to have a strong focus on the expected business performance of your decision. The business performance of the decision is measured by multiple KPIs defined by the different business stake holders and characterize how the decision is contributing to the business.
Decision Analytics for Simulations
SMARTS provides you with fully integrated decision analytics, including aggregates, reports and dashboards that you can configure to track those KPIs. As you are implementing and optimizing the decision logic, you can run simulations to assess the impact your change have on the decision, and take appropriate action.This allows you to ensure that the business performance of your decision is actually what you want before you deploy it in production.
Real-time Decision Metrics
SMARTS also provides you streaming decision analytics, allowing you to monitor the same KPIs on the live decisions as they are deployed, and to specify alerts that trigger if those KPIs deviate from limits you can set.This gives you the peace of mind that you are always kept up to date on how well the deployed decision is behaving and that you can take early action to update it should the situation need it.
There are also cases where it is not possible to necessarily know in advance the impact of a change. You may be exploring with new decision options you had never attempted before. SMARTS allows you to deploy your decision in an experimental mode – where part of your invocations will be routed to the new “experimental” version, and the rest to the proven one, and where you will be monitoring the relative performance to identify whether your “experimental” version is doing better than the proven one. In many financial services areas, this is called Champion-Challenger, in marketing or design, this is called A/B testing. With this approach, you can gradually and safely introduce decision optimizations that lead to better and better business performance.
In summary, when considering performance of decision management systems it is critical to consider the topic from a business perspective as well as a technical perspective. We hope this series has helped clarify performance related issues pertaining to decision management.