In this post, we present how to deal with the problem of noise, which is both a source of errors and biases in digital decision-making in organizations, through explicit decision rules, dashboards, and analytics. To illustrate our point, we use the example of the Sparkling Logic SMARTS decision management platform.
Noise in organizations’ decisioning and what to do about itIn an interview with McKinsey, Olivier Sibony, one of the renowned experts in decisioning, recommends algorithms, rules, or artificial intelligence to solve the problem of noise, a generator of errors and biases in decisioning in organizations. This recommendation resonates with our vision of automating decisioning — not all of the decisioning but the operational decisions that organizations make by thousands and sometimes millions per day. Think credit origination, claim processing, fraud detection, emergency routing, and so on.
In our vision, one of the best ways to reduce noise, and therefore errors and biases, is to make decisions explicit (like the rules of laws) so that those who define the decisions can test them out, one at a time or in groups, and visualize. The consequences of these choices on the organization before putting them into production. In particular, decisions should be kept separate from the rest of the system calling those decisions — the CRM, the loan origination system, the credit risk management platform, etc.
Noise reduction with explicit decision rules, dashboards, and analyticsOur SMARTS decisioning platform helps organizations make their operational decisions explicit, so that they can be tested and simulated before implementation, reducing biases that could be a failure to comply with industry regulations, a deviation from organizational policies, or a source of an applicant disqualification. The consequences of biases could be high in terms of image or fees, and even tremendous for certain sensitive industries such as financial, insurance, and healthcare services.
In SMARTS, business users (credit analysts, underwriters, call center professionals, fraud specialists, product marketers, etc.) express decisions in the form of business rules, decision trees, decision tables, decision flows, lookup models, and other intuitive representations that make decisioning self-explainable so that they can test decisions individually as well as collectively. So, at any time, they can check potential noise, errors, and biases before they translate into harmful consequences for the organization.
In addition to making development of decisioning explicit, SMARTS also comes with built-in dashboards to assess alternative decision strategies and measure the quality of performance at all stages of the lifecycle of decisions. By design, SMARTS focuses the decision automation effort on tangible objectives, measured by Key Performance Indicators (KPIs). Users define multiple KPIs through graphical interactions and simple, yet powerful formulas. As they capture decision logic, simply dragging and dropping any attribute into the dashboard pane automatically creates reports. Moreover, they can customize these distributions, aggregations, and/or rule metrics, as well as the charts to view the results in the dashboard.
During the testing phase, the users have access to SMARTS’ built-in map-reduce-based simulation capability to measure these metrics against large samples of data and transactions. Doing so, they can estimate the KPIs for impact analysis before the actual deployment. And all of this testing work does not require IT to code these metrics, because they are transparently translated by SMARTS.
And once the decisioning application is deployed, the users have access to SMARTS’ real-time decision analytics, a kind of cockpit for them to monitor the application, make the necessary changes, without stopping the decisioning application. SMARTS platform automatically displays KPI metrics over time or in a time window. The platform also generates notifications and alerts when some of the thresholds users have defined are crossed or certain patterns are detected. Notifications and alerts can be pushed by email, SMS, or generate a ticket in the organization’s incident management system.
Rather than being a blackbox, SMARTS makes decisioning explicit so that the users who developed it can easily explain it to those who will operate it. Moreover, the latter can adjust the decision making so that biases can be quickly detected and corrected, without putting the organization at risk for violating legal constraints, eligibility criteria, or consumer rights.
So, if you are planning to build a noise-free, error-free, and bias-free decisioning application, SMARTS can help. The Sparkling Logic team enjoys nothing more than helping customers implement their most demanding business requirements and technical specifications. Our obsession is not only to have them satisfied, but also proud of the system they build. We helped companies to build flaw-proof, data-tested, and scalable applications for loan origination, claims processing, credit risk assessment, or even fraud detection and response. So dare to give us a challenge, and we will solve it for you in days, not weeks, or months. Just email us or request a free trial.
AboutSparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful digital decisioning platform, accessible to business analysts and ‘citizen developers’. Sparkling Logic’s customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.
Sparkling Logic SMARTSTM (SMARTS for short) is a cloud-based, low-code, AI-powered business decision management platform that unifies authoring, testing, deployment and maintenance of operational decisions. SMARTS combines the highly scalable Rete-NT inference engine, with predictive analytics and machine learning models, and low-code functionality to create intelligent decisioning systems.
Tags: business rules • decision automation • decision management • decisioning • DMN • RPA • rule authoring • SMARTS
Today, predictive analytics is common in any data-driven business. Typically, data scientists create predictive models first, and IT staff deploy these models in a production environment. At Sparkling Logic, not only have we streamlined this process, but we’ve extended it with prescriptive decisions. Sparkling Logic SMARTS AI & ModelOps is the third built-in capability of SMARTS to cover the full spectrum of operational predictive models, from importing models and creating new ones to initiating learning tasks. But let’s start with a brief overview of the stages data has gone through.
Data: a resource, an asset, a business
Until recently, data was a resource to conduct business and as such, it was typically managed by the CIO’s organization. The organization’s mission was to build the overall data architecture, to choose a database vendor, and to design all the applications necessary to process the data from the databases to the business and functional people screens. These applications were mostly reporting, letting the business get a sense for how the business has been doing based on the collected data.
Then came the first transformation, where data went from an asset used to understand how the business has been doing to being an asset leveraged to predict how the business could potentially do in the future. Reporting was enhanced by predictive analytics. The scope of the analytics was not only what had happened, but also what was happening and what could happen. In general, these two past-focused and future-focused activities cover most of what data science is in business, with some really important use cases on marketing, sales, and customer relationship management.
However, a new transformation is underway, first in the banking, insurance, and health sectors, but will certainly penetrate other sectors. It consists of transforming analytics into automated decisions, translating predictions into prescriptions. The goal of this transformation is to create a virtuous cycle where not only data is analyzed, but this analysis is transformed into decisions and actions that generate new data, and so on. Reporting and predictive analytics are now completed by prescriptive analytics.
Anticipating this trend, the founders of Sparkling Logic designed the SMARTS decision management platform to implement this cycle of data, insights, and decisions. Sparkling Logic SMARTS comes with a built-in AI & ModelOps environment that covers the full spectrum of operationalizing predictive models, from importing models built by data scientists, to creating new ones without prior knowledge of machine learning, to launching and managing learning jobs.
Sparkling Logic SMARTS AI & ModelOps
Predictive model importBusiness analysts can import AI, machine learning, and deep learning models developed by data scientists, and leverage them in the decision logic. The models could be developed in Python, SPSS, SAS, or Project R among others. SMARTS integrates them as long as they are compliant to PMML, a standard for sharing and deploying predictive models, or are accessible as services.
SMARTS supports importing as PMML neural networks, multinomial, general, and linear/log regression, trees, support vector machines, naïve bayes, clustering, ruleset, scorecard, K-Nearest Neighbors (KNN), random forest, and other machine learning models.
In cases where models exist but are only available as specification, business analysts can easily import these models and seamlessly transform them into business rules for full transparency and easy inspection.
There may be situations where it is necessary to call an external service that is available elsewhere. This external service can be a predictive model or a data source. SMARTS provides support for remote functions, which makes it possible to invoke an external service through JSON-RPC or REST services.
BluePen predictive technologyWhen time is of the essence, when models are short-lived or when expertise needs to be confronted with knowledge captured in the data, business experts can use the BluePen learning technology to quickly create a model, potentially leveraging existing models.
BluePen lets business analysts and business experts explore and analyze data using domain knowledge and expertise to identify predictors, or, alternatively, selects the predictors for them. Then, using the selected predictors, BluePen helps them to generate a model in the form of readable decision rules, tables, or trees, and integrate them into their decision logic.
Using BluePen, users can build meaningful predictive models in hours or days, rather than the months it often takes. Users can also engineer or modify the models. As a result, without heavy investment in data analytics efforts, these models can be tested, leveraged in simulations, and quickly deployed in the context of an operational business decision.
Regardless of the business analyst’s choice, he or she can operationalize a wide range of models within SMARTS. Being able to integrate models into decision logic is a central ability to test and measure the performance of the end-to-end decisioning.
Moreover, SMARTS allows the analyst to translate the insights from many different models into a decision. Typically, data-centric organizations will have many different models which each can contribute insights into what the decision should be. The orchestration of how these insights are combined is expressed in decision logic, turning multiple discrete predictions into actual prescriptive decisions.
The benefits of combining machine learning and automated decisioning as SMARTS does are nothing less than transforming businesses into always learning organizations where data helps identify opportunities, machine learning turns that data into insights and automated decisioning turns this information into action, closing the virtuous cycle that data promises.
- Data has moved from being a resource to assess how the business has been doing, to being an asset used to predict the future of the business, and finally to an asset used to improve automated decisions
- Sparkling Logic SMARTS comes with a built-in AI & ModelOps environment that covers the full spectrum of operationalizing predictive models, from importing models, to creating new ones, to launching and managing learning jobs
- With its AI & ModelOps capability, SMARTS helps in transforming businesses into learning organizations, closing the virtuous cycle that data promises. Data feeds analytics leading to improved decisions that generates additional data in addition to profits
Sparkling Logic is a decision management company founded in the San Francisco Bay Area to accelerate how companies leverage data, machine learning, and business rules to automate and improve the quality of enterprise-level decisions.
Carole-Ann is Co-Founder, Chief Product Officer at Sparkling Logic. You can reach her at email@example.com.
In this post, I briefly introduce SMARTS Real-Time Decision Analytics capability to manage the quality and performance of operational decisions from development, to testing, to production.
Decision performanceH. James Harrington, one of the pioneers of decision performance measurement, once said, “Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” This statement is also true for decision performance.
Measuring decision performance is essential in any industry where a small improvement in a single decision can make a big difference, especially in risk-driven industries such as banking, insurance, and healthcare. Improving decisions in these sectors means continuously adjusting policies, rules, prices, etc. to keep them consistent with business strategy and compliant with regulations.
Decision performance management in SMARTSSMARTS helps organizations make their operational decisions explicit, so that they can be tested and simulated before implementation — thereby reducing errors and bias. To this end, we added a real-time decision analytics capability to the core decision management platform.
Currently used in financial and insurance services, it helps both business analysts and business users to define dashboards, assess alternative decision strategies, and measure the quality of performance at all stages of the lifecycle of decision management — all with the same interface without switching from one tool to another.
Development. From the start, SMARTS focuses the decision automation effort on tangible business objectives, measured by Key Performance Indicators (KPIs). Analysts and users can define multiple KPIs through graphical interactions and simple, yet powerful formulas. As they capture their decision logic, simply dragging and dropping any attribute into the dashboard pane automatically creates reports. They can customize these distributions, aggregations, and/or rule metrics, as well as the charts to view the results in the dashboard.
Testing and validation. During the testing phase, analysts and users have access to SMARTS’ built-in map-reduce-based simulation environment to measure these metrics against large samples of data. Doing so, they can estimate the KPIs for impact analysis before the actual deployment. And all of this testing work does not require IT to code these metrics, because they are transparently translated by SMARTS.
Execution. By defining a time window for these metrics, business analysts can deploy them seamlessly against production traffic. Real-time decision analytics charts display the measurements and trigger notifications and alerts when certain thresholds are crossed or certain patterns are detected. Notifications can be pushed by email, or generate a ticket in a corporate management system. Also, real-time monitoring allows organizations to react quickly when conditions suddenly change. For example, under-performing strategies can be eliminated and replaced when running a Champion/Challenger experiment.
Uses casesInsurance underwriting. Using insurance underwriting as an example, a risk analyst can look at the applicants that were approved by the rules in production and compare them to the applicants that would be approved using the rules under development. Analyzing the differences between the two sets of results drive the discovery of which rules are missing or need to be adjusted to produce better results or mitigate certain risks.
For example, he or she might discover that 25% of the differences in approval status are due to differences in risk level. This insight leads the risk analyst to focus on adding and/or modifying your risk related rules. Repeating this analyze-improve cycle reduces the time to consider and test different rules until he or she gets the best tradeoff between results and risks.
Fraud detection. An other example from a real customer case is flash fraud where decisions had to be changed and new ones rolled out in real time. In this case, the real-time decision analysis capability of SMARTS was essential so that the customer could spot deviation trends from normal situation directly in the dashboard and overcome the flood in the same user interface, all in real time.
Without this built-in capability, the time lag between the identification of fraud and the implementation of corrective actions would have been long, resulting in significant financial losses. In fact, with SMARTS Real-Time Decision Analytics, the fraud management for this client has gone from 15 days to 1 day.
Marketing campaign. The two above examples are taken from financial services but SMARTS real-time decision analytics helps in any context where decision performance could be immediately affected by a change in data, models, or business rules, such as in loan origination, product pricing, or marketing promotion.
In the latter case, SMARTS can help optimize promotion in real-time. Let’s say you construct a series of rules for a marketing couponing using SMARTS Champion/Challenger capability. Based on rules you determine, certain customers will get a discount. Some get 15% off (the current offering — the champion), while others get 20% (a test offering — the challenger). And you wonder if the extra 5% discount leads to more coupons used and more sales generated. With SMARTS real-time decision analytics environment, you find out the answer as the day progresses. By testing alternatives, you converge to the best coupon strategy with real data and on the fly.
ConclusionAs part of the decision lifecycle, business analysts obviously start by authoring their decision logic. As they progress, testing rapidly comes to the forefront. To this end, SMARTS integrates predictive data analytics with real-time decision analytics, enabling business analysts and business users to define dashboards and seamlessly associate metrics with the execution environment — using the same tool, the same interface, and just point and click.
- SMARTS comes with built-in decision analytics — no additional or third-party tool is required
- You can define metrics on decision results so you can measure and understand how each decision contributes to your organization’s business objectives
- Decision metrics enable you to assess alternative decision strategies to see which should be kept and which rejected
- SMARTS add-on for real-time decision analytics lets you monitor the decisions being made and make adjustments on the fly
- SMARTS’ real-time decision analytics helps in any context where decision performance could be immediately affected by a change in data, models, or business rules
AboutSparkling Logic is a decision management company founded in the San Francisco Bay Area to accelerate how companies leverage data, machine learning, and business rules to automate and improve the quality of enterprise-level decisions.
Sparkling Logic SMARTS is an end-to-end, low-code no-code decision management platform that spans the entire business decision lifecycle, from data import to decision modeling to application production.
Carlos Serrano is Co-Founder, Chief Technology Officer at Sparkling Logic. You can reach him at firstname.lastname@example.org.
Our Best Practices Series has focused, so far, on authoring and lifecycle management aspects of managing decisions. This post will start introducing what you should consider when promoting your decision applications to Production.
Make sure you always use release management for your decision
Carole-Ann has already covered why you should always package your decisions in releases when you have reached important milestones in the lifecycle of your decisions: see Best practices: Use Release Management. This is so important that I will repeat her key points here stressing its importance in the production phase.
You want to be 100% certain that you have in production is exactly what you tested, and that it will not change by side effect. This happens more frequently than you would think: a user may decide to test variations of the decision logic in what she or he thinks is a sandbox and that may in fact be the production environment.
You also want to have complete traceability, and at any point in time, total visibility on what the state of the decision logic was for any decision rendered you may need to review.
Everything they contributes to the decision logic should be part of the release: flows, rules, predictive and lookup models, etc. If your decision logic also includes assets the decision management system does not manage, you open the door to potential execution and traceability issues. We, of course, recommend managing your decision logic fully within the decision management system.
Only use Decision Management Systems that allow you to manage releases, and always deploy decisions that are part of a release.
Make sure the decision application fits your technical environments and requirements
Now that you have the decision you will use in production in the form of a release, you still have a number of considerations to take into account.
It must fit into the overall architecture
Typically, you will encounter one or more of the following situations
• The decision application is provided as a SaaS and invoked through REST or similar protocols (loose coupling)
• The environment is message or event driven (loose coupling)
• It relies mostly on micro-services, using an orchestration tool and a loose coupling invocation mechanism.
• It requires tight coupling between one (or more) application components at the programmatic API level
Your decision application will need to simply fit within these architectural choices with a very low architectural impact.
One additional thing to be careful about is that organizations and applications evolve. We’ve seen many customers deploy the same decision application in multiple such environments, typically interactive and batch. You need to be able to do multi-environment deployments a low cost.
It must account for availability and scalability requirements
In a loosely coupled environments, your decision application service or micro-service with need to cope with your high availability and scalability requirements. In general, this means configuring micro-services in such a way that:
• There is no single point of failure
○ replicate your repositories
○ have more than one instance available for invocation transparently
• Scaling up and down is easy
Ideally, the Decision Management System product you use has support for this directly out of the box.
It must account for security requirements
Your decision application may need to be protected. This includes
• protection against unwanted access of the decision application in production (MIM attacks, etc.)
• protection against unwanted access to the artifacts used by the decision application in production (typically repository access)
Make sure the decision applications are deployed the most appropriate way given the technical environment and the corresponding requirements. Ideally you have strong support from your Decision Management System for achieving this.
Leverage the invocation mechanisms that make sense for your use case
You will need to figure out how your code invokes the decision application once in production. Typically, you may invoke the decision application
• separately for each “transaction” (interactive)
• for a group of “transactions” (batch)
• for stream of “transactions” (streaming or batch)
Choosing the right invocation mechanism for your case can have a significant impact on the performance of your decision application.
Manage the update of your decision application in production according to the requirements of the business
One key value of Decision Management Systems is that with them business analysts can implement, test and optimize the decision logic directly.
Ideally, this expands into the deployment of decision updates to the production. As the business analysts have updated, tested and optimized the decision, they will frequently request that it be deployed “immediately”.
Traditional products require going through IT phases, code conversion, code generation and uploads. With them, you deal with delays and the potential for new problems. Modern systems such as SMARTS do provide support for this kind of deployment.
There are some key aspects to take into account when dealing with old and new versions of the decision logic:
• updating should be a one-click atomic operation, and a one-API call atomic operation
• updating should be safe (if the newer one fails to work satisfactorily, it should not enter production or should be easily rolled back)
• the system should allow you to run old and new versions of the decision concurrently
In all cases, this remains an area where you want to strike the right balance between the business requirements and the IT constraints.
For example, it is possible that all changes are batched in one deployment a day because they are coordinated with other IT-centric system changes.
Make sure that you can update the decisions in Production in the most diligent way to satisfy the business requirement.
Track the business performance of your decision in production
Once you have your process to put decisions in the form of releases in production following the guidelines above, you still need to monitor its business performance.
Products like SMARTS let you characterize, analyze and optimize the business performance of the decision before it is put in production. It will important that you continue with the same analysis once the decision is in production. Conditions may change. Your decisions, while effective when they were first deployed, may no longer be as effective after the changes. By tracking the business performances of the decisions in production you can identify this situation early, analyze the reasons and adjust the decision.
In a later installment on this series, we’ll tackle how to approach the issue of decision execution performance as opposed to decision business performance.
Risk Management techniques and enterprise tools have been around for some time, mostly in insurance, finance and banking. With the growth of industrial IoT and connected devices, OEMs and industrial customers have more opportunity to apply similar techniques to industrial equipment failure and maintenance problems. As some of our customers like ABT have shown, such new problems require state of the art Prescriptive Analytics tools to reduce failure risk and optimize maintenance costs in any industrial setting.
There are three obvious reasons the latest analytics tools can improve IoT failure risk management:
1. New, more granular IoT data may be difficult to correlate with failures –
Unlike the financial transactions where human behavior and fraud have been tracked for some time, machine data from newly connected, sub-components and related equipment failures are relatively new. Since there may be limited historical information correlating newly archived data with documented failures, it is essential to augment predictive analytics (machine learning) tools with traditional human experience (business rules).
2. Correlating multiple IoT components with a failure is difficult –As the sensors and components become more prevalent, it may be difficult to correlate a particular component behavior to a failure. For example, in a commercial distillery, the increased temperature of a distillate and related loss of alcohol efficiency on a hybrid still may be caused by either reduced coolant flow, blockage of a redistillation plate or a steam valve failure. By tracking sensors on each component and correlating them with human operator experience, a distilling plant can predict a more appropriate cleaning and maintenance of a particular distilling component. In other works, collecting data from multiple industrial IoT components and blending it with experience-based learning will significantly improve predicting likelihood of failure or a need for unscheduled maintenance to maintain equipment efficiency.
3. Learning improves maintenance insight, reduces costs –
Today, most OEMs have periodic scheduled maintenance whether needed or not. Frequently, such maintenance does not account for higher risk of failure due to a component problem. As a result, industrial customers experience both unnecessary maintenance and unpredictable failures, both which increase costs and prolong costly downtimes. For example, one of our customers, a major power distributor in Western Australia, combined predictive analytics with decision logic to identify power grid components more likely to fail soon.
Modern decision management platforms like Sparkling Logic SMARTS allow improving the ultimate Risk Management problem. They allow evolution of intelligent industrial machinery that learns and suggests failure before and outside scheduled maintenance intervals. I predict that using such tools, progressive OEMs and industrial customers will move to variable maintenance schedules and predict the majority of failures BEFORE they happen.
In summary, industrial customers and industrial equipment OEMs need modern tools to manage connected IoT components and equipment and ultimately implement advanced industrial IoT risk management. These techniques will result in higher uptime, lower maintenance costs and higher productivity. Such modern prescriptive analytics tools provide two key areas of expertise:
- Predictive Analytics –
- Decision Management / Rules Engines –
to quickly analyze IoT device data, visualize, predict and learn the patterns of failure and suggest best course of action or improved maintenance schedules.
to implement predictive discoveries in an easy, graphical fashion as well as to test multiple failure scenarios and instantly deploy the industrial failure risk logic. Deploying and automating improved failure risk decisions will allow even less skilled operators to manage even most complex industrial systems with great efficiency.
Learn more about how SMARTS Decision Manager can help improve your IoT failure risk.
Decision Management has been a discipline for Business Analysts for decades now. Data Scientists have been historically avid users of Analytic Workbenches. The divide between these two crowds has been crossed by sending predictive model specifications across, from the latter group to the former. These specifications could be in the form of paper, stating the formula to implement, or in electronic format that could be seamlessly imported. This is why PMML (Predictive Model Markup Language) has proven to be a useful standard in our industry.
The fact is that the divide that was artificially created between these two groups is not as deep as we originally thought. There have been reasons to cross the divide, and both groups have seen significant benefits in doing so.
In this post, I will highlight a couple of use cases that illustrate my point.
A number of organizations have adopted the idea of making use of the Decision Management approach and technologies to problems such as risk, fraud, eligibility, maximizing and more. If you read this blog, you probably already know what Decision Management brings to the table.
Decision Management is all about automating repeatable decisions in a maintainable way so that they can be optimized in a continuous fashion.
Decision systems can use Business Rules Management Systems (BRMS), but they do not need to restrict themselves to just that: they can also be built on Predictive Analytics technology; or they can even consist of a combination of both. The increasing availability of data that can be used to test, optimize decisions, or extract insights from, makes it possible for decision-centric applications to combine expertise and data to levels not seen in previous generations of applications.
In this post, we’ll outline the evolution from pure Business Rules Systems to Prescriptive Analytics platforms for decision-centric applications.Read More »