Call to Action

Webinar: Take a tour of Sparkling Logic's SMARTS Decision Manager Register Now

Uncategorized

The SMARTS way for micro-calculations


The SMARTS way for micro-calculations

Until recently, companies used rules to comply with a sectoral regulation, implement a business strategy, or automate a business process. At Sparkling Logic, we were among the few pioneers who helped customers to use business rules for micro-calculations.

Micro-calculations

By micro-calculations (analogy to Michael Ross and James Taylor’s “micro-decisions”), we mean all those simple but plentiful granular calculations that businesses often codify into large spreadsheets and use to score, rate, or price items. Think about it when you click on your smartphone on a digital bank’s app to apply for a loan, or when you go to an auto insurer’s website to find a price for your new car. To you, things seem simple, but behind the scenes there is complexity…

Indeed, as for any complex system, it is the interaction between its elements, however simple, that gives rise to its complexity. The price of insurance is not a simple numerical formula but a combination of rules and calculations, all based on data. Data is either provided by you or is internal to the insurance company. A slight change in a rule or a calculation may seem inconsequential at the micro-level but can result in incredibly significant side-effects at the macro-level. Imagine the consequences of a miscalculation in the pricing engine of the insurance company. A single miscalculation could translate into huge losses for the insurance company if the error were not visible in the rule that performs the first micro-calculation but propagates to other rules which use the result of this micro-calculation.

The SMARTS way

Based on a 20+ year experience in implementing or guiding customers to implement sophisticated data-based micro-calculation systems, the founders of Sparkling Logic have developed a way to manage the complexity of such systems. To reduce or even eliminate the potential errors that calculations can generate when modernizing or implementing a new system, they produced the following steps:

1) Start with existing data, representing past transactions. If they worked, they should continue working or at least guide the new system. It comes with a built-in engine that automatically turns data spreadsheets into a database which an application can query to perform its micro-calculations.

Take again the example of the auto insurance company. The spreadsheet may contain thousands of past transactions which, depending on the age of the primary driver, the mileage of the car, and other criteria, provides the price for this configuration. But often this price is the same for other configurations, and parts of the configuration at hand appear in other configurations, making direct use of the spreadsheet a complicated exercise. With SMARTS, the user or the application only queries the engine with a configuration and in record time it gives back the corresponding price.

2) Experiment with different representations to visualize the chain of micro-calculations that leads to the final score, rating, or price. SMARTS offers different graphical representations to express a calculation flow: tables, trees and graphs, and rules. Moreover, users can choose and switch between different representations without leaving the graphical user interface. And they can do so until they select the most appropriate representation based on the task at hand as well as the steps that they are familiar with when designing or reviewing the calculation flow.

3) Integrate testing when authoring rules. Decision logic is not software code. As a result, you cannot be satisfied with testing tools and techniques that software developers use. Testing decisions requires a different approach, different techniques, and different tools. Granted, ensuring that your decision service can compile is useful. But the end goal is really to ensure that your decision logic complies with your business objectives. To this end, SMARTS comes with an integrated dashboard where the user can define metrics against which to evaluate the rules —rule by rule, a set of rules, or the entire system.

4) Run A / B simulations. There is no such thing as a timeless business strategy. Economic conditions change suddenly, as does scoring, rating, or pricing. SMARTS allows users to run A / B tests (called Champion / Challenge simulations in credit and risk management) at any time, to evaluate the performance of different strategies, until they find the one that best responds to economic changes. Think of the eligibility criteria, the increase or decrease in prices, and all the parameters that go into the rules and calculations. For these, SMARTS supports big data simulations to experiment with different scenarios of a given strategy, using real data streams.

5) Monitor in real-time. Despite all the care you would put into designing or reviewing your spreadsheets, a typo or error might still not appear until a long time after you put your system into production. To help you manage these cases, SMARTS comes with a real-time decision analytics capability that displays measurements and triggers notifications and alerts when certain KPIs cross thresholds, or the application detects certain patterns. SMARTS pushes notifications by email or generates a ticket in a corporate management system.

Wrap-up

  • Decision management and business rules are also suitable for micro-calculations, the type of computations that businesses often codify into large spreadsheets and use to score, rate, or price items.
  • A minor change in a rule or a calculation may seem inconsequential at the micro-level but can result in significant side-effects at the macro-level. A single miscalculation could translate into huge losses.
  • Sparkling Logic has developed a way to manage the complexity of such systems: Start with existing data, experiment with different representations, integrate testing when authoring rules, run A / B simulations, and monitor in real-time.

If you are planning to upgrade or build a system with micro-calculations, SMARTS can help. The Sparkling Logic team has been involved in projects with scoring / rating / pricing data in the form of large spreadsheets.

Most often, these spreadsheets contained thousands and sometimes tens of thousands of rows. And customers were not just concerned with performance, but also maintainability. The pace of change was high and required additional monitoring and careful management to roll these rates over disparate geographies and time periods. So, just contact us or request a free trial.

Learn more


Implementing Rating Engines with Business Rules and Lookup Models, an online seminar where you will learn how SMARTS manages not only simple rating engines, but also complex pricing engines with a combinatorial explosion of specific cases and continuous evolution over time.

Best-in-class Series: Testing your Decisions, an online seminar to explore what makes decisions different, how to evaluate them, and how to automate regression tests in SMARTS.

About


Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful decision management platform, accessible to business analysts and ‘citizen developers.’ Sparkling Logic SMARTS customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.

Sparkling Logic SMARTSTM (SMARTS for short) is an all-in-one low-code platform for data-driven decision-making. It unifies authoring, testing, deployment, and maintenance of operational decisions. SMARTS combines the highly scalable Rete-NT inference engine, with predictive analytics and machine learning models, and low-code functionality to create intelligent decisioning systems.

Hassan Lâasri is a data strategy consultant, now leading marketing for Sparkling Logic. You can reach him at hlaasri@sparklinglogic.com.

Sparkling Logic SMARTSTM in 10 Questions and Answers


Sparkling Logic SMARTS in 10 Questions and Answers

Sparkling Logic helps businesses automate and improve the quality of their operational decisions with a technology platform that is powerful and simple: SMARTS for short. In this post, we present SMARTS through 10 selected questions and answers.

Q&A

1) What is SMARTS?

SMARTS is a decision management platform for business analysts and ‘citizen developers’ to author, test, simulate, deploy, run, and change decisions autonomously, without involving developers or IT beyond first installation.

2) Is SMARTS a business rules engine?

SMARTS is more than a business rules engine. It integrates multiple decision technologies into the same platform. SMARTS provides eight execution engines: A decision flow engine to sequence tasks of a business process; a state-machine engine to orchestrate tasks; a rule set engine to sequence decisions; a sequential engine that either fires all or just the first valid decision; a Rete-NT engine for inference; a lookup engine for data search in large datasets; a PMML engine to execute predictive models; and a DMN 1.3 engine to execute decision models. Depending on the problem you have, you may choose one or the other, or even combine them in the same set-up.

3) What are the typical applications for which SMARTS is the best fit?

In the financial, insurance, and healthcare services, SMARTS often won over the competition for origination and underwriting, pricing and rating engines, account management, fraud detection, and collections and recovery. More generally, SMARTS is a good fit when there are a lot of decisions that are data-based, frequently invoked, and likely to change often.

4) What is the difference between authoring business decisions and rules with SMARTS and coding them directly in the final application?

You can code decision logic but you will need detailed specifications from business analysts. This process may take too much time when compared to SMARTS. And once the decision logic is coded, it becomes complicated for business analysts to understand and take control of. SMARTS targets business-critical decision logic that either implements business models, corporate policies or industry directives in a dynamic and continually changing economy. Think of all the financial, insurance, and healthcare regulations since the financial crisis of 2008 and the changes since the coronavirus crisis of 2020. These two crises are typical examples of complex situations where business decisions not only need to be implemented quickly and accurately, but they also need to change dynamically and continuously.

5) Does SMARTS come with a decision design process?

SMARTS not only supports but it also augments the Decision Model and Notation (DMN) standard of the OMG (Object Management Group). DMN models decision dependencies very well, but not decision sequencing, which is also a natural way experts use to describe a complete decision logic. SMARTS addresses both dependency and sequencing through the combination of Pencil, RedPen, and the decision flow.

6) What machine learning models does SMARTS support?

SMARTS supports the execution of 13 machine learning models including classification, linear and logistic regression, support vector machines (SVMs), decision trees, random forests and ensemble learning, clustering, and neural networks. SMARTS uses PMML, the standardized predictive model markup language, to import and execute whatever model your data scientists have built.

7) Does SMARTS integrate with business process management platforms?

Yes, a SMARTS decision service can be natively invoked by a business process like any other service. Also, for decision-centric processes, SMARTS provides an orchestration capability.

8) What is the difference between an RPA tool and SMARTS?

If you think of a process as a sequence of “what to do”, “how to do it”, “do it”, and “report it”, then SMARTS automates the “what to do” and “how to do it” tasks while an RPA tool automates the “do it” and “report it” tasks.

9) Is SMARTS cloud-based?

SMARTS was designed from the ground-up for the cloud. Whether you have chosen to host your application or use our SaaS solution, we provide you with the most modern tools. SMARTS comes in a container, ready to install on your premises, AWS, GCP, Azure, or Aliyun. Choose yours, change your mind, no need to recode to redeploy your application.

10) What makes you unique?

Our motto is “your decisions, our business”. We enjoy nothing more than helping customers implement their most demanding business requirements and technical specifications. Our obsession is not only to have clients satisfied but also to be proud of the system they built. So dare to give us a challenge and we will solve it for you in days, not weeks, or months. Just email us or request a free trial.

In this post, we introduced SMARTS through 10 selected questions and answers. If you have more, feel free to read our blog, sign up for our webinars, or contact us. We would be happy to get back to you very quickly.

About

Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful decision management platform, accessible to business analysts and ‘citizen developers’. Sparkling Logic’s customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.

Sparkling Logic SMARTSTM (SMARTS for short) is a cloud-based, low-code, decision technology platform that unifies authoring, testing, deployment and maintenance of operational decisions. SMARTS combines the highly scalable Rete-NT inference engine, with predictive analytics and machine learning models, and low-code functionality to create intelligent decisioning systems.

Hassan Lâasri is a data strategy consultant, now leading marketing for Sparkling Logic. You can reach him at hlaasri@sparklinglogic.com.

Decision Requirements and Modeling with DMN in SMARTS


DMN 1.3 support in SMARTS

In this post, we present how Sparkling Logic continues its involvement in the DMN standard, through its graphical tool SMARTS Pencil, which business analysts use to model business decisions by drawing a diagram to form a decision process.

DMN, a bit of history

The Decision Model and Notation (DMN) was formally introduced by the Object Management Group (OMG) as a v1.0 specification in September 2015. Its goal was to provide a common notation understandable by all the members of a team whose goal is to model their organization’s decisions.

The notation is based on a simple set of shapes which are organized in a graph. This allows the decomposition of a top-level decision into more, simpler ones, whose results must be available before the top-level decision can be made. These additional decisions themselves would be decomposed, and so on and so forth until the model reaches a more complete state. In addition, the implementation of the decisions can be provided, notably in the form of decision tables (which is also a very common means of representing rules).

The normalization of the graphical formalism (the DMN graph) and of the way the business logic is implemented (e.g., decision tables) allows teams to talk about their decisions, using diagrams with a limited set of shapes.

Sparkling Logic was one of the early vendors to provide a tool to edit (and execute) these decision models: Pencil Decision Modeler. It was released in January 2015, before the standard was officially approved.

Since then, the DMN standard evolved significantly, by adding new diagram elements, new constructs and new language features, while clarifying some of the existing notions. It is now at version 1.3. And we didn’t rest on our laurels either: in SMARTS Ushuaia, we made Pencil Decision Modeler part of SMARTS, as a first-class feature and added full compliance to DMN 1.3! This post describes how SMARTS supports DMN 1.3.

Basics

DMN 1.3 still defines the building blocks which were in the original standard and which I mentioned in Talking about decisions.

As a recap:

  • A Decision determines its output based on one or more inputs; these inputs may be provided by an input data element, or by another decision
  • An input data is information used as input by one or more decisions, or by one or more knowledge sources
  • A business knowledge model represents knowledge which is encapsulated, and which may be used by one or more decisions, or another business knowledge model. This knowledge may be anything which DMN does not understand (such as a machine learning algorithm, a neural network, etc.) or a DMN construct (called a “boxed expression”, see below)
  • A knowledge source represents the authority for a decision, a business knowledge model, or another knowledge source: this is where the knowledge can be obtained (be it from a written transcription or from someone)

These blocks are organized in a graph and the links between them are called requirements.

What’s new in SMARTS’ DMN Support

More building blocks

In DMN 1.3, the following elements may also be added to a graph:

  • A decision service exposes one or more decisions from a decision model as a reusable element (a service) which might be consumed internally or externally
  • A group is used to group several DMN elements visually (with whatever semantics may be associated with the grouping)
  • A text annotation is a shape which contains a label and can be attached to any DMN element

Custom types and variables

Input data, decision and business knowledge model elements all have an associated variable, which is of a given type (string, number etc., or custom). A variable is a handle to access the value directly passed by an input data element, or calculated by the implementation of a decision or a business knowledge model, from within the decision implementation.

Custom types may be defined to group multiple properties under a single type name (with structure) or to allow variables which will hold multiple values (arrays).

Boxed Expressions

A few constructs are available to provide an implementation for a decision or a business knowledge models; they are termed boxed expressions since such expressions are shown in boxes which have a normalized representation. The following types of boxed expressions are available in DMN 1.3:

  • Literal expression: this is a simple expression which can use the available variables to calculate a result
  • Context: this is a set of entries, each combining a variable and a boxed expression. Each entry in the context can use the variables of the entries defined before it, which is like using “local variables” in some languages
  • Decision table: this is a tabular representation where rows (called rules) provide the value of outputs (supplied in action columns), depending on the value of inputs (supplied in condition columns)
  • Function: a function can be called using an invocation, by passing arguments to its parameters. The result of a function is the result of the execution of its body (which is an expression that can use the values of the passed parameters). A Business knowledge model can only be implemented by a function
  • Invocation: this is used to call a function by name, by passing values to the function’s parameters
  • List: this is a collection of values calculated from each of the boxed expressions in the list
  • Relation: this is a vertical list of horizontal contexts, each with the same entries

In addition to these, SMARTS defines an additional boxed expression, called the rule set. This is a set of named rules, where each rule is composed of a condition (an expression evaluating inputs) and action (an expression providing some values to outputs).

Helping Industry Adoption

With SMARTS Ushuaia, decision models are first-class citizens. The full compliance with DMN 1.3 means that all the DMN elements and boxed expressions, as well as the ability to interchange diagrams with other tools, are part of the package.

As is usual, any model can be tested and executed in the same context as your SMARTS decision –a decision is never made in isolation, and a model is never used in isolation either. And of course, you will benefit from the great tooling we provide.

Finally, we at Sparkling Logic strongly believe that decision management technologies should be put in the hands of all business analysts. This is why we are part of the DMN On-Ramp Group, whose mission is to provide a checklist to help customers find the DMN tool to suit your needs, educate and raise awareness about DMN, and help with DMN compliance. For a great presentation of the group, check out here.

About

Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful digital decisioning platform, accessible to business analysts and ‘citizen developers’. Sparkling Logic’s customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.

Sparkling Logic SMARTSTM (SMARTS for short) is a cloud-based, low-code, AI-powered business decision management platform that unifies authoring, testing, deployment and maintenance of operational decisions. SMARTS combines the highly scalable Rete-NT inference engine, with predictive analytics and machine learning models, and low-code functionality to create intelligent decisioning systems.

Marc Lerman is VP of User Experience at Sparkling Logic. You can reach him at mlerman@sparklinglogic.com.

If you envision modernizing or building a credit origination system, an insurance underwriting application, a rating engine, or a product configurator, Sparkling Logic can help. Our SMARTS digital decisioning platform automate decisions by reducing manual processing, accelerating processing time, increasing consistency, and liberating expert resources to focus on new initiatives. SMARTS also improve decisions by reducing risk and increasing profitability.

SMARTS Real-Time Decision Analytics


Real-time decision analytics
In this post, I briefly introduce SMARTS Real-Time Decision Analytics capability to manage the quality and performance of operational decisions from development, to testing, to production.

Decision performance

H. James Harrington, one of the pioneers of decision performance measurement, once said, “Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” This statement is also true for decision performance.

Measuring decision performance is essential in any industry where a small improvement in a single decision can make a big difference, especially in risk-driven industries such as banking, insurance, and healthcare. Improving decisions in these sectors means continuously adjusting policies, rules, prices, etc. to keep them consistent with business strategy and compliant with regulations.

Decision performance management in SMARTS

SMARTS helps organizations make their operational decisions explicit, so that they can be tested and simulated before implementation — thereby reducing errors and bias. To this end, we added a real-time decision analytics capability to the core decision management platform.

Currently used in financial and insurance services, it helps both business analysts and business users to define dashboards, assess alternative decision strategies, and measure the quality of performance at all stages of the lifecycle of decision management — all with the same interface without switching from one tool to another.

Development. From the start, SMARTS focuses the decision automation effort on tangible business objectives, measured by Key Performance Indicators (KPIs). Analysts and users can define multiple KPIs through graphical interactions and simple, yet powerful formulas. As they capture their decision logic, simply dragging and dropping any attribute into the dashboard pane automatically creates reports. They can customize these distributions, aggregations, and/or rule metrics, as well as the charts to view the results in the dashboard.

Testing and validation. During the testing phase, analysts and users have access to SMARTS’ built-in map-reduce-based simulation environment to measure these metrics against large samples of data. Doing so, they can estimate the KPIs for impact analysis before the actual deployment. And all of this testing work does not require IT to code these metrics, because they are transparently translated by SMARTS.

Execution. By defining a time window for these metrics, business analysts can deploy them seamlessly against production traffic. Real-time decision analytics charts display the measurements and trigger notifications and alerts when certain thresholds are crossed or certain patterns are detected. Notifications can be pushed by email, or generate a ticket in a corporate management system. Also, real-time monitoring allows organizations to react quickly when conditions suddenly change. For example, under-performing strategies can be eliminated and replaced when running a Champion/Challenger experiment.

Uses cases

Insurance underwriting. Using insurance underwriting as an example, a risk analyst can look at the applicants that were approved by the rules in production and compare them to the applicants that would be approved using the rules under development. Analyzing the differences between the two sets of results drive the discovery of which rules are missing or need to be adjusted to produce better results or mitigate certain risks.

For example, he or she might discover that 25% of the differences in approval status are due to differences in risk level. This insight leads the risk analyst to focus on adding and/or modifying your risk related rules. Repeating this analyze-improve cycle reduces the time to consider and test different rules until he or she gets the best tradeoff between results and risks.

Fraud detection. An other example from a real customer case is flash fraud where decisions had to be changed and new ones rolled out in real time. In this case, the real-time decision analysis capability of SMARTS was essential so that the customer could spot deviation trends from normal situation directly in the dashboard and overcome the flood in the same user interface, all in real time.

Without this built-in capability, the time lag between the identification of fraud and the implementation of corrective actions would have been long, resulting in significant financial losses. In fact, with SMARTS Real-Time Decision Analytics, the fraud management for this client has gone from 15 days to 1 day.

Marketing campaign. The two above examples are taken from financial services but SMARTS real-time decision analytics helps in any context where decision performance could be immediately affected by a change in data, models, or business rules, such as in loan origination, product pricing, or marketing promotion.

In the latter case, SMARTS can help optimize promotion in real-time. Let’s say you construct a series of rules for a marketing couponing using SMARTS Champion/Challenger capability. Based on rules you determine, certain customers will get a discount. Some get 15% off (the current offering — the champion), while others get 20% (a test offering — the challenger). And you wonder if the extra 5% discount leads to more coupons used and more sales generated. With SMARTS real-time decision analytics environment, you find out the answer as the day progresses. By testing alternatives, you converge to the best coupon strategy with real data and on the fly.

Conclusion

As part of the decision lifecycle, business analysts obviously start by authoring their decision logic. As they progress, testing rapidly comes to the forefront. To this end, SMARTS integrates predictive data analytics with real-time decision analytics, enabling business analysts and business users to define dashboards and seamlessly associate metrics with the execution environment — using the same tool, the same interface, and just point and click.

Takeaways

  • SMARTS comes with built-in decision analytics — no additional or third-party tool is required
  • You can define metrics on decision results so you can measure and understand how each decision contributes to your organization’s business objectives
  • Decision metrics enable you to assess alternative decision strategies to see which should be kept and which rejected
  • SMARTS add-on for real-time decision analytics lets you monitor the decisions being made and make adjustments on the fly
  • SMARTS’ real-time decision analytics helps in any context where decision performance could be immediately affected by a change in data, models, or business rules

About

Sparkling Logic is a decision management company founded in the San Francisco Bay Area to accelerate how companies leverage data, machine learning, and business rules to automate and improve the quality of enterprise-level decisions.

Sparkling Logic SMARTS is an end-to-end, low-code no-code decision management platform that spans the entire business decision lifecycle, from data import to decision modeling to application production.

Carlos Serrano is Co-Founder, Chief Technology Officer at Sparkling Logic. You can reach him at cserrano@sparklinglogic.com.

Low-Code No-Code Applied to Decision Management


DevelopPreviewShip

Low-code no-code is not a new concept to Sparkling Logic. From the beginning, the founders wanted to deliver a powerful yet simple product, so that a business analyst could start with data and build decision logic with built-in predictive data analytics and execution decision analytics.

Version after version, they have achieved this vision through SMARTS, an end-to-end decision management platform that features low-code no-code for business analysts and business users to develop and manage decision logic through point-and-click operations.

Low-code development

For business analysts, SMARTS offers a low-code development environment in which users can express decision logic through a point-and-click user interface to connect data, experiment with decisions, monitor execution without switching between different tools to get the job done. Depending on the nature of the decision logic at hand and user preferences, business analysts can choose on the fly the most appropriate representation to capture or update their decision logic. The resulting decision logic is seamlessly deployed as a decision service without IT intervention.

To push the simplification even further, Sparkling Logic founders turned to their customers for inspiration on their needs and developed three complementary technologies:

  • RedPen, a patented point-and-click technology that accelerates rule authoring without a need to know rule syntax or involve IT to author the rules
  • BluePen, another patented point-and-click technology to quickly create or leverage a data model and put it into production without involving data scientists or IT
  • A dynamic questionnaire to produce intelligent front-ends that reduces the number of unnecessary or redundant questions

No-code apps

In addition to low-code development capability for business analysts, SMARTS also elevates the decision logic to a simple web form-based interface for untrained business users. They can configure their decision strategies, test the updated decision logic, and promote the vetted changes to the next staging environment — without learning rules syntax.

These business apps offer a business abstraction for most tasks available in SMARTS related to configuration, testing and simulation, invocation and deployment management, as well as administration.

For example, credit risk specialists can configure loans, credit cards, and other banking products, and pricing specialists can control pricing tables, through a custom business app specific to their industry. The no-code business app enables business users to cope with environment changes whether they are related to internal policies, competition pressure, or industry regulation.

Furthermore, SMARTS tasks can also be automated through orchestration scripts. Business users can trigger these scripts through the click of a button, or schedule them to be performed automatically and seamlessly.

About

Sparkling Logic is a decision management company founded in the Bay Area to accelerate how companies leverage internal and external data and models to automate and improve the quality of enterprise-level decisions.

Sparkling Logic SMARTS is an end-to-end, low-code no-code decision management platform that spans the entire business decision lifecycle, from data import to decision modeling to application production.

Hassan Lâasri is a data strategy consultant, now leading marketing for Sparkling Logic. You can reach him at hlaasri@sparklinglogic.com.

Technical Series: Deployment Architecture and Performance


In our last post we discussed Decision Engine Performance and how SMARTS provides different engines that are optimized to their specific application. In this post we will cover how deployment architecture choices impact performance.

SMARTS provides you higher level support for implementing your decision management system than just a decision engine. In particular, it provides you with support for building decision services for micro-services as well as other service approaches.

Delivered in either repository-based or decision-based Docker containers or Virtual appliances, SMARTS decision services add to the decision engine the following, among other features:

Support for Secure Service Invocations (typically JSON over HTTPS)



Decisions may be invoked through services in an authenticated (access tokens) context. Many users may invoke the service concurrently, using any client technology that can interact with services. Sparking Logic provides Java, .NET standard, Python 3, NodeJS SDKs to facilitate the client implementation.


Support for Horizontal and Vertical Scalability

Decision engines executing within the decision service will leverage all cores available to them within the installation. Adding more cores results in the ability for the engine to support more concurrent executions, and the scalability is typically linear.



You may also deploy multiple microservice installations behind a load balancer. You will typically do that using an orchestration technology and leveraging the load balancer technologies available within your environment (on-premise or cloud).
SMARTS allows you to have multiple instances leveraging their own replicated repositories or leveraging an external repository, all implementing the same set of services. These instances are deployed behind a load balancer and provide you with scalability by adding more instances. You may also add on-demand instances (with replicated or delegated repositories) to cope with elastic loads. SMARTS automates all the process of keeping all those services in sync and updating them as you change the decision logic.


Support for High Availability


SMARTS also allows you to have redundancy in your decision services. Having multiple instances with replicated repositories removes single points of failure. You can have an instance taken out, the rest of the replicated instances can continue with the load
.

Support for No-downtime Hot Swap of Decision Logic with Full Traceability



SMARTS provides multiple levels at which you can swap decision logic. At the highest level, your lifecycle manager, without any IT intervention, can change the release of the decision logic being executed with one click, and no downtime. SMARTS will load the new release and hot swap it atomically if there is no problem. You can configure the strategy to take in case the new release is not loadable or has compilation problems: continue using the previous one and notify, stop providing the service, etc.
Of course, you can also hot swap your decision logic using other orchestration mechanisms, but those tend to involve IT.


Support for Ready-To-Execute Decision Logic



SMARTS allows you to specify when a decision service is declared to be ready to receive invocations. Typically, you want to make sure that is only the case when it is actually loaded, compiled and cached in memory, so that the first invocations hitting it do not pay the price of an update

In addition to providing support for all these performance related features, SMARTS does it all in a secure and auditable way. Decision services are configured to use read-only project releases, and the information of what release is used on any service invocation is returned to the invoker.

Finally, you should also focus on the business decision performance. We’ll discuss that topic in our next blog post.
.

Technical Series: Authentication and Access Control


Decision Management SystemA key benefit of using a Decision Management System is to allow the life-cycle of automated decisions to be fully managed by the enterprise.

When the decision logic remains in the application code, it becomes difficult to separate access to decision logic code from the rest. For example, reading through pages of commit comments to find the ones relevant to the decision is close to impossible. And so is ensuring that only resources with the right roles can modify the logic.
Clearly, this leads to the same situation you would be in if your business data were totally immersed in the application code. You would not do that for your business data, you should not do that for your business decision logic for exactly the same reasons.

Decision Management Systems separate the decision logic from the rest of the code. Thus, you get the immense benefit of being able to update the decision logic according to the business needs. But the real benefit comes when you combine that with authentication and access control:

  • you can control who has access to what decision logic asset, and for what purpose
  • and you can trace who did what to which asset, when and why

Of course, a lot of what is written here applies to other systems than Decision Management Systems. But this is particularly important in this case.

Roles and access control

The very first thing to consider is how to control who has access to what in the DMS. This is access control — but note that we also use authorization as an equivalent term.
In general, one thinks of access control in terms of roles ans assets. Roles characterize how a person interacts with the assets in the system.
And the challenge is that there are many roles involved in interacting with your automated decision logic. The same physical person may fill many roles, but those are different roles: they use the decision management system in different ways. In other words, these different roles have access to different operations on different sets of decision logic assets.

Base roles and access control needs

Typically, and this is of course not the only way of splitting them, you will have roles such as the following:

  • Administrator
    The administrator role administers the system but rarely is involved in anything else. In general, IT or operations resources are those with this role.

  • Decision definer
    The decision definer role is a main user role: this role is responsible for managing the requirements for the automated decision and its expected business performance. Typically, business owners and business analysts are assigned this role.

  • Decision implementer
    The decision implementer role is the other main user role: this role designs, implements, tests and optimizes decisions. Generally, business analysts, data analysts or scientists, decision owners, and sometimes business-savvy IT resources are given this role.

  • Decision tester
    The decision tester role is involved in business testing of the decisions: validating they really do fit what the business needs. Usually, business analysts, data analysts and business owners fill this role.

  • Life-cycle manager
    The life-cycle manager role is responsible for ensuring that enterprise-compliant processes are followed as the decision logic assets go from requirements to implementation to deployment and retirement.

More advanced needs

There may be many other roles, and the key is to realize that how the enterprise does business impacts what these roles may be. For example, our company has a number of enterprise customers who have two types of decision implementer roles:

  • General decision implementer: designs, implements the structure of the decision and many parts of it, tests and optimizes it
  • Restricted decision implementer: designs and implements only parts of the decision — groups of rules, or models

The details on what the second role can design and implement may vary from project to project, etc.

Many other such roles may be defined: those who can modify anything but the contract between the automated decision and the application that invokes, etc.

It gets more complicated: you may also need to account for the fact that only specific roles can manage certain specific assets. For example, you may have a decision that incorporates a rate computation table that only a few resources can see, although it is part of what the system manages and executes.

Requirements for the Decision Management System

Given all this, the expectation is that the DMS support directly, or through an integration with the enterprise systems, the following:

  • Role-based access control to the decision logic asset
  • And ability to define custom roles to fit the needs of the enterprise and how it conducts its business
  • And ability to have roles that control access to specific operations on specific decision logic assets

This can be achieved in a few ways. In general:

  • If all decision assets are in a system which is also managed by the enterprise authentication and access control system: you can directly leverage it
  • And if that is not the case: you delegate authentication and basic access control to the enterprise authentication and access control system, and manage the finer-grained access control in the DMS, tied to the external authentication

Authentication

Of course, roles are attached to a user, and in order to guarantee that the user is the right one, you will be using an authentication system. There is a vast number of such systems in the enterprise, and they play a central role in securing the assets the enterprise deals with.

Principles

The principle is that for each user that needs to have access to your enterprise systems, you will have an entry in your authentication system. Thus, the authentication system will ensure the user is who the user claims, and apply all the policies the enterprise wants to apply: two-factor authentication, challenges, password changes, etc. Furthermore, it will also control when the user has access to the systems.

This means that all systems need to make sure a central system carries out all authentications. And this includes the Decision Management System, of course. For example:

  • The DMS is only accessible through another application that does the proper authentication
  • Or it delegates the authentication to the enterprise authentication system

The second approach is more common in a services world with low coupling.

Requirements for the Decision Management System

The expectation is that the DMS will:

  • Delegate its authentication to the enterprise authentication and access control systems
  • Or use the authentication information provided by an encapsulating service

Vendors in this space have the challenge that in the enterprise world there are many authentication systems, each with potentially more than one protocol. Just in terms of protocols, enterprises use:

  • LDAP
  • WS-Federation
  • OAuth2
  • OpenID Connect
  • and more

Trace

Additionally, enterprises are interested in keeping a close trace of who does what and when in the Decision Management System. Of course, using authentication and the fact that users will always operate within the context of an authenticated session largely enables them to do so.
But this is not just a question of change log: you also want to know who has been active, who has exported and imported assets, who has generated reports, who has triggered long simulations, etc.

Furthermore, there are three types of usages for these traces:

  • Situational awareness: you want to know what has been done recently and why
  • Exception handling: you want to be alerted if a certain role or user carries out a certain operation. For example, when somebody updates a decision in production.
  • Forensics: you are looking for a particular set of operations and want to know when, who and why. For example, for compliance verification reasons.

A persisted and query-able activity stream provides support for the first type of usage. And an integration with the enterprise log management and communication management systems support the other types of usages.

Requirements for the Decision Management System

The expectation is that the DMS will:

  • Provide an activity stream users can browse through and query
  • And support an integration with the enterprise systems that log activity
  • And provide an integration with the enterprise systems that communicate alerts

There are many more details related to these authentication, access control and trace integrations. Also, one interesting trend is the move towards taking all of these into account for the beginning as the IT infrastructure moves to the models common in the cloud, even when on-premise.

This blog is part of the Technical Series, stay tuned for more!

[Image Designed by security from Flaticon]

Thinking Outside the Cube – Almaden Quicksilver


Fast decisions got executed here
Fast decisions got executed here

It has been quite a while since Carlos and I blogged about our hikes.  A quick blog on the topic is long overdue. You can’t blame us though for being more passionate about technology!

If you grew up watching lots of westerns like we did, the view of the hanging tree is likely to bring back a lot of memories.  It makes the place a little more dramatic under the hot sun of summer (it was long overdue too).  People used to throw rocks at the tree as a symbol of disgust for the despicable crimes committed.  As a Decision Management person, my mind was contemplating the fast decisions they made here and the lack of process that lead to some mistakes for sure.

Over a hundred tunnels and shafts gave access to the minerals
Over a hundred tunnels and shafts gave access to the minerals

 

Many tunnels and shafts spring up here and there.  The San Cristobal tunnel is worth stopping by.  You won’t be able to follow the old track very far but that is certainly enough to imagine the boring yet nerve-racking days that constituted the miners’ daily life.  Their version of fun back then was competing on their drilling abilities on the boulder brought there from Sierra Nevada I believe.  It is interesting though that they overcame so much trouble only to have a reliable point of comparison on their mining skills.  Something to inspire us on being Performance-Driven I suppose.

The top of hill overlooks San Jose.  Back in those days, it was a small downtown next to the Santa Clara mission, surrounded by miles and miles of orchards.  How I wish I could see that Spring panorama of fruit blossoms.  It must have been absolutely gorgeous!

A century ago, miles and miles of orchards surrounded downtown
A century ago, miles and miles of orchards surrounded downtown

© 2022 SparklingLogic. All Rights Reserved.