Recently at DecisionCAMP 2026, our CPO Carole-Ann Berlioz delivered a pointed warning to the decision management community. The shift from AI as Decision Assistant to Decision Architect isn’t a feature upgrade. It’s a change in risk category. In this post, we’ll cover key insights from her talk, “From Assistant to Architect: The High-Stakes Lead to AI-Driven Decision Models.”
AI as Decision Assistant
A decision assistant supports business analysts in decision management. For example, SMARTS™ AI Assistant helps users explain existing decision logic, answer queries, draft rules, generate test cases, and more. As an assistant, the human is still in control. AI works on one component of the decision at a time, and humans review and integrate every output it produces. Errors stay contained and testing usually catches them.
AI as Decision Architect
However, as AI becomes a decision architect, the consequences grow. If AI gains the authority to architect the full decision, resolve conflicts, and produce executable logic, any errors become systemic. They’re no longer easy to catch. Consider these contributing factors:
- Validation Challenge: Validating the emergent behavior of hundreds of AI-generated rules is a combinatorial problem most QA processes weren’t built for. Even when individual rules are correct, they can interact in ways that produce a wrong decision.
- Explainability Challenge: An AI-authored decision may optimize toward an objective function that resists rule-by-rule breakdown. That makes it hard for humans to explain or understand.
- The Assumptive Nature of AI: When requirements are incomplete, an AI decision architect fills the gap with statistically typical assumptions. The resulting model may be technically correct, but it will likely look like everyone else’s. It will lack the non-obvious rules that made the organization’s logic proprietary in the first place.
Why Requirements Elicitation Should Still Be Done by Business Analysts
Requirements elicitation is a political exercise. It involves building trust and mediating between stakeholders who aren’t always forthright about their needs. Seasoned business analysts know how to ask the right questions. They uncover tacit knowledge, unspoken operating principles, and edge cases that give an organization its competitive edge. AI can’t easily replicate this skill.
Additional Questions That Need to be Addressed
Keeping requirements elicitation under human control is not enough to combat the risks of promoting AI to decision architect. Before taking that leap, organizations should answer these questions:
- What types of decisions should and shouldn’t AI architect?
- What gives your decision a competitive edge, and how will you preserve it?
- Who is accountable for decision outcomes?
- How will you validate and explain a decision, both holistically and rule by rule?
- How will you identify systemic errors in production, and what guardrails will minimize exposure?
- What traceability do you need for compliance and for tracking how a decision model changes over time?
AI + Human Expertise
Moving toward AI-driven decision models changes what you need to validate, who’s accountable, and whether your decision logic still reflects the judgment that made it yours. The organizations that get this right won’t move the fastest. They’ll deliberately decide what to hand over to AI and build the right guardrails first. That means designing a system where AI and humans each focus on what they do best. Carole-Ann will discuss this and more in her upcoming webinar: Roadmap for AI-Assisted Decision Making.
Also, check out Carlos’ take on a third AI role: Runtime counselor with System One models.
Contact us today for a customized demo of SMARTS™ to learn how you can put AI to work as a power assistant — without giving up the human oversight, explainability, and competitive edge that a fully autonomous architect can quietly erode!