The HBR article’s coolest example is a 40-year-old bank that began using an LLM for credit approvals. It is initially relying on AI for credit approval but later balanced autopilot choices on approving from its Credit Committee where AI (LLM based!) handled about 15% of reviews by starting quarterly “credit roundtables.” These chats move the conversation from “Does this fit our rules?” to “Are our rules still working?”.

The real risk is not that AI makes bad decisions. Hallucinations happen, especially when context poisoned, but they are visible and fixable. The deeper danger is that AI silently replaces the process that made an organization’s judgment its own:

• Homogenization: a generic model nudges every firm toward the same “standard” answer.

• Automation bias: the bank’s committee kept meeting, but drifted into rubber-stamping AI recommendations.

• Broken knowledge transfer: juniors learn judgment by watching seniors argue over cases. If AI settles the case, nobody argues, and nobody learns.

There is a technical parallel. Training generative models on AI-generated data triggers “model collapse,” a digital game of telephone. Without a steady supply of fresh human content, each generation becomes a photocopy of a photocopy, and by the ninth generation one well-known test model had lost the plot entirely. Organizations face the same loop: if your decisions feed on machine-made decisions, your competitors’ do too, and strategy and tactics converge, whatever fine-tuning you have done.

Note: You can customize LLMs to protect corporate wisdom through context injection, custom embeddings, small language models, RAG, supervised fine-tuning and preference alignment. But these are long, hard paths that consume a lot of effort and tokens.

In short: the hidden risk of generative AI at work is not only wrong decisions, but losing the skills that make an organization uniquely good at making them right. When teams rubber-stamp LLM output, expert debate fades, juniors lose chances to learn, and distinctive judgment drifts toward the generic.

Does System One decision model help here over LLM?

Answer is Partially Yes.

Through the bank credit-committee scenario, this article explores how System One’s explicit decision contracts, confidence gates, and auditable human review can make AI-assisted decisions more disciplined while keeping expertise active. The advantage comes from more than changing models: it requires preserving the structures through which people challenge decisions, teach judgment, and revise the rules.

The bank’s two fixes target exactly those. The roundtables separate “does this meet our criteria?” from “are our criteria still right?”. The apprenticeships teach juniors to challenge scores, not just read them as the diagram below illustrates.

Where System One helps. It is better suited to this than a generic LLM, for four reasons that come straight based on the design principles explained in my recent book Mastering TypeSafe Jev (https://books.sheikaxis.com/mastering-jev):

Architecting deterministic, sub-second decision layers for mission-critical systems.

1. The criteria belong to you, in writing. A System One model doesn’t invent its own standard of a “good” decision. It answers questions you define, with criteria in your words, in a versioned contract. That contract is the documented lending criteria the roundtable debates. “Are our criteria still the right ones?” becomes a reviewable change to a file, under the same change control the book uses for thresholds.

2. It admits uncertainty, so humans get the hard cases. When evidence is thin or unusual, the probabilities flatten and the gate routes the case to people. The committee’s 15% isn’t a random sample but the cases where judgment matters most.

3. Policy stays in code and the trail is auditable. “Code computes, the model decides narrow questions, people authorise” keeps the model from becoming the policy. The audit trail records what the model said, what the human decided and why they disagreed. That is material for a roundtable.

4. Self-hosted models can learn your organization’s judgment. With Kev or THX-01 you can fine-tune on your own experts’ labelled outcomes, pulling the model towards your organization’s way of deciding rather than a generic one. A hosted model like Jev only reflects your judgment through the contract.

Don’t miss reading Appendix A: — Why Jev or Kev Instead of a Regular LLM, and How Decision Engines Work Inside in this book.

Here the examples of bank credit-committee setting, written in the same shape as the book’s FinTech contract and policy code. The criteria text and thresholds are just illustrative purpose.

decision contract

A contract is the versioned set of questions the model answers, with your own definitions of each label. The roundtable debates this file. Here, a sample contract.

CONTRACT_VERSION = "credit-contract/1.3.0"   # bump on every roundtable change

CREDIT_QUESTIONS = {
    "risk_band": {
        "type": "choice",
        "instructions": "Which risk band fits `applicant`, given `financials` and `history`?",
        "criteria": {
            "LOW":    "Stable income, low debt load, clean repayment history",
            "MEDIUM": "One weak signal (thin history or high debt load), no default flags",
            "HIGH":   "Several weak signals, or any recent missed payment",
            "REFER":  "Evidence is missing, conflicting or unusual for our book",
        },
    },
    "meets_policy": {
        "type": "noul",
        "instructions": "Does this application meet the written lending criteria in `policy_version`?",
        "criteria": {
            "true":  "Every stated criterion is satisfied by the evidence",
            "false": "At least one criterion is unmet or cannot be checked",
        },
    },
    "judgment_needed": {                      # speculative, cheap extra question
        "type": "noul",
        "instructions": "Does this case involve circumstances our criteria do not clearly cover?",
    },
}

Things to consider:

• REFER is an exit door. Missing evidence must never be forced into LOW.

• Hard rules such as “no loans above the branch limit” stay in the application code, not in the criteria text.

• judgment_needed asks the model to flag cases that fall outside the written criteria. That gives the roundtable its raw material.

Confidence Gates

The gate turns the model's probabilities into a route. Thresholds differ by action because the cost of being wrong differs. Here a Python example.

from dataclasses import dataclass

@dataclass(frozen=True)
class Gates:
    approve_min: float = 0.92     # auto-approve only when very sure
    decline_min: float = 0.97     # declines hurt customers, so even stricter
    margin_min: float = 0.30      # top option must beat the runner-up by this much
    audit_sample: float = 0.05    # random share of confident cases sent to apprentices

def route(band: str, p_top: float, margin: float, meets_policy: float,
          judgment_needed: float, g: Gates, rnd: float) -> str:
    if band == "REFER" or judgment_needed >= 0.50:
        return "committee"                       # out of criteria: humans decide
    if p_top < 0.60 or margin < g.margin_min:
        return "committee"                       # flat bars: model is unsure
    if band == "LOW" and p_top >= g.approve_min and meets_policy >= 0.90:
        return "apprentice_review" if rnd < g.audit_sample else "auto_approve"
    if band == "HIGH" and p_top >= g.decline_min:
        return "senior_review"                   # never auto-decline
    return "analyst_review"                      # middle: one human looks

How different inputs route:

Things to notice:

• Asymmetric gates: approve at 0.92, decline at 0.97, and declines are never automatic.

• Margin check: a 0.55 vs 0.45 split can have a decent top probability and still be a near coin-flip.

• Audit sample: some confident approvals go to apprentices anyway, so juniors keep seeing clear cases and the committee can check the model.

• Judgment flag: judgment_needed routes cases to the committee even when the model is confident, which feeds the roundtable.

How the pieces connect

1. The roundtable changes the criteria, so CONTRACT_VERSION goes from 1.3.0 to 1.4.0.

2. The gates stay in code. Change them only after replaying labelled cases against the new contract.

3. Every decision logs the contract version, the full probabilities and the route. If an apprentice or committee overrides the model, log the reason.

Challenges with System One

Where it makes things worse. Its strengths are also its risks:

• Fast, cheap and calibrated means more automation. A 0.95 confidence is easy to rubber-stamp, and that is the autopilot problem with better statistics.

• The gate starves juniors of learning. If only low-confidence cases reach people, juniors never see the clear-cut ones that teach the basics. They also never see confidently wrong cases, which are where challenging a score matters most.

• Fine-tuning can lock in sameness. If humans rubber-stamp and those approvals become training labels, the model learns to agree with itself. The book already warns never to use the model’s own answers as labels, and rubber-stamped labels are the same thing in disguise.

• Calibration is not wisdom. A calibrated score tells you how often the model is right on cases like past ones. It can’t tell you the criteria have gone stale. Only the roundtable question can.

Bottom line. System One fits the challenges on AI at work described in the article better than a generic LLM, because it makes the criteria explicit, separates judgment from computation, and routes uncertainty to people. But the HBR article’s real point is organizational: expertise comes from structures where people debate and teach. No model creates those structures. At best, a well-designed one leaves room for them and feeds them better material.

1. No typed number. An LLM writes “0.9” as text. A System One model scores every option in one forward pass and turns the scores into probabilities that add up to 1.

2. Confidence is distance from an even split. One tall bar means high confidence. Bars that are all about equal mean low confidence.

3. Thin evidence gives flat bars. With only “customer is upset”, four options might land near 0.30, 0.28, 0.24 and 0.18. That is a low-confidence answer, and your gate should send it to a human. With “card used in 3 countries in 1 hour”, fraud might reach 0.86.

4. Training and tuning keep the bars honest. Jev is trained so its probabilities match how often it is right (RLCD). Kev applies a fitted temperature, which rescales the bars without changing which option wins.

Interested in learning System One with two capstone projects on FinTech and Industrial IoT.

Explore the full architecture blueprint, production ready reusable foundation code, complete code implementations, and interactive diagrams in Mastering Jev: Type-Safe System 1 AI in Production book.

Architecting deterministic, sub-second decision layers for mission-critical systems.

• Deterministic Primitives: Replace brittle JSON prompt schemas with Choice, Score, and Noul.

• Sub-Second Latency: Low-overhead System 1 probabilistic inference designed for high-throughput streaming.

• Real-World Capstone: Full architectural implementation covering autonomous FinTech fraud triage and Industrial IoT edge control.

You’re also seeing a ticket from a few users on a certain browser who can’t download this year’s financial statements. Given tax season, how sure is your system that this is urgent and needs quick action?

Because LLMs are autoregressive and uncalibrated, they aren’t the best fit here—and they can cost more and 7- 8 seconds latency. Models like Jev or Kev are probabilistic, which helps make confident decisions with low latency of 70 - 500 milliseconds.

System One (Jev/Kev) as your judgement system, and System Two (LLM) steps in if you need to draft a reply for customers based on their tickets.

Would you like to try System One with a hands-on FinTech Triage Engine project like the one in the below diagram?

Explore the full architecture blueprint, production ready reusable foundation code, complete code implementations, and interactive diagrams in Mastering Jev: Type-Safe System 1 AI in Production book.

Architecting deterministic, sub-second decision layers for mission-critical systems.

• Deterministic Primitives: Replace brittle JSON prompt schemas with Choice, Score, and Noul.

• Sub-Second Latency: Low-overhead System 1 probabilistic inference designed for high-throughput streaming.

• Real-World Capstone: Full architectural implementation covering autonomous FinTech fraud triage and Industrial IoT edge control.