Scenario Testing

Scenario Testing

Scenario tests let you verify that a .cade configuration produces exact scoring outcomes, variable state changes, and event processing behavior. You declare a game state, fire events, and assert the results — all in an HCL file with a .cade.test extension.

Scenarios run against the real scoring engine, not a mock. Every scenario in docs/examples/ is executed automatically in CI, so a failing example is caught before it ships.

Writing a Scenario

A scenario file has four sections: metadata, initial state, events, and expectations.

scenario "bumper_scoring" {
  name        = "Single bumper hit awards 1000 points"
  description = "Verifies base scoring for a standard bumper device."
  tags        = ["scoring", "bumper"]

  scoring_config = "path/to/my-table.cade"

  initial_state {
    state_id     = "fresh_game"
    ball_in_play = 1

    player "player1" {
      name   = "P1"
      active = true
      score  = 0
    }

    scores = {
      "player1" = 0
    }
  }

  event "hit_bumper" {
    type = "device.bumper.hit"
    time = "0ms"
  }

  expect "score_is_1000" {
    description = "Final score must be exactly 1000"
    type        = "score"

    condition = {
      min_score = 1000
      max_score = 1000
    }
  }

  expect "all_events_succeed" {
    description = "All events processed without error"
    type        = "event"

    condition = {
      require_success = true
    }
  }
}

scoring_config

Path to the .cade file under test, relative to the project root. If omitted, the scenario runner looks for a .cade file in the same directory.

initial_state

Warps the engine to a known starting point before any events fire. You set the ball number, declare players with starting scores, and optionally pre-set variables and mode state. Every scenario should begin from an explicit state so results are deterministic.

Variables

The variables attribute accepts nested objects with mixed value types — strings, numbers, booleans, and lists can coexist in the same variable group:

initial_state {
  ball_in_play = 1

  variables = {
    player = {
      combo_count = 0
      last_shot   = "none"
      multiplier  = 1.5
    }
    session = {
      bonus_level    = 3
      targets_lit    = [1, 3, 5]
      jackpot_active = true
    }
  }
}

Mode State

Pre-set mode-specific state with the mode block. The state attribute accepts the same nested object shapes as variables:

initial_state {
  ball_in_play = 1

  mode "multiball" {
    active = true
    state = {
      balls_locked  = 2
      jackpot_value = 50000
    }
  }
}

event Blocks

Each event block injects one event into the engine. Events fire in declaration order. The time attribute controls simulated elapsed time — use "0ms" for the first event and increasing offsets for timed sequences (combos, bursts, mode timers).

event "first_hit" {
  type = "device.left_target.hit"
  time = "0ms"
}

event "second_hit" {
  type = "device.right_target.hit"
  time = "50ms"
}

Events can carry a data attribute with arbitrary key-value payloads. The payload is forwarded to event handlers exactly as declared:

event "orbit_shot" {
  type = "shot.left_orbit.complete"
  time = "100ms"
  data = {
    target     = "left_orbit"
    multiplier = 2
    combo      = true
  }
}

expect Blocks

Expectations run after all events have been processed. Each expect block has a type that determines what it checks.

Score Expectations

Use min_score and max_score together for exact checks, or set only one for range checks:

expect "exact_score" {
  description = "Awards exactly 5000 points"
  type        = "score"
  condition   = { min_score = 5000, max_score = 5000 }
}

Variable Expectations

Check that a variable reached a specific value:

expect "counter_incremented" {
  description = "Hit counter is 3 after three hits"
  type        = "variable"
  condition   = { name = "hit_count", expected_value = 3 }
}

Running Scenarios

Run a single scenario file:

cade scenario run path/to/my-scenario.cade.test

Run all scenarios in a directory:

cade scenario run docs/examples/scenarios/ --recursive

Output as JSON for CI integration:

cade scenario run docs/examples/ -r --format json

Flags

File Conventions

Scenario files use the .cade.test extension. The recommended directory layout for testing a feature is:

docs/examples/advanced/my-feature/
├── feature.cade           # Minimal config exercising the feature
├── positive.cade.test     # Scenario asserting correct behavior
└── negative.cade.test     # Scenario with wrong assertions (must fail)

The negative scenario is a safety net: it asserts a deliberately wrong outcome. If the negative ever passes, the positive scenario’s assertions are not actually checking what they claim.

Files placed under docs/examples/ are automatically picked up by the corpus test walker in CI — no additional wiring is needed.

Advanced Feature Matrix

The docs/examples/advanced/ directory contains scenario pairs for advanced config-language features. Each subdirectory exercises one feature area:

These examples double as documentation of what the config language supports in edge cases. If you’re trying to do something complex with fragments, assemblies, or mode stacking, look here for working examples.

Fuzz Testing

Fuzz testing generates random .cade configurations and feeds them through the full parse-validate-execute pipeline, catching panics and unbounded memory growth that hand-authored scenarios would miss.

Run all fuzz suites:

task fuzz

Individual suites:

task test:fuzz:config     # Random .cade generation (120s default)
task test:fuzz:engine     # Engine integration (120s default)
task test:fuzz:resilience # Event resilience with race detection (120s)
task test:fuzz:platform   # Platform driver conformance (30s)

Fuzz runs are time-bounded by default and run on a nightly schedule in CI rather than on every PR. Crash-reproducing inputs are saved automatically under testdata/fuzz/ and become permanent regression tests.

Validation Without Scenarios

For quick config validation without writing a full scenario, use cade validate:

cade validate my-table.cade
cade validate docs/examples/ -r
cade validate --strict my-table.cade

Validation checks parse correctness and structural rules but does not execute the scoring engine. Use scenarios when you need to verify runtime behavior (scores, variable mutations, event cascades).