Skip to content

Testing the rules

Before proving anything, test. Tests are quick to write, they catch most mistakes, and when you change a rule they tell you at once what changed. Lean gives you three ways to test, from lightest to heaviest.

#guard: a check in any file

#guard takes something that is true or false, and fails the build if it's false:

#guard deck.length == 52
#guard (State.deal 7 100).hand0.down.length == 26
#guard other (other 0) == 0

Unlike #eval, it prints nothing when all is well. It's the right tool for facts you want to keep true while you work: put a few beside the code you're changing.

To test a move, look at what step gave back. A match inside the #guard says what you expect for each outcome:

def kings : State :=
  { hand0 := { up := [⟨13, 0⟩] }, hand1 := { down := [⟨2, 1⟩], up := [⟨13, 2⟩] }, maxFlips := 100 }

-- A valid snap: the caller gets both kings.
#guard match step kings 1 .snap with
  | .ok (s, _) => s.hand1.size == 3 && s.hand0.size == 0
  | .error _ => false

-- Player 0 has no cards left, so the snap also ends the game.
#guard match step kings 1 .snap with
  | .ok (s, _) => s.phase == .finished (some 1)
  | .error _ => false

decide: a check Lean proves

example : deck.length = 52 := by decide
example : (deck.filter (·.rank == 13)).length = 4 := by decide
example : ∀ p : Fin 2, other (other p) = p := by decide

These look like the #guards, but they are different in kind. Each is a proof: example states a fact, and by decide proves it by having Lean work it out. There is a reason this matters. A #guard runs the code once and looks at the answer. decide asks Lean's checker, the small trusted core that checks every proof, to confirm it.

The last line shows what decide can do that a test can't: it covers every case. ∀ p : Fin 2 reads for every player p, and since there are only two, Lean checks both. A test only ever checks the cases you thought of.

decide only works when there are finitely many cases, and few enough for Lean to work through. For everything else, there are the proofs that start in a later chapter.

The project's tests

The engine's own tests live in Tests/, and make test runs them all. Snap's are in Tests/Main.lean, built on a tiny helper that records a failure instead of stopping at the first one:

def check (name : String) (ok : Bool) (detail : String := "") : T Unit := do
  if ok then pure ()
  else
    IO.println s!"FAIL {name}{if detail.isEmpty then "" else ": " ++ detail}"
    modify fun f => { f with count := f.count + 1 }

and a shorthand for making a game with given piles:

def mk (d0 u0 d1 u1 : List Card) (turn : Fin 2 := 0) (maxFlips := 100) : State :=
  { hand0 := ⟨d0, u0⟩, hand1 := ⟨d1, u1⟩, turn, maxFlips }

Each test is then one check, with a name that reads like a sentence and the condition it expects. Open Tests/Main.lean and read snapTests: it's a list of them, game situation by game situation.

Playing thousands of games

Examples only test what you think of. The tests also play whole games with a player that picks a move at random, from whatever the engine offers:

/-- A random client: picks uniformly among enabled actions offered to it. -/
def randomGame (seed : Nat) : T (Option Outcome × Nat) := do
  let cfg := Json.obj #[("seed", Json.ofNat seed), ("maxFlips", 3000)]
  let .ok (s0 : Session State) := Session.start cfg | do check "sim: start" false; return (none, 0)
  let mut s := s0
  let mut g := Rng.ofSeed (seed + 1000)
  let mut moves := 0
  for _ in [0:20000] do
    if s.outcome.isSome then break
    let (actor, g') := g.below 2
    g := g'
    let offered := (Game.actions s.nameOf s.state (.player actor)).filter (·.enabled)
    if offered.isEmpty then continue
    let (k, g') := g.below offered.length
    g := g'
    let a := offered[k]!
    let (s', reply) := s.submit actor (.obj #[("id", a.id)])
    check s!"sim {seed}: offered action accepted" (reply.get? "ok" == some (.bool true)) reply.render
    s := s'
    -- The protocol's promises, every so often (a full check is quadratic in the game's length).
    if s.log.size % 400 == 0 || s.outcome.isSome then
      for p in Tests.Protocol.problems s do check s!"sim {seed}: protocol" false p
    moves := moves + 1
  check s!"sim {seed}: 52 cards at end" (s.state.cards.length == 52 && deck.all (s.state.cards.contains ·))
  check s!"sim {seed}: log matches moves" (s.log.size == moves)
  return (s.outcome, moves)

and check that every one of them ends:

def simTests : T Unit := do
  let mut finished := 0
  for seed in [0:50] do
    let (out, _) ← randomGame seed
    if out.isSome then finished := finished + 1
  check "sim: every game finishes (win or draw cap)" (finished == 50) s!"{finished}/50"

Random games find the cases nobody thought to write down: a recycled stack, a false snap with an empty pile, a game that hits its flip limit. They don't prove anything, but they find bugs fast.

Exercises

Write #guards for these, on a freshly dealt game (State.deal 3 100):

  1. Player 1 can't flip first, and the reason given is "not your turn".
  2. A finished game refuses a snap, with the reason "game is over".
  3. After player 0's first flip, they have 25 cards face down and one face up.
Solution
def fresh : State := State.deal 3 100

-- Only player 0 may flip first.
#guard match step fresh 1 .flip with
  | .error r => r.reason == "not your turn"
  | .ok _ => false

-- A finished game refuses every move.
#guard match step { fresh with phase := .finished none } 0 .snap with
  | .error r => r.reason == "game is over"
  | .ok _ => false

-- A flip moves exactly one card from the stack to the face-up pile.
#guard match step fresh 0 .flip with
  | .ok (s, _) => s.hand0.down.length == 25 && s.hand0.up.length == 1
  | .error _ => false

The second test makes a finished game by copying a fresh one with a different phase. The rules never check how a game got into a state, so a test can build any state it likes.