Testing the rules¶
Before proving anything, test. Tests are quick to write, they catch most mistakes, and when you change a rule they tell you at once what changed. Lean gives you three ways to test, from lightest to heaviest.
#guard: a check in any file¶
#guard takes something that is true or false, and fails the build if
it's false:
#guard deck.length == 52
#guard (State.deal 7 100).hand0.down.length == 26
#guard other (other 0) == 0
Unlike #eval, it prints nothing when all is well. It's the right tool for
facts you want to keep true while you work: put a few beside the code you're
changing.
To test a move, look at what step gave back. A match inside the #guard
says what you expect for each outcome:
def kings : State :=
{ hand0 := { up := [⟨13, 0⟩] }, hand1 := { down := [⟨2, 1⟩], up := [⟨13, 2⟩] }, maxFlips := 100 }
-- A valid snap: the caller gets both kings.
#guard match step kings 1 .snap with
| .ok (s, _) => s.hand1.size == 3 && s.hand0.size == 0
| .error _ => false
-- Player 0 has no cards left, so the snap also ends the game.
#guard match step kings 1 .snap with
| .ok (s, _) => s.phase == .finished (some 1)
| .error _ => false
decide: a check Lean proves¶
example : deck.length = 52 := by decide
example : (deck.filter (·.rank == 13)).length = 4 := by decide
example : ∀ p : Fin 2, other (other p) = p := by decide
These look like the #guards, but they are different in kind. Each is a
proof: example states a fact, and by decide proves it by having Lean
work it out. There is a reason this matters. A #guard runs the code once
and looks at the answer. decide asks Lean's checker, the small trusted core
that checks every proof, to confirm it.
The last line shows what decide can do that a test can't: it covers every
case. ∀ p : Fin 2 reads for every player p, and since there are only
two, Lean checks both. A test only ever checks the cases you thought of.
decide only works when there are finitely many cases, and few enough for
Lean to work through. For everything else, there are the proofs that start in
a later chapter.
The project's tests¶
The engine's own tests live in Tests/, and make test runs them all. Snap's
are in Tests/Main.lean, built on a tiny helper that records a failure
instead of stopping at the first one:
def check (name : String) (ok : Bool) (detail : String := "") : T Unit := do
if ok then pure ()
else
IO.println s!"FAIL {name}{if detail.isEmpty then "" else ": " ++ detail}"
modify fun f => { f with count := f.count + 1 }
and a shorthand for making a game with given piles:
def mk (d0 u0 d1 u1 : List Card) (turn : Fin 2 := 0) (maxFlips := 100) : State :=
{ hand0 := ⟨d0, u0⟩, hand1 := ⟨d1, u1⟩, turn, maxFlips }
Each test is then one check, with a name that reads like a sentence and the
condition it expects. Open Tests/Main.lean and read snapTests: it's a
list of them, game situation by game situation.
Playing thousands of games¶
Examples only test what you think of. The tests also play whole games with a player that picks a move at random, from whatever the engine offers:
/-- A random client: picks uniformly among enabled actions offered to it. -/
def randomGame (seed : Nat) : T (Option Outcome × Nat) := do
let cfg := Json.obj #[("seed", Json.ofNat seed), ("maxFlips", 3000)]
let .ok (s0 : Session State) := Session.start cfg | do check "sim: start" false; return (none, 0)
let mut s := s0
let mut g := Rng.ofSeed (seed + 1000)
let mut moves := 0
for _ in [0:20000] do
if s.outcome.isSome then break
let (actor, g') := g.below 2
g := g'
let offered := (Game.actions s.nameOf s.state (.player actor)).filter (·.enabled)
if offered.isEmpty then continue
let (k, g') := g.below offered.length
g := g'
let a := offered[k]!
let (s', reply) := s.submit actor (.obj #[("id", a.id)])
check s!"sim {seed}: offered action accepted" (reply.get? "ok" == some (.bool true)) reply.render
s := s'
-- The protocol's promises, every so often (a full check is quadratic in the game's length).
if s.log.size % 400 == 0 || s.outcome.isSome then
for p in Tests.Protocol.problems s do check s!"sim {seed}: protocol" false p
moves := moves + 1
check s!"sim {seed}: 52 cards at end" (s.state.cards.length == 52 && deck.all (s.state.cards.contains ·))
check s!"sim {seed}: log matches moves" (s.log.size == moves)
return (s.outcome, moves)
and check that every one of them ends:
def simTests : T Unit := do
let mut finished := 0
for seed in [0:50] do
let (out, _) ← randomGame seed
if out.isSome then finished := finished + 1
check "sim: every game finishes (win or draw cap)" (finished == 50) s!"{finished}/50"
Random games find the cases nobody thought to write down: a recycled stack, a false snap with an empty pile, a game that hits its flip limit. They don't prove anything, but they find bugs fast.
Exercises¶
Write #guards for these, on a freshly dealt game (State.deal 3 100):
- Player 1 can't flip first, and the reason given is
"not your turn". - A finished game refuses a snap, with the reason
"game is over". - After player 0's first flip, they have 25 cards face down and one face up.
Solution
def fresh : State := State.deal 3 100
-- Only player 0 may flip first.
#guard match step fresh 1 .flip with
| .error r => r.reason == "not your turn"
| .ok _ => false
-- A finished game refuses every move.
#guard match step { fresh with phase := .finished none } 0 .snap with
| .error r => r.reason == "game is over"
| .ok _ => false
-- A flip moves exactly one card from the stack to the face-up pile.
#guard match step fresh 0 .flip with
| .ok (s, _) => s.hand0.down.length == 25 && s.hand0.up.length == 1
| .error _ => false
The second test makes a finished game by copying a fresh one with a different phase. The rules never check how a game got into a state, so a test can build any state it likes.