Use the Quickcheck CLI¶

themis-quickcheck reads stored SQLite projections without importing your benchmark code.

Scores¶

themis-quickcheck scores \
  --db .cache/themis-examples/01-hello-world-benchmark-first/themis.sqlite3 \
  --metric exact_match

Output:

ev:73c1137b0a71 demo-model  arithmetic  exact_match 1.0000  1

Filter by slice:

themis-quickcheck scores \
  --db .cache/themis-examples/04-compare-models-benchmark-first/themis.sqlite3 \
  --metric exact_match \
  --slice qa

Filter by benchmark dimension:

themis-quickcheck scores \
  --db .cache/themis-examples/01-hello-world-benchmark-first/themis.sqlite3 \
  --metric exact_match \
  --dimension source=synthetic

Failures¶

themis-quickcheck failures \
  --db .cache/themis-examples/01-hello-world-benchmark-first/themis.sqlite3 \
  --limit 20

For a fully successful run, failures prints nothing.