// DOCUMENTATION

Methodology & limitations

How benchmarks are run, what they measure, and what they do not.

Benchmarks

Koko benchmarks compare against Playwright bundled Chromium (headless) — not Google Chrome desktop. Numbers are machine-local and intended for regression tracking, not marketing absolutes.

Microbench methodology

  • Koko: zig-out/bin/koko serve + CDP navigation/evaluate
  • Chromium: chromium.launch({ headless: true })
  • Fixtures: local static HTML in koko-test/ (no CDN)
  • Navigation metric: Page.navigate / goto until domcontentloaded + DOM size probe
  • JS metric: in-page performance.now() for dom-query, JSON loop, FNV-style hash loop
  • Startup metric: process spawn until browser ready (Koko: /json/version; Chromium: launch + about:blank)
  • Startup warmup/repeats: 2/5

Crawl methodology

  • Site: live en.wikipedia.org (100 shared URLs)
  • Mode: extract — title + links via querySelector
  • Concurrency: 8 parallel workers
  • Resource sampling: every 100ms via process tree (RSS, CPU%, process count)
  • Koko: 8 isolated koko serve processes
  • Chromium: 8 tabs in 1 browser process tree

Limitations

  • Single-machine runs — CPU load affects numbers
  • Local static pages do not represent heavy SPAs or real sites
  • Playwright Chromium ≠ installed Google Chrome
  • Wikipedia crawl does not stress WebGL, hydration, or bot detection
  • Process count comparisons are architectural, not fairness claims
  • GPU utilization not available headless

Interpreting ratios

Ratio Koko / Chromium:

  • < 1.0 — Koko uses less (better for memory/CPU/time)
  • > 1.0 — Koko uses more

Cost model

total_cost = tasks × cpu_sec_per_task × $/CPU-sec
           + sessions / sessions_per_GB × $/GB-RAM

Koko's lower RSS/page and higher sessions/GB directly reduce RAM cost for agent pools and crawl fleets.