~/bench

Model benchmark

Loads a tier in this tab, runs fixed prompts, and reports load time, time to first token and decode speed. Close other model tabs first: one engine per device.

Tiers

load the vision model too and run one image prompt ignore eligibility (may crash the tab on small devices) profile run: per-pass GPU timings on the first prompt (slower while profiling; use when asked) quick (short prompt only) long system prompt only (about 450 prompt tokens, the shape a code-writing tool sends; checks prefill routes at real prompt lengths) runs per prompt init/load options JSON (applied to init and loadModel) generate options JSON, one object or a list (each entry runs the prompts once on the same load, in order) max tokens per prompt (blank = prompt default)

idle

This device

reading…