Model Router

Wide: one row per prompt

prompt_id,prompt,tiny_correct,tiny_cost,tiny_latency,tiny_confidence,frontier_correct,frontier_cost,frontier_latency,frontier_confidence
q1,What is 17 times 23?,1,0.00004,310,0.91,1,0.0121,2400,0.98
q2,Summarize this contract clause: ...,0,0.00009,620,0.77,1,0.0263,4100,0.93

Long: one row per prompt and model

prompt_id,model,correct,cost,latency,confidence,prompt
q1,tiny,1,0.00004,310,0.91,What is 17 times 23?
q1,frontier,1,0.0121,2400,0.98,What is 17 times 23?

correct

required. 1 if the model's answer was right, 0 if not (true/false and yes/no work too), or a score between 0 and 1. In the wide layout the column is <model>_correct; score is also accepted.

cost

required. What that call cost, in any unit you like: dollars, cents or tokens. The report uses the same unit, per 1,000 prompts.

confidence

optional, 0 to 1. The model's own estimate that it was right (logprob of the answer, a self-rating, a verifier score). This is what a cascade gates on. Without it, a router is learned from the prompt text instead.

prompt

optional. The prompt text, used to learn a router that picks a model before any call is made.

latency

optional, any unit. Reported, not optimised.

prompt_id

optional in the wide layout, needed in the long one to group a prompt's rows. Prompts without a row for every model are dropped.

Models are ordered by their average cost. Delimiters (comma, semicolon, tab) and quoted fields are detected. Column names are case-insensitive; cost_usd, price, tokens, is_correct, p_correct, question and similar names are recognised.