BrickAGI benchmark

Models

One card per (model, protocol) baseline. Ranked rows are current, complete, and task-set-pinned; diagnostic rows remain visible for auditability.

gpt-5.5

openai
raw-assembly-v1
CaSS 28.7%
coverage 32% cost $7.91

claude-opus-4-8-coordinator

anthropic
scaffold-assembly-v1
CaSS 100.0%
coverage 100% cost $0 agent harness legacy

claude-opus-4-8-coordinator-1shot

anthropic
scaffold-assembly-v1
CaSS 57.4%
coverage 49% cost $0 agent harness legacy

claude-opus-4-8-coordinator-blind

anthropic
scaffold-assembly-v1
CaSS 57.4%
coverage 51% cost $0 agent harness legacy

gpt-5.5

openai
scaffold-v1
CaSS 45.0%
coverage 0% cost $4.54 partial breaker DNR ×3

gemini-3-pro-preview

google
raw-v1
CaSS 39.6%
coverage 0% cost $4.20 partial breaker DNR ×5

gpt-5.5

openai
scaffold-assembly-v1
CaSS 39.3%
coverage 48% cost $7.43 partial

claude-opus-4-7

anthropic
scaffold-v1
CaSS 37.5%
coverage 13% cost $8.48 partial

gemini-3-pro-preview

google
scaffold-v1
CaSS 36.1%
coverage 9% cost $3.28 partial

gpt-5.5

openai
raw-v1
CaSS 35.0%
coverage 5% cost $4.90 partial breaker

gpt-5.4-mini

openai
scaffold-assembly-v1
CaSS 10.7%
coverage 34% cost $10.47 partial

claude-opus-4-7

anthropic
raw-v1
CaSS 0.0%
coverage 0% cost $0 incomplete