models /
gpt-5.5 scaffold-assembly-v1
Provider: openai ·
Run at 2026-05-01 ·
Submission 2f4a9e99
CaSS 35.0%
CBS 15.0% v1: 0 by design
Coverage 55%
Cost $5.80
Tokens (in/out) 1.69M / 157.0k
Tier breakdown
The 3-dimension diagnostic matrix for this baseline. CBS can read 0 while the conservative v1 validator has low definitive coverage — that's expected for v1.
| Tier | BOM pass | Scope pass | Buildability | CaSS | CBS | Coverage | n |
|---|---|---|---|---|---|---|---|
| trivial | 100.0% | 75.0% | 50.0% | 75.0% | 25.0% | 100% | 4 |
| easy | 100.0% | 50.0% | 50.0% | 50.0% | 50.0% | 100% | 4 |
| medium | 100.0% | 50.0% | 0.0% | 50.0% | 0.0% | 75% | 4 |
| hard | 75.0% | 0.0% | — | 0.0% | 0.0% | 0% | 4 |
| stretch | 25.0% | 0.0% | — | 0.0% | 0.0% | 0% | 4 |
Per-task results
All 20 tasks attempted by this baseline. Click a task to see the same row from every model's perspective.
| # | Slug | Tier | CaSS | BOM | Scope | Build | Ref sim | Iter | Tokens | Duration | Status | 3D |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0001 | flat-plate-5x7 | trivial | 0 | 1 | 0 | 1 | 0.000 | 9 | 70.5k | 5m07s | 3D | |
| 0002 | flat-plate-5x5 | trivial | 1 | 1 | 1 | 1 | 0.000 | 4 | 21.5k | 1m48s | pass | 3D |
| 0003 | staggered-column-2x2 | trivial | 1 | 1 | 1 | 0 | 0.000 | 5 | 22.3k | 1m08s | pass | 3D |
| 0004 | hollow-frame-8x8 | trivial | 1 | 1 | 1 | 0 | 0.000 | 7 | 34.0k | 2m27s | pass | 3D |
| 0005 | small-table | easy | 1 | 1 | 1 | 1 | 0.000 | 3 | 18.4k | 38.3s | pass | 3D |
| 0006 | step-pyramid | easy | 1 | 1 | 1 | 1 | 0.400 | 5 | 17.3k | 43.4s | pass | 3D |
| 0007 | simple-gate | easy | 0 | 1 | 0 | 0 | 0.000 | 3 | 24.4k | 44.6s | 3D | |
| 0008 | picture-frame | easy | 0 | 1 | 0 | 0 | 0.294 | 4 | 60.6k | 3m13s | 3D | |
| 0009 | minifig-chair | medium | 1 | 1 | 1 | 0 | 0.000 | 3 | 27.8k | 57.5s | pass | 3D |
| 0010 | bookshelf-3-shelf | medium | 0 | 1 | 0 | 0 | 0.000 | 7 | 59.5k | 2m29s | 3D | |
| 0011 | simple-bridge | medium | 0 | 1 | 0 | incon. | 0.250 | 4 | 24.3k | 1m10s | 3D | |
| 0012 | square-lantern | medium | 1 | 1 | 1 | 0 | 0.500 | 5 | 101.9k | 3m40s | pass | 3D |
| 0013 | farm-tractor | hard | 0 | 0 | 0 | incon. | 0.000 | 4 | 9.5k | 17m43s | DNC | — |
| 0014 | rolling-cart | hard | 0 | 1 | 0 | incon. | 0.000 | 10 | 103.5k | 4m18s | 3D | |
| 0015 | drawbridge | hard | 0 | 1 | 0 | incon. | 0.000 | 7 | 70.0k | 2m54s | 3D | |
| 0016 | slatted-bench | hard | 0 | 1 | 0 | incon. | 0.000 | 9 | 143.3k | 5m49s | 3D | |
| 0017 | small-house | stretch | 0 | 0 | 0 | incon. | 0.000 | 10 | 152.5k | 5m34s | DNC | — |
| 0018 | small-lighthouse | stretch | 0 | 1 | 0 | incon. | 0.000 | 12 | 155.4k | 5m40s | 3D | |
| 0019 | garden-shed | stretch | 0 | 0 | 0 | incon. | 0.000 | 10 | 174.9k | 5m54s | DNC | — |
| 0020 | fountain-with-pool | stretch | 0 | 0 | 0 | incon. | 0.000 | 10 | 558.3k | 5m14s | DNC | — |
Submitted BOMs
The final BOM the model emitted on each task it attempted.
#0001 flat-plate-5x7 trivial CaSS 0
| part_num | color_id | qty |
|---|---|---|
3020 | 15 | 4 |
3021 | 15 | 4 |
3710 | 15 | 2 |
3623 | 15 | 2 |
| total pieces | 12 | |
#0002 flat-plate-5x5 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3031 | 4 | 2 |
3710 | 4 | 4 |
3024 | 4 | 2 |
| total pieces | 8 | |
#0003 staggered-column-2x2 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3004 | 4 | 26 |
| total pieces | 26 | |
#0004 hollow-frame-8x8 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3460 | 4 | 2 |
3666 | 4 | 2 |
3023 | 4 | 4 |
| total pieces | 8 | |
#0005 small-table easy CaSS 1
| part_num | color_id | qty |
|---|---|---|
3031 | 19 | 1 |
3005 | 70 | 12 |
| total pieces | 13 | |
#0006 step-pyramid easy CaSS 1
| part_num | color_id | qty |
|---|---|---|
3958 | 19 | 1 |
3031 | 19 | 1 |
3022 | 19 | 1 |
| total pieces | 3 | |
#0007 simple-gate easy CaSS 0
| part_num | color_id | qty |
|---|---|---|
3005 | 4 | 10 |
3009 | 4 | 1 |
| total pieces | 11 | |
#0008 picture-frame easy CaSS 0
| part_num | color_id | qty |
|---|---|---|
3032 | 15 | 1 |
3009 | 70 | 4 |
3034 | 70 | 3 |
3020 | 70 | 2 |
| total pieces | 10 | |
#0009 minifig-chair medium CaSS 1
| part_num | color_id | qty |
|---|---|---|
3005 | 70 | 8 |
3021 | 70 | 1 |
3004 | 70 | 1 |
| total pieces | 10 | |
#0010 bookshelf-3-shelf medium CaSS 0
| part_num | color_id | qty |
|---|---|---|
3032 | 6 | 3 |
3010 | 6 | 18 |
| total pieces | 21 | |
#0011 simple-bridge medium CaSS 0
| part_num | color_id | qty |
|---|---|---|
3003 | 71 | 8 |
3029 | 70 | 1 |
3633 | 72 | 6 |
| total pieces | 15 | |
#0012 square-lantern medium CaSS 1
| part_num | color_id | qty |
|---|---|---|
3031 | 72 | 2 |
3010 | 72 | 6 |
3004 | 72 | 6 |
3005 | 72 | 8 |
| total pieces | 22 | |
#0013 farm-tractor DNC
timeout after 300000ms
#0014 rolling-cart hard CaSS 0
| part_num | color_id | qty |
|---|---|---|
3032 | 4 | 1 |
3009 | 4 | 2 |
3004 | 4 | 2 |
4085a | 0 | 1 |
21445 | 71 | 4 |
30027a | 15 | 4 |
30028 | 0 | 4 |
| total pieces | 18 | |
#0015 drawbridge hard CaSS 0
| part_num | color_id | qty |
|---|---|---|
3035 | 72 | 1 |
3003 | 71 | 10 |
2429 | 72 | 1 |
2430 | 70 | 1 |
3032 | 70 | 1 |
| total pieces | 14 | |
#0016 slatted-bench hard CaSS 0
| part_num | color_id | qty |
|---|---|---|
60479 | 70 | 4 |
60479 | 308 | 4 |
3032 | 72 | 2 |
3010 | 0 | 8 |
3040a | 0 | 2 |
3665 | 0 | 2 |
| total pieces | 22 | |
#0017 small-house DNC
timeout after 300000ms
#0018 small-lighthouse stretch CaSS 0
| part_num | color_id | qty |
|---|---|---|
3958 | 71 | 3 |
92947 | 4 | 6 |
92947 | 15 | 6 |
3633 | 0 | 4 |
4864a | 47 | 4 |
3062a | 14 | 1 |
3031 | 0 | 1 |
3942a | 4 | 1 |
| total pieces | 26 | |
#0019 garden-shed DNC
timeout after 300000ms
#0020 fountain-with-pool DNC
timeout after 300000ms
Run metadata
| Submission ID | 2f4a9e99-859e-471b-ae92-c94a50dbb99b |
|---|---|
| Protocol | scaffold-assembly-v1 |
| Protocol hash | sha256:394f2d859fafd42d0ac144c5f251e68ea2e7f4c06da48bf733e1c2505055ee9c |
| Run at | 2026-05-01T14:21:05.149Z |
| Total duration | 77m11s |
| Adapter health | ok |