models /
gpt-5.5 scaffold-assembly-v1
Provider: openai ·
Run at 2026-06-09 ·
Submission b94214a4
Build Points 93 / 2,461 available
CaSS 39.3%
CBS 17.9%
Coverage 48%
Cost $7.43
Tokens (in/out) 2.09M / 221.6k
Diagnostic row, not ranked. legacy task-set provenance; 29/37 task results
Tier breakdown
The 3-dimension diagnostic matrix for this submission. A tier shows no CBS until the structural prover returns a definitive verdict there.
| Tier | BOM pass | Scope pass | Buildability | CaSS | CBS | Coverage | n |
|---|---|---|---|---|---|---|---|
| trivial | 100.0% | 100.0% | 75.0% | 100.0% | 75.0% | 100% | 4 |
| easy | 100.0% | 50.0% | 66.7% | 50.0% | 50.0% | 75% | 4 |
| medium | 100.0% | 75.0% | 0.0% | 75.0% | 0.0% | 100% | 4 |
| hard | 100.0% | 50.0% | 0.0% | 50.0% | 0.0% | 25% | 4 |
| stretch | 50.0% | 0.0% | 0.0% | 0.0% | 0.0% | 25% | 4 |
| expert | 75.0% | 0.0% | 0.0% | 0.0% | 0.0% | 25% | 4 |
| master | 0.0% | 0.0% | — | 0.0% | 0.0% | 0% | 5 |
Per-task results
All 29 tasks attempted by this baseline. Click a task to see the same row from every model's perspective.
| # | Slug | Tier | CaSS | BOM | Scope | Build | Ref sim | Iter | Tokens | Duration | Status | 3D |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0001 | flat-plate-5x7 | trivial | 1 | 1 | 1 | 1 | 0.000 | 5 | 28.1k | 2m39s | pass | 3D |
| 0002 | flat-plate-5x5 | trivial | 1 | 1 | 1 | 1 | 0.000 | 4 | 20.3k | 1m26s | pass | 3D |
| 0003 | staggered-column-2x2 | trivial | 1 | 1 | 1 | 0 | 0.000 | 5 | 21.6k | 1m12s | pass | 3D |
| 0004 | hollow-frame-8x8 | trivial | 1 | 1 | 1 | 1 | 0.000 | 6 | 29.1k | 3m01s | pass | 3D |
| 0005 | small-table | easy | 1 | 1 | 1 | 1 | 0.000 | 3 | 18.7k | 1m02s | pass | 3D |
| 0006 | step-pyramid | easy | 1 | 1 | 1 | 1 | 0.400 | 3 | 9.2k | 29.6s | pass | 3D |
| 0007 | simple-gate | easy | 0 | 1 | 0 | 0 | 0.000 | 3 | 24.3k | 59.9s | 3D | |
| 0008 | picture-frame | easy | 0 | 1 | 0 | incon. | 0.048 | 10 | 257.3k | 9m17s | 3D | |
| 0009 | minifig-chair | medium | 1 | 1 | 1 | 0 | 0.000 | 4 | 29.6k | 1m15s | pass | 3D |
| 0010 | bookshelf-3-shelf | medium | 1 | 1 | 1 | 0 | 0.000 | 5 | 68.2k | 2m40s | pass | 3D |
| 0011 | simple-bridge | medium | 0 | 1 | 0 | 0 | 0.286 | 3 | 15.3k | 1m18s | 3D | |
| 0012 | square-lantern | medium | 1 | 1 | 1 | 0 | 0.429 | 3 | 41.7k | 3m07s | pass | 3D |
| 0013 | farm-tractor | hard | 1 | 1 | 1 | incon. | 0.024 | 12 | 248.0k | 5m46s | pass | 3D |
| 0014 | rolling-cart | hard | 0 | 1 | 0 | incon. | 0.000 | 18 | 141.1k | 4m06s | 3D | |
| 0015 | drawbridge | hard | 1 | 1 | 1 | incon. | 0.345 | 5 | 49.8k | 5m34s | pass | 3D |
| 0016 | slatted-bench | hard | 0 | 1 | 0 | 0 | 0.000 | 5 | 61.6k | 4m00s | 3D | |
| 0017 | small-house | stretch | 0 | 0 | 0 | incon. | 0.000 | 10 | 140.4k | 8m07s | DNC | — |
| 0018 | small-lighthouse | stretch | 0 | 1 | 0 | 0 | 0.000 | 7 | 54.2k | 3m51s | 3D | |
| 0019 | garden-shed | stretch | 0 | 0 | 0 | incon. | 0.000 | 12 | 185.2k | 9m39s | DNC | — |
| 0020 | fountain-with-pool | stretch | 0 | 1 | 0 | incon. | 0.036 | 9 | 422.8k | 6m33s | 3D | |
| 0021 | grand-bridge | expert | 0 | 1 | 0 | incon. | 0.101 | 9 | 120.3k | 7m35s | 3D | |
| 0022 | manor-house | expert | 0 | 1 | 0 | incon. | 0.109 | 6 | 118.0k | 9m41s | 3D | |
| 0023 | clock-tower | expert | 0 | 1 | 0 | 0 | 0.027 | 7 | 73.2k | 2m35s | 3D | |
| 0024 | grand-fountain | expert | 0 | 0 | 0 | incon. | 0.000 | 9 | 122.5k | 20m14s | DNC | — |
| 0025 | mosaic-smiley | master | 0 | 0 | 0 | incon. | 0.000 | 1 | 0 | 17m36s | DNC | — |
| 0026 | voxel-tree | master | 0 | 0 | 0 | incon. | 0.000 | 2 | 2.1k | 20m18s | DNC | — |
| 0027 | grand-mosaic-sunset | master | 0 | 0 | 0 | incon. | 0.000 | 3 | 6.0k | 8m28s | DNC | — |
| 0028 | mega-voxel-rocket | master | 0 | 0 | 0 | incon. | 0.000 | 1 | 0 | 10m47s | DNC | — |
| 0029 | colossal-mosaic-grid | master | 0 | 0 | 0 | incon. | 0.000 | 1 | 0 | 15m13s | DNC | — |
Submitted BOMs
The final BOM the model emitted on each task it attempted.
#0001 flat-plate-5x7 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3032 | 15 | 2 |
3666 | 15 | 1 |
3710 | 15 | 1 |
3024 | 15 | 1 |
| total pieces | 5 | |
#0002 flat-plate-5x5 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3031 | 4 | 2 |
3710 | 4 | 2 |
3024 | 4 | 1 |
| total pieces | 5 | |
#0003 staggered-column-2x2 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3004 | 4 | 26 |
| total pieces | 26 | |
#0004 hollow-frame-8x8 trivial CaSS 1
| part_num | color_id | qty |
|---|---|---|
3460 | 0 | 2 |
3666 | 0 | 2 |
3023 | 0 | 4 |
| total pieces | 8 | |
#0005 small-table easy CaSS 1
| part_num | color_id | qty |
|---|---|---|
3031 | 70 | 1 |
3005 | 0 | 12 |
| total pieces | 13 | |
#0006 step-pyramid easy CaSS 1
| part_num | color_id | qty |
|---|---|---|
3958 | 19 | 1 |
3031 | 19 | 1 |
3022 | 19 | 1 |
| total pieces | 3 | |
#0007 simple-gate easy CaSS 0
| part_num | color_id | qty |
|---|---|---|
3005 | 4 | 10 |
3009 | 4 | 1 |
| total pieces | 11 | |
#0008 picture-frame easy CaSS 0
| part_num | color_id | qty |
|---|---|---|
3036 | 70 | 1 |
3032 | 15 | 1 |
3666 | 70 | 2 |
3710 | 70 | 2 |
3937 | 70 | 1 |
3938 | 70 | 1 |
3020 | 70 | 1 |
3034 | 70 | 1 |
| total pieces | 10 | |
#0009 minifig-chair medium CaSS 1
| part_num | color_id | qty |
|---|---|---|
3005 | 70 | 8 |
11212 | 70 | 1 |
3622 | 70 | 1 |
| total pieces | 10 | |
#0010 bookshelf-3-shelf medium CaSS 1
| part_num | color_id | qty |
|---|---|---|
3009 | 6 | 9 |
3622 | 6 | 12 |
3010 | 6 | 6 |
| total pieces | 27 | |
#0011 simple-bridge medium CaSS 0
| part_num | color_id | qty |
|---|---|---|
3003 | 71 | 8 |
3029 | 71 | 1 |
6112 | 71 | 2 |
| total pieces | 11 | |
#0012 square-lantern medium CaSS 1
| part_num | color_id | qty |
|---|---|---|
3031 | 72 | 2 |
3010 | 72 | 6 |
3004 | 72 | 10 |
3005 | 72 | 8 |
| total pieces | 26 | |
#0013 farm-tractor hard CaSS 1
| part_num | color_id | qty |
|---|---|---|
3795 | 2 | 1 |
3020 | 2 | 2 |
3710 | 2 | 2 |
3023 | 2 | 2 |
3004 | 2 | 2 |
3003 | 2 | 1 |
3039 | 2 | 1 |
3023 | 71 | 1 |
4079 | 0 | 1 |
3829c01 | 0 | 1 |
4488 | 0 | 1 |
32064a | 0 | 2 |
3705 | 0 | 1 |
44293c01 | 71 | 2 |
55981c01 | 4 | 2 |
| total pieces | 22 | |
#0014 rolling-cart hard CaSS 0
| part_num | color_id | qty |
|---|---|---|
3032 | 72 | 1 |
3666 | 4 | 2 |
3023 | 4 | 2 |
4085a | 0 | 1 |
4870 | 71 | 2 |
30027a | 15 | 4 |
30028 | 0 | 4 |
| total pieces | 16 | |
#0015 drawbridge hard CaSS 1
| part_num | color_id | qty |
|---|---|---|
3034 | 72 | 1 |
3004 | 71 | 20 |
2429 | 71 | 1 |
2430 | 70 | 1 |
3032 | 70 | 1 |
| total pieces | 24 | |
#0016 slatted-bench hard CaSS 0
| part_num | color_id | qty |
|---|---|---|
60479 | 70 | 4 |
60479 | 308 | 4 |
3031 | 72 | 3 |
3010 | 0 | 8 |
3039 | 0 | 2 |
| total pieces | 21 | |
#0017 small-house DNC
timeout after 480000ms
#0018 small-lighthouse stretch CaSS 0
| part_num | color_id | qty |
|---|---|---|
3958 | 71 | 2 |
3958 | 0 | 1 |
92947 | 4 | 6 |
92947 | 15 | 6 |
3633 | 0 | 4 |
4864a | 47 | 4 |
3031 | 0 | 1 |
3942a | 0 | 1 |
| total pieces | 25 | |
#0019 garden-shed DNC
timeout after 480000ms
#0020 fountain-with-pool stretch CaSS 0
| part_num | color_id | qty |
|---|---|---|
3033 | 71 | 1 |
3030 | 71 | 1 |
6111 | 72 | 2 |
3008 | 72 | 2 |
3069b | 41 | 30 |
92947 | 71 | 4 |
3960 | 71 | 1 |
3062b | 41 | 4 |
| total pieces | 45 | |
#0021 grand-bridge expert CaSS 0
| part_num | color_id | qty |
|---|---|---|
3020 | 71 | 12 |
3031 | 71 | 4 |
3022 | 71 | 3 |
3003 | 72 | 15 |
3062a | 0 | 20 |
| total pieces | 54 | |
#0022 manor-house expert CaSS 0
| part_num | color_id | qty |
|---|---|---|
91405 | 2 | 1 |
91405 | 71 | 1 |
91405 | 4 | 1 |
3010 | 15 | 68 |
3004 | 15 | 30 |
3005 | 15 | 12 |
30179 | 15 | 1 |
3644 | 4 | 1 |
60592 | 15 | 4 |
60601 | 40 | 4 |
3039 | 4 | 32 |
| total pieces | 155 | |
#0023 clock-tower expert CaSS 0
| part_num | color_id | qty |
|---|---|---|
41539 | 71 | 1 |
3031 | 71 | 1 |
92947 | 71 | 18 |
3005 | 47 | 4 |
3942a | 4 | 1 |
| total pieces | 25 | |
#0024 grand-fountain DNC
adapter error: fetch failed
#0025 mosaic-smiley DNC
adapter error: fetch failed
#0026 voxel-tree DNC
adapter error: fetch failed
#0027 grand-mosaic-sunset DNC
timeout after 480000ms
#0028 mega-voxel-rocket DNC
adapter error: fetch failed
#0029 colossal-mosaic-grid DNC
adapter error: fetch failed
Run metadata
| Submission ID | b94214a4-e987-4f1f-b65a-329839ff5f7b |
|---|---|
| Protocol | scaffold-assembly-v1 |
| Protocol hash | sha256:394f2d859fafd42d0ac144c5f251e68ea2e7f4c06da48bf733e1c2505055ee9c |
| Run at | 2026-06-09T19:27:47.869Z |
| Total duration | 188m27s |
| Adapter health | ok |