BrickAGI benchmark

models /

gpt-5.5 raw-assembly-v1

Provider: openai · Run at 2026-06-10 · Submission a90dbf5b

Build Points 76 / 2,631 available
CaSS 28.7%
CBS 14.3%
Coverage 32%
Cost $7.91
Tokens (in/out) 2.47M / 173.6k

Tier breakdown

The 3-dimension diagnostic matrix for this submission. A tier shows no CBS until the structural prover returns a definitive verdict there.

Tier BOM pass Scope pass Buildability CaSS CBS Coverage n
trivial 100.0% 100.0% 66.7% 100.0% 50.0% 75% 4
easy 75.0% 50.0% 66.7% 50.0% 50.0% 75% 4
medium 44.4% 22.2% 0.0% 22.2% 0.0% 44% 9
hard 42.9% 28.6% 0.0% 28.6% 0.0% 14% 7
stretch 75.0% 0.0% 0.0% 0.0% 0.0% 25% 4
expert 50.0% 0.0% — 0.0% 0.0% 0% 4
master 0.0% 0.0% — 0.0% 0.0% 0% 5

Per-task results

All 37 tasks attempted by this baseline. Click a task to see the same row from every model's perspective.

# Slug Tier CaSS BOM Scope Build Ref sim Iter Tokens Duration Status 3D
0001 flat-plate-5x7 trivial 1 1 1 1 0.000 3 18.5k 1m35s pass 3D
0002 flat-plate-5x5 trivial 1 1 1 1 1.000 5 29.8k 1m22s pass 3D
0003 staggered-column-2x2 trivial 1 1 1 0 0.000 5 17.6k 43.1s pass 3D
0004 hollow-frame-8x8 trivial 1 1 1 incon. 0.000 4 13.6k 1m40s pass 3D
0005 small-table easy 1 1 1 1 0.000 3 16.4k 43.1s pass 3D
0006 step-pyramid easy 1 1 1 1 0.400 4 9.7k 23.9s pass 3D
0007 simple-gate easy 0 1 0 0 0.000 3 12.1k 30.6s 3D
0008 picture-frame easy 0 0 0 incon. 0.000 9 196.3k 5m25s DNC —
0009 minifig-chair medium 1 1 1 0 0.000 3 25.7k 58.4s pass 3D
0010 bookshelf-3-shelf medium 0 1 0 0 0.000 7 73.5k 3m17s 3D
0011 simple-bridge medium 0 1 0 0 0.286 4 19.2k 1m02s 3D
0012 square-lantern medium 1 1 1 0 0.500 6 67.1k 3m30s pass 3D
0013 farm-tractor hard 1 1 1 incon. 0.000 12 280.4k 5m06s pass 3D
0014 rolling-cart hard 0 1 0 incon. 0.000 10 128.9k 2m58s 3D
0015 drawbridge hard 1 1 1 0 0.000 8 92.8k 3m34s pass 3D
0016 slatted-bench hard 0 0 0 incon. 0.000 8 120.6k 6m11s DNC —
0017 small-house stretch 0 1 0 incon. 0.255 15 330.7k 5m42s 3D
0018 small-lighthouse stretch 0 1 0 0 0.007 10 94.3k 5m35s 3D
0019 garden-shed stretch 0 0 0 incon. 0.000 11 193.3k 5m23s DNC —
0020 fountain-with-pool stretch 0 1 0 incon. 0.011 10 654.0k 5m57s 3D
0021 grand-bridge expert 0 1 0 incon. 0.000 5 40.3k 2m06s 3D
0022 manor-house expert 0 1 0 incon. 0.051 8 207.8k 7m29s 3D
0023 clock-tower expert 0 0 0 incon. 0.000 1 0 368ms DNC —
0024 grand-fountain expert 0 0 0 incon. 0.000 1 0 551ms DNC —
0025 mosaic-smiley master 0 0 0 incon. 0.000 1 0 84ms DNC —
0026 voxel-tree master 0 0 0 incon. 0.000 1 0 77ms DNC —
0027 grand-mosaic-sunset master 0 0 0 incon. 0.000 1 0 256ms DNC —
0028 mega-voxel-rocket master 0 0 0 incon. 0.000 1 0 185ms DNC —
0029 colossal-mosaic-grid master 0 0 0 incon. 0.000 1 0 245ms DNC —
0030 bonded-plate-7x7 medium 0 0 0 incon. 0.000 1 0 314ms DNC —
0031 bonded-platform-9x5 medium 0 0 0 incon. 0.000 1 0 381ms DNC —
0032 frame-6x6 medium 0 0 0 incon. 0.000 1 0 121ms DNC —
0033 grand-table-6x6 medium 0 0 0 incon. 0.000 1 0 625ms DNC —
0034 great-pyramid-8x8 medium 0 0 0 incon. 0.000 1 0 281ms DNC —
0035 wide-gate hard 0 0 0 incon. 0.000 1 0 142ms DNC —
0036 throne hard 0 0 0 incon. 0.000 1 0 204ms DNC —
0037 tall-column-2x2 hard 0 0 0 incon. 0.000 1 0 128ms DNC —

Submitted BOMs

The final BOM the model emitted on each task it attempted.

#0001 flat-plate-5x7 trivial CaSS 1

5x7 white bonded flat surface
part_num color_id qty
3032 15 2
3666 15 1
3710 15 1
3024 15 1
total pieces 5

#0002 flat-plate-5x5 trivial CaSS 1

Flat Red 5x5 Surface
part_num color_id qty
78329 4 10
total pieces 10

#0003 staggered-column-2x2 trivial CaSS 1

Red Staggered 2x2 Column
part_num color_id qty
3004 4 26
total pieces 26

#0004 hollow-frame-8x8 trivial CaSS 1

8x8 Hollow Square Frame
part_num color_id qty
3460 0 2
3666 0 2
2420 0 4
total pieces 8

#0005 small-table easy CaSS 1

Minifig Clearance Table
part_num color_id qty
3031 19 1
3005 70 12
total pieces 13

#0006 step-pyramid easy CaSS 1

Tan Step Pyramid
part_num color_id qty
3958 19 1
3031 19 1
3022 19 1
total pieces 3

#0007 simple-gate easy CaSS 0

Simple Red Gate
part_num color_id qty
3003 4 10
3007 4 1
total pieces 11

#0008 picture-frame DNC

timeout after 300000ms

#0009 minifig-chair medium CaSS 1

Minifigure Chair
part_num color_id qty
3005 70 8
3021 70 1
3004 70 1
total pieces 10

#0010 bookshelf-3-shelf medium CaSS 0

Brown Small Bookshelf
part_num color_id qty
3032 6 3
3010 6 18
total pieces 21

#0011 simple-bridge medium CaSS 0

Small Footbridge
part_num color_id qty
3003 71 8
3029 71 1
6112 71 2
total pieces 11

#0012 square-lantern medium CaSS 1

4x4 Dark Grey Lantern
part_num color_id qty
3031 72 2
3010 72 6
3004 72 6
3005 72 8
total pieces 22

#0013 farm-tractor hard CaSS 1

Small Green Farm Tractor
part_num color_id qty
21445 71 4
72206 14 2
56145 14 2
4267 0 2
3020 2 3
3001 2 1
3004 2 2
3710 2 2
4079 0 1
3829c01 0 1
3024 14 2
4070 0 1
total pieces 23

#0014 rolling-cart hard CaSS 0

Small rolling open cart
part_num color_id qty
3032 71 1
3009 4 2
3004 4 2
3023 71 1
61252 0 1
4488 0 2
56902 71 4
30028 0 4
total pieces 17

#0015 drawbridge hard CaSS 1

Small Castle Drawbridge
part_num color_id qty
41539 72 1
3003 71 10
3032 70 1
4275a 0 2
4276a 0 2
total pieces 16

#0016 slatted-bench DNC

timeout after 300000ms

#0017 small-house stretch CaSS 0

Small White House
part_num color_id qty
3867 2 1
3008 15 15
3004 15 18
30179 15 1
3861b 6 1
3853 15 1
3855 47 1
3037 4 12
total pieces 50

#0018 small-lighthouse stretch CaSS 0

Small Striped Lighthouse
part_num color_id qty
3958 71 3
3022 71 1
92947 4 6
92947 15 6
4864a 47 4
3633 71 4
3942a 4 1
total pieces 25

#0019 garden-shed DNC

timeout after 300000ms

#0020 fountain-with-pool stretch CaSS 0

10x10 Garden Fountain
part_num color_id qty
3033 71 1
3030 71 1
3010 72 8
3004 72 2
98138 41 36
92947 71 4
3960 71 1
total pieces 53

#0021 grand-bridge expert CaSS 0

Grand Bonded 20x4 Bridge
part_num color_id qty
3035 71 4
3031 71 2
3003 72 15
3062a 0 12
total pieces 33

#0022 manor-house expert CaSS 0

Two-Bay White Manor House
part_num color_id qty
91405 19 1
3010 15 70
3004 15 14
3005 15 16
30179 15 1
3644 4 1
60592 15 4
60601 47 4
3032 70 8
3039 70 32
total pieces 151

#0023 clock-tower DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0024 grand-fountain DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0025 mosaic-smiley DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0026 voxel-tree DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0027 grand-mosaic-sunset DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0028 mega-voxel-rocket DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0029 colossal-mosaic-grid DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0030 bonded-plate-7x7 DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0031 bonded-platform-9x5 DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0032 frame-6x6 DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0033 grand-table-6x6 DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0034 great-pyramid-8x8 DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0035 wide-gate DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0036 throne DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

#0037 tall-column-2x2 DNC

adapter error: OpenAI 429: { "error": { "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.", "type": "insufficient_quota", "param": null, "code": "insufficient_quota" } }

Run metadata

Submission IDa90dbf5b-c426-4842-b506-dacdd26b1c1d
Protocolraw-assembly-v1
Protocol hashsha256:ba354b3a8041d12ddf491577fb232f529f67bf54ecc996af73b7d45fb671d3ff
Run at2026-06-10T16:33:36.476Z
Total duration71m15s
Adapter healthok