BrickAGI benchmark

models /

claude-opus-4-8-coordinator-1shot scaffold-assembly-v1

Provider: anthropic · Run at 2026-06-08 · Submission b1a9c0de

Build Points 739 / 2,631 available
CaSS 57.4%
CBS 48.2%
Coverage 49%
Cost $0
Tokens (in/out) 0 / 0
Diagnostic row, not ranked. legacy task-set provenance

Tier breakdown

The 3-dimension diagnostic matrix for this submission. A tier shows no CBS until the structural prover returns a definitive verdict there.

Tier BOM pass Scope pass Buildability CaSS CBS Coverage n
trivial 100.0% 100.0% 100.0% 100.0% 100.0% 100% 4
easy 100.0% 100.0% 66.7% 100.0% 50.0% 75% 4
medium 44.4% 44.4% 100.0% 44.4% 44.4% 44% 9
hard 57.1% 57.1% 100.0% 57.1% 42.9% 43% 7
stretch 100.0% 100.0% 100.0% 100.0% 100.0% 100% 4
expert 0.0% 0.0% — 0.0% 0.0% 0% 4
master 0.0% 0.0% — 0.0% 0.0% 0% 5

Per-task results

All 37 tasks attempted by this baseline. Click a task to see the same row from every model's perspective.

# Slug Tier CaSS BOM Scope Build Ref sim Iter Tokens Duration Status 3D
0001 flat-plate-5x7 trivial 1 1 1 1 1.000 1 0 0ms pass 3D
0002 flat-plate-5x5 trivial 1 1 1 1 1.000 1 0 0ms pass 3D
0003 staggered-column-2x2 trivial 1 1 1 1 1.000 1 0 0ms pass 3D
0004 hollow-frame-8x8 trivial 1 1 1 1 1.000 1 0 0ms pass 3D
0005 small-table easy 1 1 1 1 0.000 1 0 0ms pass 3D
0006 step-pyramid easy 1 1 1 1 1.000 1 0 0ms pass 3D
0007 simple-gate easy 1 1 1 0 1.000 1 0 0ms pass 3D
0008 picture-frame easy 1 1 1 incon. 0.238 1 0 0ms pass 3D
0009 minifig-chair medium 1 1 1 1 0.833 1 0 0ms pass 3D
0010 bookshelf-3-shelf medium 1 1 1 1 0.056 1 0 0ms pass 3D
0011 simple-bridge medium 1 1 1 1 1.000 1 0 0ms pass 3D
0012 square-lantern medium 1 1 1 1 1.000 1 0 0ms pass 3D
0013 farm-tractor hard 1 1 1 1 0.313 1 0 0ms pass 3D
0014 rolling-cart hard 1 1 1 1 0.429 1 0 0ms pass 3D
0015 drawbridge hard 1 1 1 1 1.000 1 0 0ms pass 3D
0016 slatted-bench hard 1 1 1 incon. 0.438 1 0 0ms pass 3D
0017 small-house stretch 1 1 1 1 0.089 1 0 0ms pass 3D
0018 small-lighthouse stretch 1 1 1 1 0.133 1 0 0ms pass 3D
0019 garden-shed stretch 1 1 1 1 0.011 1 0 0ms pass 3D
0020 fountain-with-pool stretch 1 1 1 1 0.063 1 0 0ms pass 3D
0021 grand-bridge expert 0 0 0 incon. 0.000 1 0 0ms DNC —
0022 manor-house expert 0 0 0 incon. 0.000 1 0 0ms DNC —
0023 clock-tower expert 0 0 0 incon. 0.000 1 0 0ms DNC —
0024 grand-fountain expert 0 0 0 incon. 0.000 1 0 0ms DNC —
0025 mosaic-smiley master 0 0 0 incon. 0.000 1 0 0ms DNC —
0026 voxel-tree master 0 0 0 incon. 0.000 1 0 0ms DNC —
0027 grand-mosaic-sunset master 0 0 0 incon. 0.000 1 0 0ms DNC —
0028 mega-voxel-rocket master 0 0 0 incon. 0.000 1 0 0ms DNC —
0029 colossal-mosaic-grid master 0 0 0 incon. 0.000 1 0 0ms DNC —
0030 bonded-plate-7x7 medium 0 0 0 incon. 0.000 1 0 0ms DNC —
0031 bonded-platform-9x5 medium 0 0 0 incon. 0.000 1 0 0ms DNC —
0032 frame-6x6 medium 0 0 0 incon. 0.000 1 0 0ms DNC —
0033 grand-table-6x6 medium 0 0 0 incon. 0.000 1 0 0ms DNC —
0034 great-pyramid-8x8 medium 0 0 0 incon. 0.000 1 0 0ms DNC —
0035 wide-gate hard 0 0 0 incon. 0.000 1 0 0ms DNC —
0036 throne hard 0 0 0 incon. 0.000 1 0 0ms DNC —
0037 tall-column-2x2 hard 0 0 0 incon. 0.000 1 0 0ms DNC —

Submitted BOMs

The final BOM the model emitted on each task it attempted.

#0001 flat-plate-5x7 trivial CaSS 1

flat-plate-5x7
part_num color_id qty
3032 4 2
3666 4 2
3710 4 2
3024 4 2
total pieces 8

#0002 flat-plate-5x5 trivial CaSS 1

flat-plate-5x5-bonded
part_num color_id qty
78329 4 10
total pieces 10

#0003 staggered-column-2x2 trivial CaSS 1

staggered-column-2x2
part_num color_id qty
3004 15 26
total pieces 26

#0004 hollow-frame-8x8 trivial CaSS 1

hollow-frame-8x8
part_num color_id qty
3460 15 2
3666 15 2
3023 15 4
total pieces 8

#0005 small-table easy CaSS 1

small-table
part_num color_id qty
3005 70 12
3031 70 1
total pieces 13

#0006 step-pyramid easy CaSS 1

Tan Step Pyramid (6x6 / 4x4 / 2x2)
part_num color_id qty
3032 19 1
3795 19 1
3031 19 1
3022 19 1
total pieces 4

#0007 simple-gate easy CaSS 1

Simple Gate (two 5-brick pillars + 1x6 lintel)
part_num color_id qty
3005 15 10
3666 15 1
total pieces 11

#0008 picture-frame easy CaSS 1

Standing Picture Frame (brown frame, white picture, brown foot)
part_num color_id qty
3033 70 1
3036 70 1
3009 70 8
3004 70 3
3032 15 1
total pieces 14

#0009 minifig-chair medium CaSS 1

minifig-chair
part_num color_id qty
3005 4 8
3022 4 1
3004 4 1
total pieces 10

#0010 bookshelf-3-shelf medium CaSS 1

bookshelf-3-shelf
part_num color_id qty
3622 70 18
3795 70 3
3623 70 6
total pieces 27

#0011 simple-bridge medium CaSS 1

simple-bridge
part_num color_id qty
3003 71 8
3035 71 1
3020 71 2
3032 71 2
3005 71 12
total pieces 25

#0012 square-lantern medium CaSS 1

Square Lantern (4x4, 5-brick walls, windows)
part_num color_id qty
3031 72 2
3010 72 8
3005 72 4
total pieces 14

#0013 farm-tractor hard CaSS 1

Farm Tractor (tractor_basic)
part_num color_id qty
3035 2 1
30157 2 2
56145 0 2
92402 0 2
4624 0 2
3641 0 2
3003 2 2
4079 2 1
3062b 2 1
3460 2 2
3666 2 2
3023 2 2
total pieces 21

#0014 rolling-cart hard CaSS 1

Rolling Cart (open-top, 4 rubber-tyred wheels, front bar hitch)
part_num color_id qty
3032 70 1
30157 70 2
4624 0 4
3641 0 4
3010 70 8
4623 70 1
total pieces 20

#0015 drawbridge hard CaSS 1

drawbridge-basic
part_num color_id qty
3795 71 1
3004 71 10
3937 71 1
3830 70 1
3710 70 1
3020 70 1
total pieces 15

#0016 slatted-bench hard CaSS 1

Slatted Park Bench (0016)
part_num color_id qty
3010 70 8
3022 70 10
3031 70 3
3710 70 6
3710 308 6
60479 70 4
3024 70 19
3062b 308 4
total pieces 60

#0017 small-house stretch CaSS 1

0017-small-house
part_num color_id qty
92438 15 2
3005 15 136
60596 70 1
60616 70 1
60593 15 1
3037 4 8
3039 4 4
total pieces 153

#0018 small-lighthouse stretch CaSS 1

Small LEGO Lighthouse (0018)
part_num color_id qty
3032 71 2
3795 71 2
3795 72 54
3022 72 54
3031 71 1
3031 4 1
3941 4 6
3941 15 6
3005 47 4
3062b 71 4
3024 15 4
6233 4 1
total pieces 139

#0019 garden-shed stretch CaSS 1

Garden Shed (single-pitch, dark tan walls, dark grey roof)
part_num color_id qty
41539 72 1
92438 72 1
60596 70 1
60616 70 1
60593 69 1
6065 28 1
3005 69 144
total pieces 150

#0020 fountain-with-pool stretch CaSS 1

Garden Fountain with Pool (10x10)
part_num color_id qty
3030 71 1
3033 71 1
3004 71 54
3460 71 6
3623 71 4
3460 43 30
3623 43 20
3941 71 4
4032b 71 1
3062b 43 6
total pieces 127

#0021 grand-bridge DNC

no solution produced

#0022 manor-house DNC

no solution produced

#0023 clock-tower DNC

no solution produced

#0024 grand-fountain DNC

no solution produced

#0025 mosaic-smiley DNC

no solution produced

#0026 voxel-tree DNC

no solution produced

#0027 grand-mosaic-sunset DNC

no solution produced

#0028 mega-voxel-rocket DNC

no solution produced

#0029 colossal-mosaic-grid DNC

no solution produced

#0030 bonded-plate-7x7 DNC

no solution produced

#0031 bonded-platform-9x5 DNC

no solution produced

#0032 frame-6x6 DNC

no solution produced

#0033 grand-table-6x6 DNC

no solution produced

#0034 great-pyramid-8x8 DNC

no solution produced

#0035 wide-gate DNC

no solution produced

#0036 throne DNC

no solution produced

#0037 tall-column-2x2 DNC

no solution produced

Run metadata

Submission IDb1a9c0de-1f42-4ddd-8aaa-coord1shot14
Protocolscaffold-assembly-v1
Protocol hashsha256:394f2d859fafd42d0ac144c5f251e68ea2e7f4c06da48bf733e1c2505055ee9c
Run at2026-06-08T00:00:00Z
Total duration0ms
Adapter healthn/a