Yuanhao Feng

Back

The first article covered the DSL instruction set and the Grid data model. This one picks up where that left off: how the model selects and arranges DSL ops, and how the system keeps the process under control.

CubeMuse’s build pipeline has gone through three generations. Each redesign was driven by concrete failures in the previous one.

v1: One-Shot Generation#

The first approach was straightforward. The user typed a prompt; one LLM call produced a complete DSL program. The engine extracted JSON from the response, ran auto-repair and version migration, then validated syntax and scale. If validation passed, the program was executed and written to the Grid. On failure, diagnostics were fed back to the model for a second attempt, with two calls allowed in total.

A single response had to handle architectural design, spatial layout, correct coordinates, and valid syntax all at once. With everything packed into one shot, output tended toward simple structures. Longer programs drifted from earlier constraints, and spatial reasoning often failed, particularly when aligning coordinates.

v1 could produce simple builds, but complex structures were out of reach for a single generation pass.

v2: Staged Pipeline#

v2 split the concentrated workload of v1 across multiple stages, each responsible for one type of decision.

The pipeline ran in five steps:

  1. S0: Design brief. A stronger model read the user’s request and produced a DesignBrief: style direction, material palette, and an estimated volume envelope.
  2. S1: Layout planning. The same stronger model turned the brief into a layout IR (intermediate representation), specifying volumes, relative positions, and connectivity requirements for each section.
  3. Layout resolution. Code resolved relative positions to absolute bounding boxes, checked collision and connectivity, and generated a skeleton of rooms, doorways, and roofs.
  4. S2: Component selection. A faster Flash-tier model chose components and their parameters for the existing skeleton, assigning each to a semantic slot. Placement coordinates were computed by code. If this step failed, the skeleton alone was still usable.
  5. S3: Detail pass. Code appended seeded detail ops to add plinths, sills, and cornices without calling a model. The same seed always produced the same result.

This was the design-time model assignment. At runtime, the pipeline could also be configured to use a single model throughout.

The pipeline handled far more complex structures than v1, but each stage could only move forward. Once S1 locked in a layout, later steps could not revisit it. The skeleton generator was oriented toward buildings, and there was no mechanism for the system to inspect the current work and loop back. When a skeleton was off, layering components and details on top did not fix it.

v1 One-shot generation One call writes the whole DSL program, then validation catches errors Prompt LLM writes it all design, layout, coords, syntax Validate Grid On failure: feed diagnostics back and retry (max 1 round) Bottleneck One call bears all the cognitive load: design, layout, coordinates, syntax v2 Fixed pipeline Staged compilation: the LLM makes only semantic decisions, deterministic code does the geometry S0 Brief strong-tier LLM (configurable) S1 Layout strong-tier LLM (configurable) Resolve layout code only coordinate skeleton S2 Components Flash-tier LLM (configurable) S3 Details code only, 0 tokens detail op Grid One-way flow, no going back Bottleneck: rigid stages. Later steps cannot revise earlier ones, so a wrong skeleton dooms the rest. Skeletons assume a building layout. v3 Bounded tool loop The model picks its own next step from a toolbox, under a hard budget Budget boundary (build steps / tokens / wall-clock / total calls) LLM decides tool + arguments Execute tools + compile + Grid Observe text + vision Toolbox Build add_ops replace_group remove_group Recon search_library get_entry inspect look: needs a vision model configured Finish finish Same model, same 12 requests: v2 returned 9 programs, v3 returned 11. One run each; visual quality not evaluated.

v3: Bounded Tool Loop#

v3 dropped the fixed stage sequence. Instead, the model received a set of tools and decided what to do on each turn. After the server executed a tool call, the observation was appended to the conversation, and subsequent turns could adjust course based on what had already been built.

A typical build session: the model called search_library to browse reference pieces, get_entry to read one in detail, then add_ops to start building. After a few construction steps it ran inspect to check a structure summary and lint report, noticed the roof proportions were off, and used replace_group to swap the roof section by its named handle. With a vision model configured, it could also call look to generate four-angle screenshots and get a visual description. Finally it called finish to end the session.

Eight Tools#

The tools fall into three groups:

Build tools. add_ops appends a batch of ops to the current program and can add or override material roles at the same time. replace_group swaps a top-level op by its id. Named groups provide editing handles for local structures like roofs or entrances. remove_group deletes a top-level op by id. If removing it would break downstream anchor references, the entire candidate change is rejected.

Query and inspection tools. search_library returns summaries of matching reference entries; get_entry fetches the full text by id, so the model only expands what it is interested in. inspect returns a structure summary, a static-analysis (lint) report, and on-demand ASCII slices for checking layer heights, block distributions, and so on. look renders four-angle images of the current build and sends them to a vision model for description; it is only available when vision is successfully configured.

Termination tool. finish ends the current build. It requires a non-empty work product but does not impose a quality-score gate. When the model considers a build complete is guided primarily by the system prompt.

Candidate Validation and Commit#

After the model selects a tool and its arguments, the build session, BuildSession, validates syntax and scale on a candidate program, executes it, and checks that the result is non-empty. Only when all checks pass does the new state replace the current one. If a modification is rejected, the last accepted state is preserved. Once a valid build has been committed at least once, subsequent failures cannot destroy it.

If the budget is exhausted or consecutive errors trigger a stop condition, the system wraps up and returns the most recent valid build. If no non-empty build was ever committed, the session returns a failure.

Comparison Data#

To compare the two pipeline generations, a single model ran v2 and v3 against the same 12 prompts, covering buildings, trees, a ship, and pixel art. v2 successfully returned an executable program for 9 of the 12; v3 for 11. These numbers measure whether a program was delivered at all, not visual quality, and each prompt was run only once.

The Castle Case: Scale and Budget Limits#

One castle generation log exposed a flaw in the original budget rules: query and inspection actions consumed build steps, so by the time the model discovered problems, it had almost no room to fix them.

At the time, every tool call cost one step, with a cap of 12. Roughly 7 went to construction, 3 to reference queries (search_library and get_entry), 1 to a look check, and 1 was rejected because a group child was missing the required op field.

The model found a watchtower-small reference but ended up generating the tower itself rather than placing the component with place_component. Between queries, visual inspection, and the rejected call, 5 of the 12 calls, or 41.67% of the budget, added no blocks to the build.

look did not run until call 10, leaving only 2 calls. The visual diagnosis flagged the build as too small, resembling a house rather than a castle, and lacking a gatehouse and crenellations. But the build envelope was fixed at 48×48×48, and with so few calls remaining, the model had no practical way to rework the structure.

The final build measured 31×20×34 with 4,681 blocks. The epic-castle reference the model had retrieved was 103×47×128 with 48,696 blocks, roughly ten times larger by block count.

Castle failure trace: 5 of 12 calls produced no geometry Real trace from before the budget revision. Under the old rules every call cost a step, capped at 12. The 12 calls, grouped by action type Build 7 Query 3 look 1 Reject 1 ~7 build turns: add_ops / replace_group etc. 3 queries: found watchtower-small, yet built its own tower 1 look: 4-view screenshots + vision model description 1 rejected: a group child lacked op; whole step REJECTED look came too late calls 1–9 call 10 only 2 left Right diagnosis: too small, house-like, no gate or crenellations but the envelope was locked at 48³, so it could not act on it The build had about a tenth of the reference’s blocks This build 31x20x34, 4,681 blocks epic-castle 48,696 blocks The reference it retrieved itself, 103x47x128 (reference card reading at the time). Main cause: the prompt hard-coded a 48³ building envelope (later fixed by ScalePlan). The same 12 calls under the new M-tier rules illustrative; actual runs vary Old rules steps 12/12 New rules (M) build 7/14 recon 4/6 rejected 1/6 Recon and rejects: no build steps, still tokens and time

After this session, the budget rules were revised in two ways: query and inspection actions received their own allowance that did not consume build steps, and a ScalePlan was introduced to allocate resources according to the expected size of each request.

Scale Tiers and Budget Control#

Tool Call Accounting#

The revised rules first check whether a call errored, then distinguish build actions from query and inspection actions. A successful build action and finish each cost one build step. Query and inspection actions draw from a separate allowance; within that allowance they cost zero build steps, and only count against build steps once the allowance is exceeded. Rejected calls have their own 6-call allowance, with the same overflow rule. If tools return errors three times in a row, the loop stops; one successful return resets the consecutive-error counter.

This lets the model query references and inspect its work without eating into construction budget. However, query and inspection actions within their allowance still consume tokens and wall-clock time, so the system also enforces token usage, elapsed time, and total call count.

Four Caps#

Build steps, token usage, elapsed time, and total call count each have an independent ceiling. Hitting any one of them stops the loop.

  • Build steps: computed from the tier and category, capped at 32.
  • Tokens: cumulative promptTokens + completionTokens of the build model, including query, inspection, and rejected turns. The vision model’s token usage from look goes into a separate ledger.
  • Elapsed time: defaults to 240 seconds, configurable.
  • Total calls: bounds the number of loop iterations. Total call cap = effective build steps + query and inspection allowance + 6.

Thresholds are checked after a full step completes; the current call may exceed a threshold, but the next turn will not start.

The loop also issues two nudges: a retrieval reminder after the first successful build if no search has been run, and a visual-check reminder when half the build steps are spent without a look.

Budget caps Four caps are checked after every step and any one stops the loop. The current call may overshoot a cap. Step accounting: three kinds of action Build actions add_ops / replace_group / remove_group / finish cost 1 build step each Recon actions search_library / get_entry / inspect / look free up to S 4 / M 6 / L 8 / XL 10, then cost build steps Rejected actions failed validation / malformed / single op too large first 6 free, then cost build steps; 3 in a row stops the loop Four caps (M tier, build steps hit first) Steps 14/14 hit Tokens 118k/150k Time 85s/240s Calls 19/26 Call cap = steps + recon + retry allowance = 14 + 6 + 6 19 calls = 14 build + 4 recon + 1 rejected; vision logged apart Recon and rejected actions still spend tokens and time. The token, time and call caps keep the worst case bounded. Per-step flow LLM picks action tool + arguments Deterministic run commits only if valid Count budget build / recon / rejected Cap check No cap hit: next step Stop & deliver return snapshot finish / 3 rejects / tokens time / steps / calls Safety net: after any successful commit, a budget stop or 3 straight rejects returns the last valid state. With none, the run fails.

ScalePlan Tiers#

“Build a chair” and “build a castle” need different resources. ScalePlan assigns a scale tier to each request and derives a target box defining the expected dimensions. The tier uses the user’s explicit choice first; if the user did not specify, it falls back to a classification result; if neither is available, it defaults to M.

The table below shows baseline values for the building category with no explicit overrides:

TierTarget BoxBuild StepsBuild TokensQuery and Inspection AllowanceLook AllowanceLoop Call Cap
S24×24×2410150,0004220
M48×48×4814150,0006226
L84×84×8422250,0008336
XL120×120×12030350,00010446

Effective build steps = min(round(tier base × category budgetScale), 32). Loop call cap = effective build steps + query and inspection allowance + 6. The target box is computed as 48 × tier multiplier, rounded per axis.

The target box is used in prompts and for evaluating build dimensions. Before execution, checkScale validates the volume declared by each individual op, rejecting any that exceeds the limit. If the overall build exceeds the hard ceiling on any axis, lint emits a warning.

Evaluation Results and Limitations#

Block Count Changes Across the Same Prompts#

To measure the effect of the scale and budget changes, both test runs used the v3 pipeline, the mimo-v2.5-pro model, and the tool_calls invocation method, with each prompt run once. Across the 11 prompts that succeeded in both runs, the average block count rose from 1,535 to 14,982, roughly 9.76 times the original count.

Before the adjustments, 11 of 13 prompts returned a program on the first run; after re-running the 2 failures, 12 succeeded in total. The post-adjustment batch was labeled eval v4 (v4 is the evaluation number; the pipeline is still v3). This batch added two L-tier prompts, a castle and a cathedral, for 15 prompts total, all of which returned a program. However, only 1 was ended by the model calling finish; the rest hit a budget cap: 8 exhausted tokens, 4 hit the time limit, and 2 used up their build steps.

Among the 11 prompts used for comparison, the post-adjustment oak tree contained 92,790 blocks, accounting for 56.31% of the group’s total block count. The average is therefore heavily influenced by that single sample.

Limits of the Comparison#

Between the two runs, the time limit and transport reliability were also adjusted alongside scale and budget rules. The block-count increase confirms that generation scale grew, but it does not isolate how much each change contributed, nor does it say anything about visual quality. Those questions require controlled experiments with one variable at a time.

Follow-Up Revision and Scoped Edits#

When a user says “make the roof red,” there is no need to rebuild the entire structure. Follow-up requests carry the existing program via revise.programText, with the edit instruction in source:"user" and instruction. When prior work exists, the system defaults to revision mode; the UI also provides an option to start from scratch.

A revision begins by loading the old program into a BuildSession. The model then modifies sections through their named handles. Top-level detail ops are temporarily stripped and restored at the end with the original seed. Revisions inherit the original build’s category, tier, and proportions, but with fewer build steps and a token budget scaled to 0.6× of the original.

If a scope is specified, every candidate commit checks block-by-block whether anything outside the scope has changed. Any out-of-scope modification causes the entire change to be rejected, preventing the model from rewriting walls while it was asked to fix the roof.

Handle-based replacement requires top-level ops to carry an id. Hand-edited content without ids is modified primarily through appended ops. Scoped revision currently passes deterministic tests, but end-to-end validation with a real model completing a full revision flow has not yet been recorded.

Closing#

From v1 to v2, the main problem was that a single generation pass carried too many responsibilities. By v3, the focus shifted to letting the model inspect and adjust what it had already built. Tools provide the means to modify; budget rules bound how long the process can run.

Current evaluations are not sufficient to quantify quality improvements. Each prompt was run only once, and the average block count is visibly skewed by a single large sample. Future work needs repeated trials with visual quality metrics, and end-to-end validation of the revision flow with a real model.

From One-Shot Generation to Bounded Agents
https://me.yuanhaofeng.com/en/blog/cubemuse-agent-pipeline
Author Yuanhao Feng
Published at September 16, 2026