← Dev Log
BallOrcs11 min read

The canyon gets a finale, the game gets a voice, and the crash was in the cache

BallOrcs has sound. Rolling, jumping, landing, picking up an orb — and a piece of music for each level, fading into the next one as you move between them.

In plain words

BallOrcs has sound. Rolling, jumping, landing, picking up an orb — and a piece of music for each level, fading into the next one as you move between them.

Level 3 has an ending now. The platform at the top of the canyon has grown into a walled run past five rows of erupting geysers, so the last level of the slice is no longer the quickest one to finish. The air changes as you climb it, too: dead and grey at the bottom, clearing into blue sky at the two moments you ride a wind column up.

Winning a level gets a celebration rather than a panel appearing. And a crash on Windows that had been shipping for weeks turned out not to be in the game at all.

One of the 120 reference screenshots the build compares itself against every run. All of them were re-recorded this week, because halving the number of shadow passes moved every shadow edge in the game. It also shows the two bits of text that are on screen for the entire game — the orb counter and the control hint — which until this week ignored the text-size slider.

Under the hood

  • The Windows crash was a poisoned object cache (a558b9c, #115). game.exe had been dying on launch with 0xC0000005 since before the issue was filed. Not our C++, not godot-cpp, not the engine pin, not the renderer, not Sentry. Proven rather than inferred: same commit, same submodule pins, byte-identical extension_api.json, same pinned toolchain container, one variable — whether the shared SCons cache is mounted. CI's published artifact and a cache-built DLL are both 2,587,136 bytes and both crash; the same source built without the cache is 2,591,744 bytes and boots, plays, and travels 50.1 m. The pool held objects back to 2026-07-10 — seven weeks and several pin bumps in one flat directory — against an SConstruct comment asserting that "keys hash the full command line and inputs, so caching does not affect determinism". That assumption is what the bug was made of. Two strong theories died on the way: release-vs-debug (a fresh release DLL works) and a stale extension_api.json (pulled CI's actual file off the runner's mount and diffed it — identical).
  • And the gate that would have caught it. No job in the pipeline had ever executed a Windows binary — playtest needs: build:linux and soaks the Linux artifact — so a Windows-only crash on load shipped green for weeks. smoke:windows now launches game.exe --headless --quit on the Windows runner and asserts exit 0, blocking, which it could not have been while the crash was live. It uses Start-Process -Wait deliberately: a GUI-subsystem binary launched bare detaches, the shell returns 0 immediately, and the job passes without having waited for anything. Follow-ups namespaced the engine's cache too (b43d000, keyed on the pinned engine commit — the engine is the more dangerous half, since a subtly miscompiled engine object would not crash loudly the way #115 did) and purged the 11 GB flat pool, and stamped the three reusable build artifacts with the revision that produced them (63a2f56), because "exists" and "was built from what we are building now" are different questions.
  • Audio, synthesised from scratch (18663dd, #46). Four cues on the SFX bus — a rolling bed whose volume and pitch follow ground speed, jump, land, orb pickup — plus one looping track per level on the Music bus, crossfaded on scene change. No third-party audio: everything is sine partials, seeded shaped noise and envelopes, and tools/audio/generate_audio.py is committed this time (#72 generated its blips with a script that was not, so this pass started from nothing). Loops are seamless by construction — every partial is a whole number of cycles across the buffer, so the waveform meets itself at the wrap; roll_loop is imported uncompressed because it repeats every half second and QOA's block boundary at the wrap is audible. No C++ change: PlayerBall emits only respawned, so game_sfx.gd polls is_grounded() and reads the velocity from the frame before contact resolved — by the time the ball reports grounded, the collision has already cancelled its fall speed. The new suite caught three things, including a loop check that decoded QOA codec bytes as PCM, reported a 28,529-unit click in a sample-exact loop, and then passed on the three files it was still misreading.
  • The L3 finale is a hazard gauntlet (08b01c9, #37) — the W9 cut-line review cut the mini-boss and kept this, assembled from soak-tested pieces with no new hazard type. The Fount platform grows 12 m → 40 m, walled to an 8 m lane (on the bare platform every hazard could be strolled round at x=6). Five geyser rows alternate — a pair at x=±3 leaving the 4 m centre, a single at x=0 leaving two 3 m side lanes — so the route is centre → side → centre → side → centre. Orbs sit on the lane the required route does not take, so the greedy line is the risky one. Lap time 10.0 → 11.7 s, which stops L3 tying L1 for quickest; it has not reached L2's 17.5 s and does not claim to.
  • A harness bug hid behind it for two soak cycles. The first full soak came back 0 wins, 20 respawns, the bot walking backwards off a ledge twenty times. The cause was nodes.sort_custom(...String(a.name) < String(b.name)) — waypoints sorted lexicographically, so Waypoint10 came before Waypoint2 and the route ran 1 → 10 → 11 → 2. Every previous level had four waypoints or fewer, where the two orders agree. Diagnosed from the bot's path trace, not its report: the report said goal_not_reached, which is true and useless; the path said it turned round at z=-77.4.
  • L3 tuning: cap the columns with height, not force (1a5427e, #39). Over half an L3 run was spent in the air — 95 s of 180, 48.8% of samples at Fount height — because both updraft columns launched the ball ~4 m above the ledge it was aiming at. Reducing lift_force does not work: 32 makes the level uncompletable, 36 gives 5 wins and 43 respawns, and 38 is the first reliable value, because the force is not setting the height, it is beating the horizontal push that carries the ball out of the column. Column height 8 m → 4 m with lift unchanged: wins 17 → 18 on every seed, 0 respawns throughout, airborne 53.1% → 37.5%, Fount-height occupancy 48.8% → 26.9%.
  • The winds return as you climb (0913a20, #38). Three stages, two triggers, keyed off UpdraftFan.body_lifted rather than a generic progress event — the columns are the only way up, one per tier, so the air returns at the two moments the player makes the wind work for them. STILL (fog 0.0055, drained grey) → RISING (0.0034) → ALIVE (0.0018, clear blue), forward-only and idempotent, because body_lifted fires on every entry and the player re-enters on every lap and every respawn. The authored environment is STILL, so a run that never reaches a column still looks deliberate. The Hollow King sits 196 m down the canyon where the starting fog hides most of him and resolves as the air clears.
  • An eruption now burns what is standing in it (d05bb24, #114). flame_geyser only ever called respawn() from body_entered, so rolling in during the erupt window killed you and standing still in the column did not — which inverts what the telegraph is for. The transition into ERUPT now sweeps get_overlapping_bodies(). tests/hazards pins both halves, because the obvious fix (respawn everything overlapping, every frame) would pass a test that only checks the first.
  • Shadows were 100 of Level 1's 130 draw calls (faaafca, #79). Ablating one variable at a time: stock 130, glow off 130, shadows off 30. Thirty objects producing ~120 shadow draws is the signature of 4-split PSSM, and no level set directional_shadow_mode, so all three inherited Godot's default. 2 splits rather than orthogonal — 76 draws against 66, and orthogonal is visibly stair-stepped throughout, so it buys nothing. Draw calls p95: L1 143 → 82 (-43%), L2 95 → 61 (-36%), L3 113 → 70 (-38%). Stated plainly in the commit: the halving is a count and transfers to any GPU, but llvmpipe renders on the CPU, so this demonstrates 60fps nowhere. The 120-shot baseline was re-recorded from the pipeline that produced the change (fab1f23) — 115 of 120 anchors moved, 50 past the diff threshold.
  • Forward+ is the default again (e20c379, #69). The gl_compatibility default was the L1 stopgap for an engine that could not compile Forward+ shaders at runtime; the current pinned build carries glslang. Desktop defaults to forward_plus, mobile to Forward Mobile, web stays gl_compatibility (the only web renderer), each degrading — verified on the Iris Xe reference laptop as Vulkan, then D3D12 with Vulkan hidden, then GL. The camera probe is pinned to gl_compatibility, because the playtest container's mesa ships lavapipe and an unpinned probe would follow the new default onto Forward+ and mismatch all 120 anchors at once.
  • The HUD readability problem was wiring, not styling (c4490e5, #75). "Prompt text size" has existed since #78 and scaled the tutorial prompt; the orb counter and the control hint — the two pieces of text on screen for the whole game — ignored it, because options_panel.gd applied the setting by calling each consumer by hand and the HUD was never on the list. SettingsManager now emits text_scale_changed and consumers subscribe. The base font size is captured once and always multiplied from the authored value, or every slider nudge would compound and ratchet the HUD upward across a session. The results panel — the screen every player sees on every level win — had never been tested, because the bot deliberately skips it; it turned out to be untestable at all, since menu_util.gd named an autoload directly and bare autoload identifiers resolve at parse time, so anything preloading it failed to compile under --script.
  • A celebration, and the global it plays with (295ec81, #76). Light burst, orb fountain, jingle, and a slow-motion beat between touching the Fount and the results panel, skippable from the first frame with jump, ability or accept. Engine.time_scale is the hazard: nothing else in the project touches it, it is global, and left low the whole game runs in treacle with no menu setting a player could use to recover. Restored on finish, on skip, and in _exit_tree; mutation-tested by dropping that last one, which fails with time_scale left at 0.817518 after being freed mid-beat. The bot does not see any of it, on the same --playtest branch that already skipped the results panel — proof rather than assertion, the soak after the change reports 7 wins, 0 respawns, 620.9 m, byte-identical to the run before it.
  • The ability effects had been invisible to every real character (a01fa8d, #23). All three fed back through material_override on PlayerBall/Mesh, and character_visual.gd hides that node whenever the character has a skin — which every character made in the barber shop has. It survived because the playtest bot boots with no saved character: the soak and a player were looking at different balls, and only one of them had a ball to tint. Worse, releasing a tint with material_override = null deleted the character's material, since appearance.gd tints component meshes through the same property. Verification measures pixels, because every state-level assertion available would have passed on the bug above.
  • Ambient particles, and what the perf oracle cannot tell us (e8f3031, #71). Landing ring scaled by impact, roll trail following ground speed, speed lines near top speed — adopting level1.tscn's authored RollDust/LandDust emitters where present rather than replacing them. The render probe had to be fixed first: holding the ball still measured the trail at 40 changed pixels skinless against 399 with the mesh hidden, a 10x gap produced entirely by the ball sitting on top of its own ground dust. Real roll dust is left behind a moving ball. Honest caveat recorded: the three emitters add zero measured draw calls because the camera probe takes no input, so the ball never rolls, lands or reaches top speed and all three sit at emitting = false for the run.
  • The robot playtesters grew four new senses. A coverage heatmap (ffaafd5, #81) drawn against the level's extent rather than the bots' — a map scaled to where the bots went can never show the corner they never reached — and tracing the segment between samples rather than plotting them, since path.jsonl writes six positions a second and the first version reported 1.6% coverage for a level the bot had run end to end. UI journeys (77e86b7, #85) that assert visibility, focus and settings after each press, deliberately not position, size or colour, because a journey that fails on a restyle is a journey people delete. Frontier mode (b32a7fd, #84), which commits to any 4 m voxel no bot has visited and taboos the ones that do not pan out — +17% and +10% coverage over the monkey on L3, and a stated tie-or-loss on L1, where a nearly saturated linear course has no pockets to be curious about. And a perf oracle whose tick_ms gate now compares against a nine-run envelope (eef28a3, #120), because nine CI runs of functionally identical builds spanned 0.246–0.353 ms — 43.5% peak-to-peak against a +25% gate, so it fired on about a third of runs with no code change.
  • 81% of the web download is one engine artifact (57ac947, #52). Measured against the real pipeline artifact: 53.6 MB on disk, 12.9 MB over the wire, of which index.side.wasm is 10.47 MB gzipped and index.pck — the game's own content, and everything the project has been optimising — is 4.3%. It is not unstripped; the 44 MB is the cost of dlink_enabled=yes, which GDExtension games require. Applied: stop shipping a 131 KB source map, via a platform_post_export hook, after measuring that the obvious exclude_filter fix was a no-op (packs exported with and without it are byte-identical at 823,508).
  • Smaller things that were still wrong. Escape inside the Options panel tore down the entire pause menu and unpaused the game, losing your place two changes into rebinding keys — it now backs out one level per press (294d756, #118). Build manifests recorded run_id: "local" on real CI jobs, because in_container forwards a fixed env list that never included CI_PIPELINE_ID (288b7ac, #116). And the perf report's headline printed the literal p95 for every regression while the table directly above it said "gated on p50" — the number was right and the sentence was wrong, which is the more expensive kind, since the headline is what gets pasted into an issue (daf7832).
#audio#levels#art#rendering#testing#ci