Files
Project-M/.claude/skills/dots-dev/references/validation-harness.md
T

107 lines
8.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Validation harness — driving Unity Play mode safely from MCP
Consolidates the Play-mode validation lore that used to live only in machine-local memories (screenshot-pause bug,
sleep-poll ban, execute_code traps, headless-run flakiness) plus the proven server-inject harness. Load when running
Phase 6 (verify ladder) or any live diagnosis. Committed + in-repo: this is the cross-machine home of these rules.
## Ground rules (violating these produces MISDIAGNOSES, not just failures)
1. **Never `Thread.Sleep` / sleep-poll inside `execute_code`.** It runs on the main thread — it freezes the entire
sim, and state read "after waiting" is state from a FROZEN world. This manufactured the retracted WasAllReady
"save deadlock" (2026-07-06). Spread observation across separate MCP calls instead — each round-trip gives the
sim hundreds of ms to advance.
2. **The bridge's game-view capture PAUSES the editor** (`EditorApplication.Step`). Check `isPaused` first, unpause
after any bridge capture. Prefer `ScreenCapture.CaptureScreenshot(path)` inside `execute_code` — async, lands at
frame end; Read the PNG on the NEXT MCP call (the round-trip covers it).
3. **An unfocused editor** throttles Edit mode to near-idle (MCP looks hung; test INIT stalls — pass
`init_timeout=120000`) and can leave a STALE Burst binary after Bursted query-set edits. Ask the operator to focus
Unity for compile/test/Burst-heavy phases. `Application.runInBackground` only helps in Play mode.
4. **Play-enter resets static presentation configs** via `[RuntimeInitializeOnLoadMethod]` — a value poked via
`execute_code` does not survive the next play-enter. Conversely: if a whole presentation slice is silent, FIRST
probe its static config (`return ProjectM.Client.XFeelConfig.Enabled;`) before theorizing structurally — the
2026-07-07 harvest-feedback bug was found by that one read after a plausible transform-hierarchy hypothesis
proved wrong.
## execute_code traps
- Runs as a **method body**: no `using` directives (parsed as statements) — fully-qualify every type.
- Identify worlds by **`world.Name == "ServerWorld"/"ClientWorld"`** (flags overlap a shared `Game` bit).
- **`Type.GetMethod(name, flags)` throws AmbiguousMatchException** on overloaded methods (EntityManager generics,
netcode statics) — iterate `GetMethods()` and filter by name/param-count, or call concrete generics directly in
compiled code.
- Rukhanka `ParameterValue` is a union — read `floatValue`/`boolValue` per the parameter type.
- Return a built string (`StringBuilder`) — it's the only channel out.
## The server-inject harness (validate replicated presentation WITHOUT gameplay driving)
Proven 2026-07-07 (NodeFeedbackSystem). Validates a replicated-state → client-presentation path end-to-end in ~4
MCP calls, no expedition run, no player positioning:
1. **Spawn** (server world): query the ghost prefab (`ComponentType` + `EntityQueryOptions.IncludePrefab`, pick the
one with `Prefab`), `Instantiate`, then `em.GetComponentData<LocalTransform>(prefab).WithPosition(playerPos +
offset)`**never `LocalTransform.FromPosition`** (resets Scale=1; Scale replicates → consistent-but-wrong).
Find the player via `KinematicCharacterBody` + `LocalTransform`.
2. **Defeat relevancy**: a region-tagged ghost spawned at base gets culled from base players —
`em.SetComponentData(inst, new RegionTag { Region = 0 /* Base */ })` in the same call.
3. **Inject state changes server-side** (`SetComponentData` on the replicated component) in one call; **read the
client** in the NEXT call (round-trip ≥ several ticks — replication has happened).
4. **Assert numerically, then screenshot.** e.g. client `PostTransformMatrix.c0.x` vs the expected
`lerp(min, 1, frac)` and child `LocalToWorld` scale (`math.length(ltw.c0.xyz)`) vs `bakedScale × ptm`. Numbers
prove the math; the screenshot proves it renders (both matter — see visual limits below).
Ghost/LEG inspection idioms: `em.GetBuffer<LinkedEntityGroup>(root)`; render children = LEG entries with
`MaterialMeshInfo` (the root usually has none); parenting = `Parent` component; a baked child's uniform authoring
scale lands in `LocalTransform.Scale` (non-uniform → its own `PostTransformMatrix`). A root `PostTransformMatrix`
DOES propagate into parented children's `LocalToWorld` (verified live on 6.5.0).
## Headless expedition-run driving (flaky — know the failure modes)
> The specific nouns below (`PlayerReady`, expedition launch, respawn-reset) are **co-op-Hades-era** and become
> salvage under the LANTERN pivot ([[DR-048_Lantern_Adoption_Full_Pivot]]) — they will be re-meaned to the
> **descent lifecycle** (muster → Bell → pockets) when that lands. The *harness patterns* (server-inject, the
> `EditorApplication.update` guardian, capture-in-the-same-call) are direction-agnostic and stay valid.
- Launch by setting server `PlayerReady=1` from clean Staging. **Post-abort re-launch is intermittent** — a clean
Play restart launches reliably on the first ready; re-readying after an abort may not.
- **An AFK player dies to the swarm in ~15 s** and `Health` resets to the class value on respawn — a
`SetComponentData` HP buff does NOT stick across respawn; the run then aborts (0 expedition players → Returning)
and tears down the room.
- Therefore: **capture evidence in the SAME `execute_code` call that first detects the condition** (e.g.
`enemies>0`) — a separate "then screenshot" round-trip loses the window.
- **Multi-sample observations (movement deltas, before/after) that can't fit one call: install an
`EditorApplication.update` guardian delegate** (proven 2026-07-09, enemy-backstop smoke). The closure runs every
editor frame: it can out-race damage (`Health.Current = Max` per frame — beats the respawn-reset trap above),
park the player, take timed snapshots, and stash results in `EditorPrefs` (survives across `execute_code`
calls — statics do NOT). Always: wrap the body in try/catch that UNSUBSCRIBES on error, add a hard timeout
unsubscribe (~90 s), self-unsubscribe when done, and delete the `EditorPrefs` keys after reading them.
- Prefer the server-inject harness above whenever the question doesn't actually require a live run.
## Visual/asset verification limits
- **A dark/stylized frame masks material bugs** — verify VALUES: `shader.GetPropertyType(idx)`-guard before
`GetColor`/`GetFloat`/`GetTexture` (e.g. `S_General._BaseColorMultiply` is a float; `GetColor` returns black).
- **Skinned meshes lie to readers**: `WorldRenderBounds`/bone transforms return the bind-pose AABB, not the deformed
pose — deformed-pose visual bugs need the operator's eyes (or a screenshot), not entity reads.
- Body tint/flash validation: drive the stock EG `URPMaterialPropertyBaseColor` override on the render (LEG)
children; verify on the LOCAL player (persistent, centered) rather than a transient enemy.
- Forcing an animation pose: disable the drive system, write params via `AnimatorParametersAspect`; measure
ground-contact via feet-Y only on non-deformed geometry.
## EditMode fixtures for prediction-gated systems (07-21, AbilityFireSystemConeTests)
- **`NetworkTime.Flags` is INTERNAL** — a system gated on `IsFirstTimeFullyPredictingTick` silently no-ops in any
plain test world (default Flags = 0). Fixture fix: box the struct + `typeof(NetworkTime).GetField("Flags",
NonPublic|Instance).SetValue(boxed, NetworkTimeFlags.IsInPredictionLoop | IsFirstTimeFullyPredictingTick)`.
Test-only; never reflect internals in shipped code.
- **Input-buffer presses**: an `InputBufferData<PlayerInput>` entry with a set `InputEvent` keeps answering
`GetDataAtTick` for every later tick — it re-fires the moment a cooldown re-opens (the MeleeComboTests C6
lesson, buffer-shaped). Always push a press at T **and a release (default command) at T+1**.
- Damage paths gated `WorldUnmanaged.IsServer()` need `new World(name, WorldFlags.Game | WorldFlags.GameServer)`.
## Test-run mechanics
- `run_tests(mode="EditMode", assembly_names=["ProjectM.Tests.EditMode"])` → poll `get_test_job`
(`wait_timeout=60`). Unfocused editor: `init_timeout=120000` and retry once before diagnosing.
- A test count LOWER than expected with zero failures = tests not discovered (raw-written `.cs` without
`refresh_unity scope=all mode=force`), not a pass.