{
 "rows": [
  {
   "ts": "2026-07-20T09:21:02",
   "date": "2026-07-20",
   "source": "session",
   "kind": "wrongtarget",
   "summary": "Probed local :7869 (3060 Ti) believing it was Amethyst; local box owns :7869, Amethyst is reached via the :7870 tunnel \u2014 read the wrong forge's state and nearly acted on it"
  },
  {
   "ts": "2026-07-20T09:36:36",
   "date": "2026-07-20",
   "source": "session",
   "kind": "dataloss",
   "summary": "Fresh weapon_forge run overwrote existing gens: default --seed-base 1 + deterministic prompt stream = identical filenames; and pod dropped to gens/<cat>/ not gens/_incoming/<cat>/ (non-parity). Silent overwrite, flat count masked it."
  },
  {
   "ts": "2026-07-21T13:34:42",
   "date": "2026-07-21",
   "source": "autoloop:bow",
   "kind": "crash",
   "summary": "bow: stage returned error"
  },
  {
   "ts": "2026-07-21T13:34:52",
   "date": "2026-07-21",
   "source": "autoloop:bow",
   "kind": "crash",
   "summary": "bow: stage returned error"
  },
  {
   "ts": "2026-07-22T04:45:55",
   "date": "2026-07-22",
   "source": "autoloop",
   "kind": "crash",
   "summary": "PermissionError: [WinError 5] Access is denied: 'C:\\\\Users\\\\Cody\\\\Desktop\\\\IronForge\\\\autoloop_state.json.tmp' -> 'C:\\\\Users\\\\Cody\\\\Desktop\\\\IronForge\\\\autoloop_state.json'"
  },
  {
   "ts": "2026-07-22T06:09:20",
   "date": "2026-07-22",
   "source": "autoloop",
   "kind": "crash",
   "summary": "PermissionError: [WinError 5] Access is denied: 'C:\\\\Users\\\\Cody\\\\Desktop\\\\IronForge\\\\autoloop_state.json.tmp' -> 'C:\\\\Users\\\\Cody\\\\Desktop\\\\IronForge\\\\autoloop_state.json'"
  },
  {
   "ts": "2026-07-22T08:02:20",
   "date": "2026-07-22",
   "source": "art_widget",
   "kind": "repeat",
   "summary": "Art widget goes blank whenever a new render pod is added: the collector blocks on ALL forge ports (BASE..BASE+SPAN) serially at 2s timeout each before/around the folder scan; new dash pods have no forge and their dead/half-open tunnels TIME OUT (not refuse), so each tick overruns the 2s refresh interval and the widget holds a dimmed last-good grid. Adding a card widens the dead-port span and/or le"
  },
  {
   "ts": "2026-07-22T08:39:37",
   "date": "2026-07-22",
   "source": "live_feed",
   "kind": "repeat",
   "summary": "Wrong GPU labels on minkforge.dev/live traced to TWO bugs. (1) Label double-source: the local 3060 forge on :7869 is fed to live_feed BOTH via /v1/live HTTP probe (labeled by port -> '3060Ti') AND via the folder scan of cards-v1 (labeled by filename -> 'local 3060'); the two items carry different (port,job_id) so dedup can't collapse them, so the same card shows twice with two labels. (2) Seed bac"
  },
  {
   "ts": "2026-07-22T11:49:00",
   "date": "2026-07-22",
   "source": "autoloop:warhammer",
   "kind": "crash",
   "summary": "warhammer: stage returned error"
  },
  {
   "ts": "2026-07-22T11:52:08",
   "date": "2026-07-22",
   "source": "autoloop:warhammer",
   "kind": "crash",
   "summary": "warhammer: stage returned error"
  },
  {
   "ts": "2026-07-22T12:33:20",
   "date": "2026-07-22",
   "source": "opus",
   "kind": "repeat",
   "summary": "Dash racer pods (no forge server) come up INVISIBLE to HUD + art feed; fleet_wire.py wire is forge-only (allocates port+tunnel) so it's the wrong tool, and dash_ctl.py wires nothing on create. Correct path (add name to HUD_FLEET_PODS + wire_restart.py restart; drain dir into ART_GALLERY_DIRS) exists but isn't forced, so every dash pod is manually re-wired."
  },
  {
   "ts": "2026-07-22T14:19:36",
   "date": "2026-07-22",
   "source": "self",
   "kind": "repeat",
   "summary": "archive_mine.py OUT_DIR hardcoded to output/taste/archive-mine with no --out-dir flag; mining a new tag-set (8k card pool) overwrote the corpus-wide archive-mine manifests. No data lost (manifests recompute from intact embeds) but forced a full 160k regen. Backup-rename guard failed silently (path or device name error). FIX: add --out-dir flag so each mine writes its own dir."
  },
  {
   "ts": "2026-07-22T18:22:27",
   "date": "2026-07-22",
   "source": "autoloop:crowns-circlets",
   "kind": "crash",
   "summary": "crowns-circlets: stage returned error"
  },
  {
   "ts": "2026-07-23T06:28:38",
   "date": "2026-07-23",
   "source": "session",
   "kind": "other",
   "summary": "blocked-fix idle-burn: rife A5000 sat RUNNING-idle ~5.5h (~$0.90) after rm was classifier-blocked; standing order already covered the delete \u2014 see CIP_BLOCKED_FIX_IDLE_BURN.md"
  },
  {
   "ts": "2026-07-23T11:54:49",
   "date": "2026-07-23",
   "source": "autoloop:warhammer",
   "kind": "waste",
   "summary": "a stage TIMED OUT"
  },
  {
   "ts": "2026-07-23T11:54:49",
   "date": "2026-07-23",
   "source": "autoloop:warhammer",
   "kind": "crash",
   "summary": "warhammer: stage returned error"
  },
  {
   "ts": "2026-07-23T13:18:06",
   "date": "2026-07-23",
   "source": "vultr_ctl",
   "kind": "waste",
   "summary": "Vultr smoke boxes launched fire-and-forget; bootstraps died pre-heartbeat, self-destroy never fired, 0.635/hr dead burn found by accident 2.25h later"
  },
  {
   "ts": "2026-07-23T13:31:29",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "waste",
   "summary": "RunPod S3 403s bare Python-urllib UA from Vultr IPs; FAILED flare also 403d -> 2 box generations silent, gen-1 nuked undiagnosed"
  },
  {
   "ts": "2026-07-23T13:31:29",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "crash",
   "summary": "Ubuntu 24.04 debian-owned rich 13.7.1 (no RECORD) kills pip upgrade on smoke box"
  },
  {
   "ts": "2026-07-23T13:31:29",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "crash",
   "summary": "hpsv3 imports matplotlib but doesn't declare it; smoke-hps died at last import"
  },
  {
   "ts": "2026-07-23T13:31:29",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "other",
   "summary": "Cloudflare WAF 401s any S3 PUT whose body is a bare 16-hex token; kit VERSION stamp un-PUT-able"
  },
  {
   "ts": "2026-07-23T13:31:29",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "other",
   "summary": "Vultr API key IP-restricted -> box-side self-destroy 401s; cleanup is always local nuke + dead-man"
  },
  {
   "ts": "2026-07-23T13:43:34",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "crash",
   "summary": "hpsv3 also imports torch.utils.tensorboard undeclared \u2014 second consecutive smoke-hps FAILED past matplotlib; fix: tensorboard added to hps deps"
  },
  {
   "ts": "2026-07-23T13:43:34",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "crash",
   "summary": "caption stage ran caption_dataset.py by absolute path with cwd=ROOT but no PYTHONPATH \u2014 No module named engine; absolute path puts script dir on sys.path not cwd; fix: PYTHONPATH=ROOT in stage env"
  },
  {
   "ts": "2026-07-23T13:43:34",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "waste",
   "summary": "retry chicken-egg: relaunched provision pulls fixed kit but keeps executing already-imported old code; fix: scp changed files to live boxes \u2014 retries must run the code you fixed"
  },
  {
   "ts": "2026-07-23T13:49:15",
   "date": "2026-07-23",
   "source": "vultr-smoke",
   "kind": "other",
   "summary": "HPSv3 7B fp32-on-CPU peaks 63.6GB RSS, OOM-SIGKILL on 62GB-usable 64GB Vultr box; real smoke finding \u2014 HPS-CPU is a fallback tier needing >64GB RAM or big swap; fixed this run with 40GB swap, loaded clean in 23s"
  },
  {
   "ts": "2026-07-23T15:02:16",
   "date": "2026-07-23",
   "source": "foreman",
   "kind": "repeat",
   "summary": "cp1252 non-ASCII chars in .ps1/.vbs launchers desync the cp1252 PowerShell/WScript parser (bogus 'Missing closing }'), silently failing scheduled tasks; the .py-only cp1252 guard had a launcher-shaped blind spot"
  },
  {
   "ts": "2026-07-23T23:52:04",
   "date": "2026-07-23",
   "source": "cody-session",
   "kind": "wrongtarget",
   "summary": "Obsidian A5000 terminated mid-render (bluewitch a5000-bw lane, 100% GPU ~7h) and replaced by a bare pod 'Larimar-migration' on the same host; run deposited 0 imgs / no results.json to S3 = full loss of that render. No harvestable output. Termination happened outside the working session."
  },
  {
   "ts": "2026-07-23T23:58:20",
   "date": "2026-07-23",
   "source": "session",
   "kind": "repeat",
   "summary": "fault-H HPS-CPU OOM fix declared 'in code' but was incomplete: in-place fp32 lowers RESIDENT to ~28GB yet the 64GB box still SIGKILL(9)'d at the hps stage because provision NEVER created the swap the CIP note itself required. Fix = _ensure_swap() before the hps subprocess + batch-1 + RSS instrumentation."
  },
  {
   "ts": "2026-07-24T01:52:47",
   "date": "2026-07-24",
   "source": "rife_retrain_gate.py",
   "kind": "other",
   "summary": "verdict json saved to state BEFORE the armed block added fire_cmd/gate_cmd, so persisted last_verdict never carried the commands a downstream reader (V15 page) needs"
  },
  {
   "ts": "2026-07-24T01:52:47",
   "date": "2026-07-24",
   "source": "runpod-mcp",
   "kind": "waste",
   "summary": "4 A5000 create attempts stockout-failed in a row; list-gpu-types maxCount=10 is POOL CAPACITY not live availability"
  }
 ],
 "total": 31,
 "by_kind": {
  "wrongtarget": 2,
  "dataloss": 1,
  "crash": 12,
  "repeat": 6,
  "other": 5,
  "waste": 5
 }
}