Commit graph

29,133 commits

Author SHA1 Message Date
Takoroka
69e7033d82 Texture cache: stop the churn, the per-frame image leak and garbage textures
- Unmapped GPU addresses get a stand-in CPU address fixed by the GPU
  address. A fresh one per lookup meant FindImage never found an image
  there again: title 1's bloom chain at 0xf0000-0x15b600 was re-created
  on every use (~5,400 images per 600 frames), filling VRAM.
- ASTC whose blocks carry no alpha (exact endpoint-mode scan reusing the
  decoder's block parsing) is re-compressed to BC1 instead of BC3, half
  the size; about 70% of title 1's ASTC qualifies. Copies between BC1
  and BC3 hosts are skipped (their guest data is uploaded instead).
  EDEN_NX2_ASTC_BC1=0 turns it off.
- Deleting an image cancels every decode still queued for it. An image
  re-decoded mid-decode has two; the first to finish cleared IsDecoding,
  the second was not cancelled and was uploaded into the next image to
  take the slot: garbage textures (the title logo, road patches) and a
  staging overrun when that image was smaller.
- Diagnostics: images created and deleted per 600 frames by shape,
  collector deletions, async decodes, BC1/BC3 split; collector
  thresholds overridable (EDEN_NX2_TC_EXPECTED_MIB/CRITICAL_MIB); GPU
  ASTC decode wait timing (EDEN_NX2_ASTC_FLUSH=1 submits without it).

Title 1 attract loop: device memory 4.8-5.1 GB instead of 5.9-6.1 GB, no
decoded textures evicted, ~17 -> 22-23 fps.
2026-10-28 19:32:48 -04:00
Takoroka
eb9f32a073 eden-cli: crash handler prints the faulting stack first; log close requests
MiniDumpWithIndirectlyReferencedMemory left an empty file on Eden's
address space, so print the exception, address and stack as
module+offset before writing a smaller dump, and report a failed write.
A window close or quit request is now logged, to tell it from a crash.
2026-10-28 19:32:48 -04:00
Takoroka
525929363f nx2:gsv 991: size decoder image pools (title 1 movie abort)
nn::codec::CalculateDecoderImagePoolBuffer sends 991(preference, count)
and reads a size and an alignment. Auto-stubbed, the reply kept the
request words, the frame size read as 0x167027f'00000001 and title 1's
movie player aborted on its budget check when its first movie started.
Answer a block-linear NV12 size (4 bytes per pixel for other layouts).
2026-10-28 19:32:48 -04:00
Takoroka
669dcfacee Texture cache: log VMA blocks per memory type with the periodic memory log
Device memory use is ~3 GB above what the images account for; this shows
where. The per-decode log drops to debug level.
2026-10-28 18:36:20 -04:00
Takoroka
cfb9245911 eden-cli: EDEN_NX2_CRASH_DUMP writes a minidump on unhandled exceptions
A debugger changes the timing enough to hide some crashes; this catches
them at full speed. dbghelp.dll is loaded only when a dump is written.
2026-10-28 18:36:20 -04:00
Takoroka
9a86143287 Notes: row 102 resolved (async decode use-after-free) 2026-10-28 17:35:05 -04:00
Takoroka
5505e1ddae Texture cache: fix use-after-free in async texture decode
TickAsyncDecode locked a finished decode's mutex, erased the context (freeing
the mutex) and only then let the lock go out of scope, unlocking freed
memory. With ASTC re-compressed on the CPU (accelerate_astc=CpuAsynchronous)
title 1 hit this a few minutes in: cdb reported "Free Heap block modified
after it was freed" at the mutex offset, and a run without the debugger
crashed. The lock is now released before the erase.

Also, deleting an image while it is being decoded (deletions other than the
collector, which already skips them) cancels the decode instead of
uploading into a freed slot.

Verified under cdb past 330 s (failed at 243 s twice before).
2026-10-28 17:34:48 -04:00
Takoroka
9677090712 nx2 services: callbacks no longer outlive their sessions
Under cdb title 1 reported "Free Heap block modified after it was freed"
when it recreated its display session (~4 min in); a run without the
debugger crashed at 113 s. The vi2 vsync timer callback (HostTiming thread)
captured the session and signalled its events; the destructor unscheduled
the timer (which does not wait for a running callback) and closed the
events. The syncpt slot actions (GPU thread) had the same race with
CloseEvent, as did the frame-capture and dump callbacks.

vsync events now live in state shared with the callback and are emptied
under its lock before closing; syncpt actions check a shared "alive" flag
under a mutex the destructor takes; capture/dump callbacks hold no session
pointers.
2026-10-28 17:22:54 -04:00
Takoroka
eb3edd5a84 Notes: rows 99-102 (VRAM/BC3, transcode pool, Medal layer, host crash) 2026-10-28 17:16:06 -04:00
Takoroka
d30b8dcef3 ASTC transcode pool: an eighth of the hardware threads (at least 2)
With half (8 on a Ryzen 7 8845HS), re-compressing streamed ASTC textures to
BC3 took ~3.5 cores away from the six emulated cores and the GPU thread.
Title 1 attract loop, 65-150 s: 8 threads 16.0 fps, 4 threads 12.6, 2
threads 18.9 (scene noise is large). EDEN_NX2_TRANSCODE_THREADS overrides.
2026-10-28 17:15:59 -04:00
Takoroka
81e7bc4112 ASTC: re-compress to BC3 except huge arrays, which stay RGBA8 and trimmed
Title 1 has ~700 ASTC 8x8 textures (~280 MB as stored); decoded to RGBA8
they took ~4.5 GB, pushing Eden past the 8 GB of an RTX 4070 Laptop
(8.4-9.8 GB in use) and the texture cache into permanent aggressive
collection. With astc_recompression=Bc3 (CPU, asynchronous) they fit
(4.6-5.6 GB). The two 8192x8192x68 terrain arrays would still not fit as
BC3, so ASTC 2D arrays over 1 GiB as RGBA8 always use the GPU decoder with
trimmed levels (RGBA8 host image, UNORM/sRGB views), and the decoder pass
now exists whenever the GPU lacks native ASTC.

Also logs every 600 frames: device memory against the collector thresholds,
the largest images, totals by format, and buffer cache totals.

Title screen 65-150 s: 10.1 -> 16.0 fps.
2026-10-28 17:08:43 -04:00
Takoroka
b661f30210 Notes: title 1 rows 92-98 (colour cast, terrain, sin/cos, movie abort, pop-in, frame rate) 2026-10-28 16:44:43 -04:00
Takoroka
e69090383e Performance diagnostics and small wins for title 1
- eden-cli: SDL called SDL_AppIterate back to back, keeping a host core at
  100% polling input; run the main callbacks at 240 Hz.
- EDEN_NX2_IDLE_SPIN_US: optional spin before an idle core blocks (measured:
  no gain on title 1, so off by default).
- EDEN_NX2_SVC_TRACE=<first>:<last>: trace SVCs (with thread ids) for a range
  of presents only.
- MME DMA reads flush GPU-written memory first (never triggered on title 1,
  kept for correctness); render-enable override histogram instead of the
  first-400 log.

Measured on title 1's title screen (RTX 4070 Laptop): GPU ~20% busy, the six
emulated cores ~0.9 host cores in total, Eden's GPU thread 45-70% busy and
largely waiting: on host fences for semaphore acquires (GPU accuracy High
delays every release to host completion), on ASTC upload Finish(), and in a
1 ms poll sleep. gpu_accuracy=Low: ~12.5 -> ~16 fps.
2026-10-28 16:44:24 -04:00
Takoroka
64c39aa0b7 Switch 2 shaders: MUFU.SIN/COS take turns, Eden's Maxwell MUFU radians
SM70+ removed RRO: compilers multiply the angle by 1/2pi (FMUL.RZ by
0.15915494) and MUFU.SIN/COS compute sin/cos(2pi x). Eden's Maxwell MUFU
computes sin/cos of radians (its RRO is a plain move), so every lowered sine
and cosine used the wrong angle (wind animation, rotations, the ocean FFT
twiddles). Scale the operand by 2pi first.
2026-10-28 15:44:48 -04:00
Takoroka
b8a07d6d5b Keep huge ASTC arrays from a lower mip on the host (title 1 terrain)
Title 1's terrain virtual textures are 8192x8192x68 ASTC 8x8 sRGB arrays with
12 levels: 1.5 GB in guest memory (fully backed) but 23 GB each as the RGBA8
Eden decodes ASTC to, so they were created unbacked and the near terrain
sampled black. A GPU-decoded ASTC 2D array over 1 GiB decoded now keeps only
the levels from the first one that fits in 512 MiB (here 3: 1024x1024, 380
MiB); host mip 0 is that guest level. Uploads read each layer's tail of the
mip chain into a compact buffer and decode only those levels, views shift
their level range, and these images are never downloaded. Normalized
coordinates and derivative LOD address the smaller image correctly; the
nearest terrain is lower resolution than on the console.

Also logs how unbacked images are used (views, render targets, blits, DMA).
2026-10-28 15:37:34 -04:00
Takoroka
6d7acfb990 Fix the P6 scratch save in R2UR/MATCH/HFMA2.RELU; GPU memory dump tool
The colour cast on title 1 (magenta/cyan, then red/green, changing per
course) came from the sky/atmosphere volume shader (a pixel shader drawing
a 32^3 RGBA16F table slice by slice), which starts with an unguarded R2UR.
R2UR, MATCH and HFMA2.RELU use P6 as scratch and saved/restored it with the
uniform-guard mask, 1 << (6 - 7) for an unguarded instruction: undefined, in
practice bit 31, which shifted into the opcode bits and turned the save into
another instruction. They now always save and restore P6.

Debugging aids used to find it:
- EDEN_NX2_DUMP=<present>:<addr>:<size>,...: flush GPU memory ranges at a
  present (on the GPU thread) and save them next to the frame captures;
  nx2work/vol3d.py untiles Switch 2 3D textures from such dumps.
- GPU::RunOnGpuThread.
- Logs of Switch 2 3D images and descriptors.
2026-10-28 15:19:44 -04:00
Takoroka
2d48fb6539 Jump-table programs end at their self-branch; aligned wide zero stores
- With a BRX the jump targets live in a constant buffer, so capture and
  lowering keep going past EXITs to the compiler's closing branch-to-self
  (a title 1 compute program jumped to 0xae0, just past the first EXIT).
- 128-bit stores of RZ use R248-R251 (Eden wants 4-aligned data registers).
- The rejected-program cache covers environment construction, where native
  programs are lowered (it never applied before), and logs the stage slot.

Title 1 reaches its title screen with the ground drawn.
2026-10-28 14:50:32 -04:00
Takoroka
410ef99109 IMAD.SHL as a shift; EDEN_NX2_REJECT; log where rejected programs are bound
- IMAD by a power of two lowers to SHL imm, the form Eden's jump-table
  tracking expects (a title 1 compute program failed "track indirect branch").
- EDEN_NX2_REJECT=<hex addresses>: treat those native programs as rejected,
  to bisect a behaviour change without rebuilding.
- A rejected graphics program logs its stage slot, header and slot registers
  once (for the task/mesh pipeline setup).
- Notes: rows 87-92, title 1 status.
2026-10-28 14:44:18 -04:00
Takoroka
033012cb0a Switch 2 shaders: LEA with a negated addend (ISCADD bit 48) 2026-10-28 14:39:20 -04:00
Takoroka
a9f07b419b Switch 2 shaders: compute forms (LDS uniform base, ATOMG, RED, STG.U16) and more
- LDS [Ra.Xn + URb]; ATOMG.ADD at a register address; RED at [URa+offset];
  8/16-bit STG; REDUX.MAX; R2UR and REDUX into any uniform register.
- UIMAD immediate; F2I from a uniform register; FSET.BF combined with a
  predicate; TXQ single component to an odd register; SHF.L.W.HI of an
  immediate by a register.

Title 1's dumped programs: 1654 of 1667 lower (the rest: 12 task/mesh
programs using ISBERD/ISBEWR, and one SURED).
2026-10-28 14:36:54 -04:00
Takoroka
27301da553 Cache rejected native shaders and log each rejection once
A rejected program was lowered again (and logged) on every draw or
dispatch: 1.7M log lines in 13 minutes of title 1. The shader cache keeps
the first exception per address until guest memory there is written, and
the error logs are deduplicated.
2026-10-28 14:31:00 -04:00
Takoroka
fcfcb5049f Switch 2 shaders: calls, uniform predicates, wide multiply and more forms
- RET: accept any copy of the call link into the return register in the
  function (MOV or IMAD.MOV); library functions may EXIT; @Pg EXIT Pc.
- PLOP3 with any table (SEL to 0/~0, LOP3 sets the predicate), including a
  uniform third source; UPLOP3 (with a uniform guard); UP2UR; ULOP3 with a
  uniform predicate output and guard.
- UIMAD.WIDE.U32, uniform USHF, SGXT.U32, FMNMX with a uniform source,
  I2F.U8 from a uniform register, I2F.U16 (.H1), F2I.FTZ floor/nearest,
  TXQ width/height masks, FSWZADD any swizzle, PRMT any byte selection.
- LEA.HI with an immediate addend; 0x923 is ISBERD, not PIXLD (removed).

Title 1's dumped programs: 1379 -> 1446 of 1473 lower.
2026-10-28 14:31:00 -04:00
Takoroka
a4b40f7ba3 Switch 2 shaders: IADD3 negated third source, LEA.HI with addend, uniform SHF, UR16-UR27
- IADD3 with a third source negated (bit 75).
- LEA.HI Rd, Ra, Rb, RZ: Rb plus the high word; bit 73 sign-extends.
- SHF with a uniform shift amount (0xc19) and SHF.R.U32 of the low word.
- Uniform registers UR0-UR21 in R216-R237 and UR22-UR27 in R240-R245 (pairs
  stay adjacent); ULDC and uniform operands use the mapping.
Title 1: 1359 of 1434 dumped programs lower.
2026-10-28 14:00:31 -04:00
Takoroka
2f657ac0a1 Vulkan: unbacked sparse images for textures too large to back
Title 1 samples an 8192x8192, 12-level, 68-layer array (a sparse virtual
texture, ~23 GB): vkCreateImage failed and Eden aborted, or VMA put it in
system memory and staged a full upload (34 GB private). Images over 2^28
texels are now created sparse-resident with no memory bound; the texture
cache neither uploads nor downloads them. Images merely over the VRAM
budget are allocated without the budget limit instead of failing.
2026-10-28 13:55:29 -04:00
Takoroka
bb0eb5ec48 Switch 2 shaders: centroid IPA, 8/16-bit LDG, UP0-UP6, PIXLD.COVMASK, USHF.R, 2D_ARRAY surfaces
Title 1 (768 dumped programs): 386 lowered cleanly, now 701.
- IPA: bits 76-77 are the sample mode (most varyings are centroid), mapped to
  Maxwell bits 52-53.
- LDG: U8/S8/U16/S16 sizes as Maxwell's.
- Uniform predicates UP0-UP6 (UISETP outputs, USEL/VOTEU, uniform execution
  guards) in R239 bit 6-n, swapped into P(6-n).
- PIXLD.COVMASK reads all-ones (no sample mask input; a mask write ANDs it).
- USHF.R.U32 of the low or high register.
- SULD/SUST 2D_ARRAY (dimension 4, as Maxwell's).
2026-10-28 13:55:29 -04:00
Takoroka
dae499692d nx2: notes on title 1 reaching its title screen 2026-10-28 13:23:14 -04:00
Takoroka
9e3f5cea40 Switch 2: nx2:gpu 215 DestroySparseRegion; log 214 arguments
nn::gpu::sdk names 214 CreateSparseRegion and 215 DestroySparseRegion.
215 (range in words 3-6, job slot in word 8) drops the GPU mapping and
completes its job behind the GPU; it was auto-stubbed and never signalled.
2026-10-28 13:22:59 -04:00
Takoroka
4fc2ffa89b video_core: 64 GiB device address space
Switch 2 titles map most of their memory twice plus GPU-only regions;
title 1 ran out of the 16 GiB SMMU space. The buffer cache page table and
memory tracker cover the wider space.
2026-10-28 13:22:59 -04:00
Takoroka
e1cc47ac43 Switch 2: per-channel MME data RAM; one QMD table for all channels
Macro code stays shared by the channels of a process, but each channel
now has its own MME data RAM. Title 1's channels keep their own QMD ring
base in data word 0x27; with one shared RAM every channel used the last
channel's ring, QMDs collided, releases came early or never and the GPU
waited on semaphores (4 s timeouts per frame).

The QMD scheduler state is one table for all channels: a QMD's
dependency counter is state of the QMD, scheduled and decremented from
different channels.

Diagnostics: EDEN_NX2_QMD_TRACE (scheduler loads, counts, launches) and
EDEN_NX2_MME_DUMP (compute macro code, calls, data word 0x27 changes).
2026-10-28 13:22:59 -04:00
Takoroka
862880b400 nx2: notes on title 1's vsync hang and what it hits next 2026-10-28 12:31:54 -04:00
Takoroka
ed97c9b9dc debugger: monitor handles lists each handle's type and service event name
gdbrsp.py gains raw and monitor commands.
2026-10-28 12:31:45 -04:00
Takoroka
508f3400bd Switch 2: signal nx2:vi2 31/32 events on every vblank
Title 1's main thread waits on the vi2 32 event until it has fired twice
before it submits GPU work; Eden's stub never signalled it and the title
hung before its first frame. Signal the 31 and 32 events at 60 Hz.
2026-10-28 12:31:44 -04:00
Takoroka
da234f31e6 nx2: notes on the GPU-only memory fix and title 1's hang 2026-10-28 12:12:26 -04:00
Takoroka
8e7edfaa98 Switch 2: back GPU-only regions outside the application pool
nx2:gpu 214 regions are memory the graphics server keeps itself, but Eden
took their pages from the application pool: title 4's six 300 MB regions
left it 1.8 GB less than the 9.55 GiB the kernel reports, all of which
its heap and VAMM plan to use. Add 8 GiB of DRAM to the 12 GB layout for
the non-secure system pool and back the regions from there; the device
memory tracker now covers 32 GiB of physical memory.
2026-10-28 12:03:11 -04:00
Takoroka
98871aae52 kernel: EDEN_NX2_MEM_TRACE logs heap, physical memory and memory info requests 2026-10-28 12:03:10 -04:00
Takoroka
f93966bc22 nx2: notes on the 100 MiB ColPaintBuild failure behind the recurring crash 2026-10-28 11:46:21 -04:00
Takoroka
47bd739011 nx2: notes on the six-core debugger run 2026-10-28 11:35:31 -04:00
Takoroka
08c5ffd642 core: emulate six CPU cores
Switch 2 applications are given cores 0..5 (title 4's npdm) and start a
job worker per core. With four emulated cores, 4 and 5 were folded onto
0 and 1, so two cores ran two workers each. Switch 1 titles keep using
cores 0..3; core 3 stays the shared system core.
2026-10-28 11:29:20 -04:00
Takoroka
cea331e4c7 nx2: notes on which heap the scene's GPU pools come from 2026-10-28 11:14:08 -04:00
Takoroka
d9ee15f545 nx2: notes on the scene heap running out at the recurring crash 2026-10-28 11:05:44 -04:00
Takoroka
0d1fab27e8 nx2: notes on the recurring worker-thread crash; gdbrsp waits without a timeout 2026-10-28 10:49:26 -04:00
Takoroka
c489bae2d0 nx2: read debug switches once; key scripts from a file
Several debug switches were read with std::getenv on every macro call, QMD,
semaphore write or draw. The UCRT getenv locks and scans the environment,
thousands of times a frame; a long EDEN_NX2_KEY_SCRIPT made title 4 crawl
at 2 fps. Each is now read once. EDEN_NX2_KEY_SCRIPT=@<file> reads the
script from a file. gdbrsp.py gains 'hits' (log registers, frame chain and
object words at breakpoints, following the stopped thread).
2026-10-28 07:47:45 -04:00
Takoroka
fcd02f8778 nx2: unmap GPU memory in submission order
NormalUnmapGpuMemory4K and destroying GPU memory are jobs the console's
graphics server completes behind the work already submitted. Run the GMMU
unmaps on the GPU thread after the command lists queued before them
(GPU::RunInOrder) and signal the job's event only then, instead of unmapping
and signalling at once while Eden's GPU thread may still be behind.

Also EDEN_NX2_FRAMES_AHEAD (experimental): wait for the previous frame's
fence at present instead of this one's.
2026-10-28 07:47:45 -04:00
Takoroka
36b6b30452 nx2: ATOMG and local spills with an immediate offset
ATOMG.E.ADD through a uniform pointer and STL/LDL.32 at RZ only took a zero
offset; title 4's loading screen uses [UR + 4] and [RZ + 4]. Maxwell ATOM
has a signed 20-bit offset at bits 28-47, STL/LDL a 24-bit one at 20-43.
2026-10-28 07:47:45 -04:00
Takoroka
a89b3087a9 nx2: method data continues across GPFIFO entries
Title 4 calls a 3D macro with a 6-word IncreaseOnce header, ends the main
GPFIFO entry after the first word and supplies the other five in a 5-word
subroutine entry. The split-method parking (an entry elsewhere was treated
as a call with a clean method state) fed those parameters to the parser as
headers: the macro ran with garbage, later draws read megabytes of unmapped
memory and the host driver timed out at the loading screen after the dock
scene. As on the hardware, the method state now simply continues; the old
guess stays behind EDEN_NX2_PARK_SPLIT_METHODS.

Debug aids: splits are logged at debug level with the words around them,
and a 3D write that interrupts a macro's parameters dumps the last 48 writes.
2026-10-28 07:47:45 -04:00
Takoroka
5046a37c95 nx2: notes for the semaphore, pacing and open-file fixes 2026-10-28 05:55:09 -04:00
Takoroka
c81a146fd5 vfs: keep the open-file cache below the CRT limit
The real VFS kept up to 8192 host files open, which is also the most the
Windows C runtime allows (_setmaxstdio(8192)). A game read from a loose RomFS
folder fills that within minutes (title 4: ~7300 handles after 160 s); from
then on every other open failed with "too many files open" - its next scene
pack, its save file, the shader cache - and the game waited forever on the
load. Cap the cache 512 below _getmaxstdio() on Windows.
2026-10-28 05:55:09 -04:00
Takoroka
42989ad8a9 nx2: pace Switch 2 presents to a 60 Hz vblank
The console flips one frame per vblank and holds the rest in the swapchain,
which caps a game at 60 fps. vi2 229 reports every buffer slot free at once,
so nothing paced title 4: it presented at 90-160 fps and its frame-stepped
timing, audio included, ran ahead. Hold each present until the next vblank,
scaled by the speed limit, without banking time lost to slow frames.
2026-10-28 05:55:09 -04:00
Takoroka
5d27ccd0a0 nx2: semaphore releases latch the acquires they satisfy
Switch 2 channels that wait for an exact semaphore value poll on the console
and see every release. Eden resumes a waiting channel only between command
lists and writes releases when fences signal, often several back to back, so
an acquire-equal could miss its value and wait forever. Title 4's 3D queue
ticks a counter that two other queues wait on value by value; under the
loading after the post-customization save one of them missed a tick and the
game froze (channel 4 waiting for 0x5293 with the counter at 0x5310).

Pending acquires now register with Nx2SemaphoreWaiters before fences run,
and every query-cache payload write and QMD add/max release latches the
waiters it meets at the moment it is written.
2026-10-28 05:41:45 -04:00
Takoroka
fd768d8530 nx2: notes for the freed-memory crash and the ASTC UI fix 2026-10-28 05:16:04 -04:00