TickAsyncDecode locked a finished decode's mutex, erased the context (freeing
the mutex) and only then let the lock go out of scope, unlocking freed
memory. With ASTC re-compressed on the CPU (accelerate_astc=CpuAsynchronous)
title 1 hit this a few minutes in: cdb reported "Free Heap block modified
after it was freed" at the mutex offset, and a run without the debugger
crashed. The lock is now released before the erase.
Also, deleting an image while it is being decoded (deletions other than the
collector, which already skips them) cancels the decode instead of
uploading into a freed slot.
Verified under cdb past 330 s (failed at 243 s twice before).
Under cdb title 1 reported "Free Heap block modified after it was freed"
when it recreated its display session (~4 min in); a run without the
debugger crashed at 113 s. The vi2 vsync timer callback (HostTiming thread)
captured the session and signalled its events; the destructor unscheduled
the timer (which does not wait for a running callback) and closed the
events. The syncpt slot actions (GPU thread) had the same race with
CloseEvent, as did the frame-capture and dump callbacks.
vsync events now live in state shared with the callback and are emptied
under its lock before closing; syncpt actions check a shared "alive" flag
under a mutex the destructor takes; capture/dump callbacks hold no session
pointers.
With half (8 on a Ryzen 7 8845HS), re-compressing streamed ASTC textures to
BC3 took ~3.5 cores away from the six emulated cores and the GPU thread.
Title 1 attract loop, 65-150 s: 8 threads 16.0 fps, 4 threads 12.6, 2
threads 18.9 (scene noise is large). EDEN_NX2_TRANSCODE_THREADS overrides.
Title 1 has ~700 ASTC 8x8 textures (~280 MB as stored); decoded to RGBA8
they took ~4.5 GB, pushing Eden past the 8 GB of an RTX 4070 Laptop
(8.4-9.8 GB in use) and the texture cache into permanent aggressive
collection. With astc_recompression=Bc3 (CPU, asynchronous) they fit
(4.6-5.6 GB). The two 8192x8192x68 terrain arrays would still not fit as
BC3, so ASTC 2D arrays over 1 GiB as RGBA8 always use the GPU decoder with
trimmed levels (RGBA8 host image, UNORM/sRGB views), and the decoder pass
now exists whenever the GPU lacks native ASTC.
Also logs every 600 frames: device memory against the collector thresholds,
the largest images, totals by format, and buffer cache totals.
Title screen 65-150 s: 10.1 -> 16.0 fps.
- eden-cli: SDL called SDL_AppIterate back to back, keeping a host core at
100% polling input; run the main callbacks at 240 Hz.
- EDEN_NX2_IDLE_SPIN_US: optional spin before an idle core blocks (measured:
no gain on title 1, so off by default).
- EDEN_NX2_SVC_TRACE=<first>:<last>: trace SVCs (with thread ids) for a range
of presents only.
- MME DMA reads flush GPU-written memory first (never triggered on title 1,
kept for correctness); render-enable override histogram instead of the
first-400 log.
Measured on title 1's title screen (RTX 4070 Laptop): GPU ~20% busy, the six
emulated cores ~0.9 host cores in total, Eden's GPU thread 45-70% busy and
largely waiting: on host fences for semaphore acquires (GPU accuracy High
delays every release to host completion), on ASTC upload Finish(), and in a
1 ms poll sleep. gpu_accuracy=Low: ~12.5 -> ~16 fps.
SM70+ removed RRO: compilers multiply the angle by 1/2pi (FMUL.RZ by
0.15915494) and MUFU.SIN/COS compute sin/cos(2pi x). Eden's Maxwell MUFU
computes sin/cos of radians (its RRO is a plain move), so every lowered sine
and cosine used the wrong angle (wind animation, rotations, the ocean FFT
twiddles). Scale the operand by 2pi first.
Title 1's terrain virtual textures are 8192x8192x68 ASTC 8x8 sRGB arrays with
12 levels: 1.5 GB in guest memory (fully backed) but 23 GB each as the RGBA8
Eden decodes ASTC to, so they were created unbacked and the near terrain
sampled black. A GPU-decoded ASTC 2D array over 1 GiB decoded now keeps only
the levels from the first one that fits in 512 MiB (here 3: 1024x1024, 380
MiB); host mip 0 is that guest level. Uploads read each layer's tail of the
mip chain into a compact buffer and decode only those levels, views shift
their level range, and these images are never downloaded. Normalized
coordinates and derivative LOD address the smaller image correctly; the
nearest terrain is lower resolution than on the console.
Also logs how unbacked images are used (views, render targets, blits, DMA).
The colour cast on title 1 (magenta/cyan, then red/green, changing per
course) came from the sky/atmosphere volume shader (a pixel shader drawing
a 32^3 RGBA16F table slice by slice), which starts with an unguarded R2UR.
R2UR, MATCH and HFMA2.RELU use P6 as scratch and saved/restored it with the
uniform-guard mask, 1 << (6 - 7) for an unguarded instruction: undefined, in
practice bit 31, which shifted into the opcode bits and turned the save into
another instruction. They now always save and restore P6.
Debugging aids used to find it:
- EDEN_NX2_DUMP=<present>:<addr>:<size>,...: flush GPU memory ranges at a
present (on the GPU thread) and save them next to the frame captures;
nx2work/vol3d.py untiles Switch 2 3D textures from such dumps.
- GPU::RunOnGpuThread.
- Logs of Switch 2 3D images and descriptors.
- With a BRX the jump targets live in a constant buffer, so capture and
lowering keep going past EXITs to the compiler's closing branch-to-self
(a title 1 compute program jumped to 0xae0, just past the first EXIT).
- 128-bit stores of RZ use R248-R251 (Eden wants 4-aligned data registers).
- The rejected-program cache covers environment construction, where native
programs are lowered (it never applied before), and logs the stage slot.
Title 1 reaches its title screen with the ground drawn.
- IMAD by a power of two lowers to SHL imm, the form Eden's jump-table
tracking expects (a title 1 compute program failed "track indirect branch").
- EDEN_NX2_REJECT=<hex addresses>: treat those native programs as rejected,
to bisect a behaviour change without rebuilding.
- A rejected graphics program logs its stage slot, header and slot registers
once (for the task/mesh pipeline setup).
- Notes: rows 87-92, title 1 status.
- LDS [Ra.Xn + URb]; ATOMG.ADD at a register address; RED at [URa+offset];
8/16-bit STG; REDUX.MAX; R2UR and REDUX into any uniform register.
- UIMAD immediate; F2I from a uniform register; FSET.BF combined with a
predicate; TXQ single component to an odd register; SHF.L.W.HI of an
immediate by a register.
Title 1's dumped programs: 1654 of 1667 lower (the rest: 12 task/mesh
programs using ISBERD/ISBEWR, and one SURED).
A rejected program was lowered again (and logged) on every draw or
dispatch: 1.7M log lines in 13 minutes of title 1. The shader cache keeps
the first exception per address until guest memory there is written, and
the error logs are deduplicated.
- RET: accept any copy of the call link into the return register in the
function (MOV or IMAD.MOV); library functions may EXIT; @Pg EXIT Pc.
- PLOP3 with any table (SEL to 0/~0, LOP3 sets the predicate), including a
uniform third source; UPLOP3 (with a uniform guard); UP2UR; ULOP3 with a
uniform predicate output and guard.
- UIMAD.WIDE.U32, uniform USHF, SGXT.U32, FMNMX with a uniform source,
I2F.U8 from a uniform register, I2F.U16 (.H1), F2I.FTZ floor/nearest,
TXQ width/height masks, FSWZADD any swizzle, PRMT any byte selection.
- LEA.HI with an immediate addend; 0x923 is ISBERD, not PIXLD (removed).
Title 1's dumped programs: 1379 -> 1446 of 1473 lower.
- IADD3 with a third source negated (bit 75).
- LEA.HI Rd, Ra, Rb, RZ: Rb plus the high word; bit 73 sign-extends.
- SHF with a uniform shift amount (0xc19) and SHF.R.U32 of the low word.
- Uniform registers UR0-UR21 in R216-R237 and UR22-UR27 in R240-R245 (pairs
stay adjacent); ULDC and uniform operands use the mapping.
Title 1: 1359 of 1434 dumped programs lower.
Title 1 samples an 8192x8192, 12-level, 68-layer array (a sparse virtual
texture, ~23 GB): vkCreateImage failed and Eden aborted, or VMA put it in
system memory and staged a full upload (34 GB private). Images over 2^28
texels are now created sparse-resident with no memory bound; the texture
cache neither uploads nor downloads them. Images merely over the VRAM
budget are allocated without the budget limit instead of failing.
Title 1 (768 dumped programs): 386 lowered cleanly, now 701.
- IPA: bits 76-77 are the sample mode (most varyings are centroid), mapped to
Maxwell bits 52-53.
- LDG: U8/S8/U16/S16 sizes as Maxwell's.
- Uniform predicates UP0-UP6 (UISETP outputs, USEL/VOTEU, uniform execution
guards) in R239 bit 6-n, swapped into P(6-n).
- PIXLD.COVMASK reads all-ones (no sample mask input; a mask write ANDs it).
- USHF.R.U32 of the low or high register.
- SULD/SUST 2D_ARRAY (dimension 4, as Maxwell's).
nn::gpu::sdk names 214 CreateSparseRegion and 215 DestroySparseRegion.
215 (range in words 3-6, job slot in word 8) drops the GPU mapping and
completes its job behind the GPU; it was auto-stubbed and never signalled.
Switch 2 titles map most of their memory twice plus GPU-only regions;
title 1 ran out of the 16 GiB SMMU space. The buffer cache page table and
memory tracker cover the wider space.
Macro code stays shared by the channels of a process, but each channel
now has its own MME data RAM. Title 1's channels keep their own QMD ring
base in data word 0x27; with one shared RAM every channel used the last
channel's ring, QMDs collided, releases came early or never and the GPU
waited on semaphores (4 s timeouts per frame).
The QMD scheduler state is one table for all channels: a QMD's
dependency counter is state of the QMD, scheduled and decremented from
different channels.
Diagnostics: EDEN_NX2_QMD_TRACE (scheduler loads, counts, launches) and
EDEN_NX2_MME_DUMP (compute macro code, calls, data word 0x27 changes).
Title 1's main thread waits on the vi2 32 event until it has fired twice
before it submits GPU work; Eden's stub never signalled it and the title
hung before its first frame. Signal the 31 and 32 events at 60 Hz.
nx2:gpu 214 regions are memory the graphics server keeps itself, but Eden
took their pages from the application pool: title 4's six 300 MB regions
left it 1.8 GB less than the 9.55 GiB the kernel reports, all of which
its heap and VAMM plan to use. Add 8 GiB of DRAM to the 12 GB layout for
the non-secure system pool and back the regions from there; the device
memory tracker now covers 32 GiB of physical memory.
Switch 2 applications are given cores 0..5 (title 4's npdm) and start a
job worker per core. With four emulated cores, 4 and 5 were folded onto
0 and 1, so two cores ran two workers each. Switch 1 titles keep using
cores 0..3; core 3 stays the shared system core.
Several debug switches were read with std::getenv on every macro call, QMD,
semaphore write or draw. The UCRT getenv locks and scans the environment,
thousands of times a frame; a long EDEN_NX2_KEY_SCRIPT made title 4 crawl
at 2 fps. Each is now read once. EDEN_NX2_KEY_SCRIPT=@<file> reads the
script from a file. gdbrsp.py gains 'hits' (log registers, frame chain and
object words at breakpoints, following the stopped thread).
NormalUnmapGpuMemory4K and destroying GPU memory are jobs the console's
graphics server completes behind the work already submitted. Run the GMMU
unmaps on the GPU thread after the command lists queued before them
(GPU::RunInOrder) and signal the job's event only then, instead of unmapping
and signalling at once while Eden's GPU thread may still be behind.
Also EDEN_NX2_FRAMES_AHEAD (experimental): wait for the previous frame's
fence at present instead of this one's.
ATOMG.E.ADD through a uniform pointer and STL/LDL.32 at RZ only took a zero
offset; title 4's loading screen uses [UR + 4] and [RZ + 4]. Maxwell ATOM
has a signed 20-bit offset at bits 28-47, STL/LDL a 24-bit one at 20-43.
Title 4 calls a 3D macro with a 6-word IncreaseOnce header, ends the main
GPFIFO entry after the first word and supplies the other five in a 5-word
subroutine entry. The split-method parking (an entry elsewhere was treated
as a call with a clean method state) fed those parameters to the parser as
headers: the macro ran with garbage, later draws read megabytes of unmapped
memory and the host driver timed out at the loading screen after the dock
scene. As on the hardware, the method state now simply continues; the old
guess stays behind EDEN_NX2_PARK_SPLIT_METHODS.
Debug aids: splits are logged at debug level with the words around them,
and a 3D write that interrupts a macro's parameters dumps the last 48 writes.
The real VFS kept up to 8192 host files open, which is also the most the
Windows C runtime allows (_setmaxstdio(8192)). A game read from a loose RomFS
folder fills that within minutes (title 4: ~7300 handles after 160 s); from
then on every other open failed with "too many files open" - its next scene
pack, its save file, the shader cache - and the game waited forever on the
load. Cap the cache 512 below _getmaxstdio() on Windows.
The console flips one frame per vblank and holds the rest in the swapchain,
which caps a game at 60 fps. vi2 229 reports every buffer slot free at once,
so nothing paced title 4: it presented at 90-160 fps and its frame-stepped
timing, audio included, ran ahead. Hold each present until the next vblank,
scaled by the speed limit, without banking time lost to slow frames.
Switch 2 channels that wait for an exact semaphore value poll on the console
and see every release. Eden resumes a waiting channel only between command
lists and writes releases when fences signal, often several back to back, so
an acquire-equal could miss its value and wait forever. Title 4's 3D queue
ticks a counter that two other queues wait on value by value; under the
loading after the post-customization save one of them missed a tick and the
game froze (channel 4 waiting for 0x5293 with the counter at 0x5310).
Pending acquires now register with Nx2SemaphoreWaiters before fences run,
and every query-cache payload write and QMD add/max release latches the
waiters it meets at the moment it is written.
The GPU ASTC decoder unswizzles block-linear data itself and never applied
the native (Switch 2) GOB swizzle that the other unswizzle paths use, so
ASTC textures came out with 16-byte blocks swapped wherever the image
varies. Title 4's UI is ASTC (10x5, 4x4, 8x8): portraits, buttons and the
squid cursor had hatched edges and clean interiors. Pass the native tiling
flag to the decoder and swap offset bits 5 and 6 as
block_linear_unswizzle_2d.comp does.
nx2:gpu 213 (NormalUnmapGpuMemory4K) and 102 (destroy GPU-accessible
memory) were no-ops, so Eden kept freed memory mapped for the GPU. Title 4
frees its customization-room memory right after saving and reuses it as
ordinary heap; pending GPU->CPU write-backs for the old buffers then landed on
the new objects, and two threads crashed on garbage pointers ~5 s later
(the post-customization freeze).
Unmap the range from every address space (the GMMU unmap drops cached
buffers/images and their pending downloads), keep the registry's mapping
list in step, re-map 211 requests even at the memory's own address, and
drop freed regions from the semaphore mirror table, which every 8 KiB
allocation used to fill for good after 32 entries.
EDEN_NX2_KEY_RECORD records keyboard transitions as "f<frame>:code" against a
global count of guest presents; EDEN_NX2_KEY_SCRIPT replays them on the same
frames (or by seconds, as before), so a played session reproduces exactly.
Debug aids: the PC/LR/SP of the first unmapped reads, NSO load addresses,
EDEN_NX2_IPC_SUMMARY (request counts every 5 s), EDEN_NX2_FORCE_LOD0.
Unmapped-access logging is rate limited and the per-draw/semaphore spam is
demoted, so long sessions stay under the log size cap.
- 64-bit adds: IADD3/UIADD3 with a carry-out predicate (unsigned compare of
the sum) and IADD3.X/UIADD3.X with a carry-in predicate (SEL + add); uniform
predicates go through R239 like UISETP. Used by a compute pass the game runs
thousands of times while loading the next area.
- TMML (0xb69) LOD query, 2D: 8.8 fixed-point results as on Maxwell.
- Vertex stores to a[0x5c] (an Ampere per-primitive hint, likely shading rate)
are dropped; Eden had no attribute for it and refused the pipelines.
All 864 shaders from a 5-minute scripted session past the intro now lower,
except the opt-in tensor-core upscaler (bilinear stand-in).
Native shader capture counted CALL.REL.NOINC as a function call, so a program
using it waited for a RET after its EXIT and read on into the following
shaders until a garbage branch target made it fail. NOINC calls push no
return address (the compiler uses them as jumps; the lowering already treats
them so): count them as forward branches.
That shader builds title 4's 3D colour-grading LUT once per scene; with it
rejected the LUT stayed empty and the tonemap turned every frame black. The
intro cutscene now renders lit and graded, with subtitles, through the
bilinear stand-in for the tensor-core upscaler.
Also: rejected captures dump to EDEN_NX2_SHADER_DUMP as rejected-<addr>.bin
and report the offending instruction.
