std::chrono::microseconds::rep is long on Linux and long long on Windows,
so std::max over the long long stats and a converted duration did not
compile with GCC. Return long long from the conversion.
NVN2 gives a texture in a compressible memory pool its GPU address only
after NvRmGpuMappingModify switched the pool's mapping to the texture's
kind. NvRm fails that call unless the GPU characteristics report
NVGPU_GPU_FLAGS_SUPPORT_MAPPING_MODIFY, so such textures kept address 0
and draws into them went nowhere: title 1's cup-opening movie stayed on
its first frame (also its 640x360 and 512x512 passes).
Report the flag and answer the request nn::gpu::sdk then sends, gpu
sub-session 212 (NormalModifyMemoryKind4K), with the wanted kind and a
signalled job slot. The movie now plays on screen and the race follows.
Switch 2 titles size their heap from the total memory and stop at the
first unimplemented command, so they only booted with the 12 GiB layout
and EDEN_NX2_AUTO_STUB set by hand. Detect them from main.npdm before the
kernel starts and apply the layout as a per-game value (the configured
layout comes back once the game is closed), and auto-stub unknown
commands from Switch 2 processes.
- (std::min)/(std::max) where windows.h's macros are visible
- LAMBDA_FORCEINLINE: always_inline under clang (clang-cl defines
_MSC_VER but rejects [[msvc::forceinline]] on lambdas)
- ImageBase::FindView: find_if, as MSVC's STL takes its vectorized find
for ImageViewInfo under clang-cl and fails a size assertion
- parentheses for -Wlogical-op-parentheses in the Ampere lowering
At shutdown is_powered_on is cleared before the GPU thread is stopped.
A channel suspended on a semaphore acquire that can resume then went
through DispatchCalls, which returns at once when powered off, so the
acquire stayed pending and ResumePending counted it as progress forever:
the GPU thread never got back to its stop request and closing the game
hung in ThreadManager::NotifyShutdown.
The decoder service used to complete every submitted frame at once
without decoding it. Each decoder service (gsv 902/906/1002 -> TIPC 1)
now decodes on a thread of its own with FFmpeg (VP9, VP8, H.264;
video_core/host1x/nx2_video_decoder), in software or on the host GPU per
the NVDEC emulation setting.
- 101 takes the codec name ("video/vp9") from its A buffer.
- 109 queues the A buffer's frame; the ready event fires while the queue
has room.
- 311/314 reply a NativeSyncHandle {~syncpoint, 0, value}: the service
allocates a syncpoint and increments it per finished frame, frames
filling output images in register/release order like the client's
ring. 314 waits for the client's release fence before reuse.
- 213 returns the frame's size, pts (TimeSpan, ns) and end of stream.
- gsv 911 creates a decoder image pool (count images in one memory) and
gsv 961 returns the 128-byte CpuUserImageDescriptor that
nn::codec::ReadDecoderOutputImage* copy frames out with; images are
pitch-linear NV12, RGBA or BGRA.
- Each new video command is logged once in full; EDEN_NX2_VDEC_DUMP=<dir>
saves every 120th decoded frame's luma.
Title 1's Mushroom Cup opening (2560x1440 VP9, 1423 frames) now decodes
in real time and the game goes on to the race. The picture on screen
still stays on the first frame: the game memcpys each frame into GPU
memory and the texture cache keeps its first upload (notes 148).
nn::syncpt::sdk's CancelSyncPointEvent sends 202 and then always waits on
the slot's event, so 202 now signals it also when no request was pending.
Title 1's movie player cancels each output-buffer slot before its first
request and hung there forever (the cup-opening screen stayed black and
the decoder was never started).
Slots are reused by different client objects: the movie player got slot
0 after NVN's fence waits had left its event signalled, read that as a
decoded frame, found none and aborted with 2111-3604. 201 now clears the
slot's event before registering the new request.
Newer SDKs (nn::codec::video::detail::InitializeServerContext, title 4)
send 906(u8, u64 decoder library version) where older ones send 902.
Auto-stubbed, it returned no handle and title 4 aborted with
InvalidHandle (0xE401) when its first movie started, about 30 minutes
into a new game. 906 now hands out the same session as 902.
FixedPipelineState stored alpha_test_ref whatever the alpha test
function. Title 1 animates the register with the test off (text fading
in and out), so every value made a new pipeline whose shaders were
lowered and translated on the GPU thread: bursts of 600-1,900 pipelines
in 10 s (up to 5 s of GPU thread) whenever text was on screen. The
reference is now kept only while the test compares with it (not Always
or Never). Driving title 1's menus: 400+ pipelines for already-known
shaders before, 0 after; one shader lowered 79 times before, 2 after.
EDEN_NX2_PIPELINE_DIFF=1 logs which fixed-state fields differ when a new
pipeline reuses known shaders.
Clean A/B in title 1 (3 rounds each): 1.46x the draws a second of one
thread, frames 17% shorter, no crashes, front register copies equal to
the back's in 57,089 checks. EDEN_NX2_SPLIT=0 runs everything on one
thread, =1 keeps macros on the back.
The front's per-write paths no longer check guarded statics
(GpuSplit::Get, Nx2Profile::Enabled, the skip-same switches) and the
ring's method writes are inline. Notes row 145.
The GPU thread (front) takes the CPU's commands, parses command lists
and runs the puller; engine method calls and everything behind them
(registers, draws, dispatches, the rasterizer and its caches) run in
order on a second thread (back), fed through a single-producer
single-consumer ring (gpu_split.h). Work the front did on back-owned
state (end of command list flushes, channel binds, semaphore releases,
syncpoint increments, flush/invalidate requests, presentation) is
deferred into the same ring; the front waits for the back only where it
needs a result from that state: semaphore acquires and MME DMA reads.
EDEN_NX2_SPLIT=1 runs macros on the back. EDEN_NX2_SPLIT=2 runs the 3D
and compute macros on the front against copies of the registers they
read (Maxwell3D/KeplerCompute Front*), drops state writes that keep a
register's value there, and sends the rest. EDEN_NX2_SPLIT_VERIFY=1
compares the copies with the back's registers every 4096 macro calls
(0 differences in 40,193 checks in title 1); EDEN_NX2_SPLIT_CHECK=1
logs the stack of any other thread using the Vulkan scheduler while the
back runs. Switch 2 titles, Vulkan, non-accurate GPU only.
Title 1, 3D attract loop: ~44,000 draws a second on one thread, ~47,400
with EDEN_NX2_SPLIT=1, 50,000-66,500 with EDEN_NX2_SPLIT=2.
Swapchain::AcquireNextImage waits for the main scheduler's tick of the
image's last use; when that tick was not submitted yet, Scheduler::Wait
flushed, recording into the current chunk on the present thread while
the GPU thread recorded into it too (corrupt command buffers, crashes
in vkCmdEndRenderPass and in CommandChunk::Record). With async
presentation the present thread now waits up to 20 ms for the
recording thread to submit the tick, and goes on without the wait
otherwise (the copy to the swapchain has its own fences).
EDEN_NX2_GPU_PROFILE now times engine method calls (the work a split GPU thread's back half
would run) and counts semaphore acquires and those that wait. Title 1: method calls ~343 ms
a second (6.9 million), front-half work (command parsing, macros) ~333 ms, acquires ~2,700 a
second of which ~220 wait. Notes 140 (macro compiler) and 141 (draw setup).
Title 1 calls macros ~1.3 million times a second; most were replayed from the macro memo,
whose lookups missed the CPU caches inside the GPU thread (~8%), and the rest interpreted.
macro_turing_jit.h compiles all of the uploaded macro memory with Xbyak: each instruction a
block falling through to the next, branches and loops through a jump table, with the
interpreter's semantics (lanes reading operands before either writes, predication as two
variants, the END delay slot, loops, carry). Loop checks only where a loop can end, END
checks only after an END, the instruction limit counted on indirect jumps. Emits and register
reads call back into C++. It is recompiled when the macro code changes and used for 3D and
compute macros on x86-64; EDEN_NX2_MME_JIT=0 goes back to the memo and the interpreter.
Checked against the interpreter on 200,000 recorded calls (registers, data RAM, every
method write, FIFO use): identical. In the game: macro calls 148 ns instead of 174 (two
profiled runs each), no failures, 0 crashes in 3 runs. NX2_GPU_THREAD_SPLIT.md describes
the next step, splitting the GPU thread.
The buffer collector compared the whole device's memory use with thresholds from the
device size, so with title 1's textures filling an 8 GB GPU (~6.8 GB used, buffers ~200
MiB of it) it ran aggressively every frame: up to 64 buffers evicted and recreated later,
the GPU-written ones downloaded first with a wait for the GPU (~5% of the GPU thread).
For Switch 2 titles it now runs when the buffers themselves pass 1 GiB (aggressive past
2 GiB); below the aggressive mode it also keeps buffers holding GPU-written data, which
it would have to download. Other titles keep the device-wide thresholds. Buffer
downloads by the collector: 5.4% -> 0%; the cache settles around 540 MiB.
ProcessMethodCall and the draw manager behind it act only on executable methods, so
CallMethod stops at the dirty flags for the rest (macro replays write ~2.4 million of
those a second). No change beyond run-to-run noise in two profiled runs; it only removes
work. Notes: row 133 corrected (those courses are bound by the GPU thread, not the host
GPU: 20-40% busy), rows 135-137 (same-value writes, native compute registers, 2D blits).
Most native compute register writes (QMD fields its macros replay for thousands of
dispatches a second) only store the value; they went through ProcessNativeMethod's
macro, QMD, upload and semaphore checks. CallMethod now stores them through a table of
plain methods. ProcessNativeMethod's own time 2.2% -> 1.6% of the GPU thread.
Title 1 makes ~6 million 3D register writes a second and ~60% of them write the value
the register already holds (counted with EDEN_NX2_GPU_PROFILE), most from macro memo
replays. For registers without side effects (not in execution_mask: draws, clears,
constant buffer data and binds, semaphores, ...) such a write changes nothing, but it
set dirty flags that made the next draw re-check that state. CallMethod and the sunk
writes now skip them after shadow RAM processing. Two profiled runs each: macro calls
169 ns instead of 185, draw setup 5.80 us instead of 5.95. EDEN_NX2_SKIP_SAME=0 turns it
off.
The buffer collector downloaded each evicted buffer's GPU-written data with its own
wait for the GPU. Without per-fence downloads (d530bc4) most of title 1's evicted
buffers are GPU-written, and on an 8 GB GPU near its budget it evicts some every frame.
Their ranges now go to one staging buffer, copied with one wait.
Title 1 makes thousands of small inline-to-memory writes between draws, and each one
invalidated the buffer, texture, shader and query caches on its own (~7.5% of the GPU
thread after 0fe581f). Guest memory is still written at once, query results pending
there are still dropped at once, and writes into GPU-written buffer data are still
uploaded in place; the buffer, texture and shader caches now get the merged ranges at
the next FlushCaching, which runs before every draw, dispatch, clear, copy and 2D blit.
Inline writes: 4.4% of the GPU thread (deferred invalidation included in FlushCaching,
3.9% -> 3.1%). EDEN_NX2_INLINE_DEFER=0 invalidates at once.
The collector's periodic log now also counts evictions that downloaded the image first
(each a full GPU wait). EDEN_NX2_TC_MIN_AGE=<frames> keeps images used more recently
than that (a VRAM thrash test on 8 GB: 120 frames made no difference over three runs).
Common::unordered_map/set defaulted to std::hash, a byte-wise FNV-1a on MSVC for
integers and pointers, which boost's flat containers then mix again: the GPU thread's
page maps (shader and query cache invalidation, per inline upload) spent ~1% in it.
Integer and pointer keys now go through as they are and boost mixes them.
BufferCache::IsRegionGpuModified visited every GPU-written range inside the query just
to answer yes or no; without per-fence downloads those ranges stay fragmented and it was
~4% of title 1's GPU thread (inline uploads). RangeSet::Intersects answers with one
lookup. Inline uploads: 10.5% -> 7.5% of the GPU thread.
Eden copied every GPU-written buffer range back to guest memory at the next fence.
Title 1 signals a fence per compute dispatch, so that was 250-330 MB/s in 13,000-20,000
ranges a second, with the fencing thread holding the buffer cache while it wrote them.
Measured: the native MME's DMA reads never touched those pages (they read query words
written on the CPU side), and with reactive flushing the guest CPU read GPU-written
memory at most 64 times in 200 s. Without them, for Switch 2 titles (native tiling):
fences ~1.1 us instead of ~5, draw setup 6.1 us instead of 6.4-6.7, macro calls 160 ns
instead of 167-170, GPU-thread lock spinning 1.8% (was 5.5%). Regions stay marked GPU
written and are still flushed on demand. EDEN_NX2_BUFFER_DOWNLOADS=0/1 forces either
way. Reactive flushing reads of GPU-written memory are now counted in the log.
Calls stored again with another shape leave their old record in the memo's arena;
title 1 filled its 16 MiB every ~2 s and clearing it made the next ~20,000 calls
interpret again. The live records are now copied to a new arena (a clear only when
they alone fill three quarters of it). The memo statistics count hits on the last
call's slot and compactions. With EDEN_NX2_GPU_PROFILE the GPU thread also reports
how much downloaded buffer data lands on pages the native MME has DMA-read.
Every 8 draws the GPU thread handed a command chunk to the Vulkan worker and woke it
with a system call (~2% of the GPU thread at ~55,000 draws a second). Chunks now go
every 32 draws (a full chunk still goes on its own), and the worker is woken only
when it is waiting; a busy worker pops the queue before waiting again.
Title 1's main thread reads CNTPCT ~17 million times a second while it waits for the
GPU (~23% of its samples are the counter read). On an SMT core that spin takes
resources from the sibling thread, often the GPU thread. Two reads within ~1.7 us of
each other now execute two PAUSE instructions first; the guest sees the same values.
EDEN_NX2_CNTPCT_PAUSE=0 turns it off.
Nearly every macro call arrives whole in one method run (~1,000,000 a second in title
1); its parameters were appended to a std::vector first (~2% of the GPU thread). They
are now copied to a stack array. Still a copy: the command list is often guest memory
read in place, which the guest may reuse while the macro runs, and the macro memo
stores the parameters after the call. Accurate GPU mode keeps the vector (it rewrites
parameters in place). EDEN_NX2_MACRO_COPY=1 always uses the vector.
The fencing thread now takes every fence the host GPU has already passed (up to 64)
and pops their texture and buffer downloads under one lock acquisition per cache,
instead of one per fence (title 1 signals 5,000-7,000 a second; the GPU thread spun,
slept and woke the fencing thread for each). Queries and operations keep their
per-fence order. In the sampler, the GPU thread's spinning on the cache locks went
from 7.0% to 5.5%; frame times are within the attract loop's run-to-run noise.
EDEN_NX2_FENCE_BATCH=0 releases one fence at a time.
DeviceMemoryManager kept one four-entry page translation cache for every thread. Map
(on a guest service thread, e.g. nx2:gpu 211 job mappings) cleared it while the GPU
thread was between its null check of an entry and the copy from it: the GPU thread
then read from address 0 plus the page offset. Title 1 crashed this way in about half
of its 150 s test runs (BindHostGraphicsUniformBuffer -> ReadBlockUnsafe, fault at
0x600). Each thread now keeps its own cache, which Map and Unmap invalidate through a
generation counter. 0 crashes in 6 runs after.
Switch 2 titles signal a fence for each compute dispatch's release (5,000-7,000 a
second in title 1). With fast GPU fences the callback runs at once, so a fence with
no downloads, queries or queued operations does nothing: those (~20%) are no longer
made (one in 64 still is, for the operations fences carry). Each cache's lock is now
taken once per fence (the flush check took both together first). The overlap range
set's Subtract restarted its search after every erase (quadratic); the fencing thread
runs it for every downloaded range while holding the buffer cache. The GPU profiler
counts fences, skipped fences and buffer download bytes.
46% of title 1's macro calls repeat the previous call of the same
macro exactly: a per-start slot holding that record's offset skips the
hash and the map probe for them (the record is still in the CPU cache).
The interpreter's parameter FIFO was filled before the memo lookup; it
is now filled only for calls that are interpreted.
Checked against the previous interpreter on 200,000 recorded 3D-scene
calls: identical writes, registers and data RAM.
Translating title 1's 28,102-instruction compute shader took 3.4 s on
the GPU thread (a freeze the first time it runs), three quarters of it
in GlobalMemoryToStorageBufferPass:
- BreadthFirstSearch checked "visited" with a linear scan per argument,
quadratic once a search walks thousands of instructions; past 64 it
now uses a hash set (same visit order, same result).
- The pass searched again for every global memory access; accesses
through the same address now share the result (the IR does not
change while storage buffers are collected).
That shader now takes 0.85 s; other new pipelines also cost less.
- The texture and buffer caches' mutexes spin briefly before sleeping
(Common::SpinRecursiveMutex): the GPU thread, the fencing thread and
CPU write traps take them tens of thousands of times a second for
microseconds, and a sleeping waiter's wake-up took longer than that.
Title 1's GPU thread slept 7.7% of its time there, now 3.7%.
- Fences commit and pop texture and buffer downloads separately, and
only when there are any (one fence per compute dispatch in Switch 2
titles): the fencing thread no longer holds both caches while it
copies buffer downloads.
- Inline uploads and pre-draw invalidation take the caches one at a
time again: holding one while std::lock waited for the other waited
longer.
- MemoryManager::ReadBlockUnsafe reads within one 4 KiB page through a
single translation (as Read<T> does) instead of the page walk:
texture and sampler descriptors, several per draw (2.6% -> 0.1% of
title 1's GPU thread).
- UsageTracker skips ranges it tracked just before (until Reset): draws
bind the same multi-megabyte storage buffers again and again, and
each Track wrote a word per 4 KiB (2.0% -> 0.7%).
- Small compute uniform buffers the GPU has not written are streamed,
as graphics ones are; synchronizing them re-protected pages an inline
upload had just unprotected, two VirtualProtect calls per dispatch
(1.9% -> 0.7%, page protection 3.2% -> 2.3%).
SyncOperation appended to uncommitted_operations without a lock while
SignalFence moved that list into pending_operations under `guard` on
another thread; a list corrupted this way crashed the fencing thread.
They now share a lock of their own, held for no callbacks (SignalFence
runs callbacks under `guard`, so taking `guard` here could deadlock).
Native titles' thresholds came from Eden's device memory figure, which
is capped near 7 GB: on a tester's RTX 4070 SUPER (11.2 GB budget) title
1 kept evicting 800-1,600 textures per 600 frames with 3 GB unused. They
now follow the budget (critical at budget - 768 MiB, expected at budget
- 1536 MiB, re-read every 60 frames); the EDEN_NX2_TC_* overrides still
win, and the thresholds are logged only when they move by 256 MiB.
Title 1 calls both ~75 times a second with the display name "default".
The auto stub's reply (success, no data) works; it also logged two lines
per call, which made a tester's 18 minute log 50 MB. Same reply, no log.
It runs before most draws (constant buffer updates go through the
invalidation accumulator) and took the texture and buffer cache mutexes
one after the other.
- Compute engine macros replay repeated calls like 3D ones (title 1's
compute macro path fell from ~27% to ~14% of the GPU thread).
- AccelerateInlineToMemory takes the buffer and texture cache mutexes
together instead of one after the other.
- ShaderCache::InvalidatePagesInRegion looks at the rejected programs
only when the write can reach one (bounds kept as they are added);
it ran a locked scan for every inline upload and pre-draw
invalidation.
Inline uploads went from ~10% to ~4% of the GPU thread.
The flat map re-hashed the already mixed u64 keys with MSVC's byte-wise
FNV-1a (~1.4% of the GPU thread), and calls stored again after a data
RAM guard failed appended new records, so the arena filled with stale
copies and was cleared every few seconds. EDEN_NX2_MME_MEMO_STATS logs
the counters: in title 1, 97.3% of calls replay, 2.1% fail a guard.
- A channel waiting on a semaphore the CPU releases slept 1 ms per poll
(~25 waits a second in title 1); poll for 0.5 ms first.
- The GPU profile zones read the wall clock (the TSC) instead of
QueryPerformanceCounter, which cost ~5% of the GPU thread when on.
