diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 422efb5fdc..bdf9d048e8 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,5 +1,4 @@ - [ ] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions). -- [ ] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md). - [ ] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability. ------------------- diff --git a/.gitignore b/.gitignore index 20ed5e93c1..873b99bec2 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ # Build directory /[Bb]uild*/ doc-build/ +out/ AppDir/ uruntime diff --git a/CMakeLists.txt b/CMakeLists.txt index fcd3fa19ae..f647204a4b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -304,16 +304,6 @@ if (ARCHITECTURE_arm64 AND (ANDROID OR LINUX)) add_compile_definitions(HAS_NCE=1) endif() -set(ENABLE_SCARLETNX_DEFAULT ON) -if ((NOT LINUX AND NOT FREEBSD) OR (NOT CMAKE_C_COMPILER_ID MATCHES "Clang" OR NOT CMAKE_CXX_COMPILER_ID MATCHES "Clang")) - set(ENABLE_SCARLETNX_DEFAULT OFF) -endif() -option(ENABLE_SCARLETNX "Enable the ScarletNX recompiler CPU backend (requires Clang and LLVM)" "${ENABLE_SCARLETNX_DEFAULT}") -if (ENABLE_SCARLETNX) - set(HAS_SCARLETNX 1) - add_compile_definitions(HAS_SCARLETNX=1) -endif() - if (YUZU_ROOM) add_compile_definitions(YUZU_ROOM) endif() diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8c16de9ddb..c87dd11dbe 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,15 +16,14 @@ Eden is free, open-source, copyleft software, licensed under the terms of the [G ### Policies -- We are trialing a slightly relaxed AI policy to allow high-quality contributors to use AI *responsibly*. Please read our [detailed AI policy](docs/policies/AI.md) for more info. -- Usage of any form of profanity or otherwise unsavory language is generally discouraged. - - This is primarily because it rarely helps to actually understand what's going on. - - Remember that your comments should be focused and actually address what's happening. With very few exceptions, expletives are actively detrimental at best. -- New code must follow the same general style as the surrounding codebase. Exceptions may be granted in certain cases. +- AI usage is allowed, but we reserve the right to close and rework your PR if you don't fully understand the changes you're making. + - As a common courtesy, please disclose your AI usage under the PR description, as `Assisted-by: ()`. +- Usage of any form of profanity or otherwise unsavory language is discouraged but not strictly disallowed. +- New code must follow the same style as the surrounding codebase. Exceptions may be granted in certain cases. - Maintainers reserve the right to change your patches and pull requests at will. We will try to avoid this. - You should generally respect all decisions made by the [code owners](docs/CODEOWNERS) in your particular subsystem. - However, if you feel they are overstepping or are incorrect, don't be afraid to stand your ground! Maintainers are not always correct. -- While we do *not* adhere to the terms of a formal code of conduct, you will generally be expected to respect other developers, contributors, and community members. +- While we do *not* adhere to the terms of a formal code of conduct, you are expected to treat other developers, contributors, and community members respectfully. - You **must** have basic knowledge of [Git](https://git-scm.com/learn). Knowing how to manage your branches and follow proper fork policies is a necessity. ### Where do I start? @@ -38,7 +37,8 @@ You may also search for open issues on [GitHub](https://github.com/eden-emulator There are five primary ways to contribute. - Submit pull requests directly to our source tree - - You must first request an account on our Forgejo. See [Signup](#how-do-i-sign-up) + - You must first [create an account](https://git.eden-emu.dev/user/sign_up). + - You may use an internal account, or you may use a GitHub/Codeberg account linked to our instance. - Submit pull requests to our [GitHub mirror](https://github.com/eden-emulator/Issue-Reports) - Note that this is subject to being removed by DMCA, so don't rely on this. - Submit pull requests to our [Codeberg mirror](https://codeberg.org/eden-emu/eden) @@ -50,31 +50,8 @@ There are five primary ways to contribute. To test your changes, ensure to read the [build documentation](./docs/Build.md) for your specific platform(s) to ensure everything compiles and works properly. You should also make sure that your branch is up-to-date with the upstream `master` branch before opening a pull request. -### How do I sign up? - -
-To sign up and begin contributing... (click to open) - -Email [crueter@crueter.xyz](mailto:crueter@crueter.xyz) with the following format: - -```txt -Subject: [Eden Git] Registration Request -Username: -Email: -I wish to sign up because... -``` - -Received mail that does not follow this format will be ignored. - -Once your request is processed, you will receive a confirmation email with your temporary password and some information on Git access and policies. If you do not receive a response within 48 hours, feel free to send another email. - -> [!WARNING] -> Some email providers may place the response email in your spam/junk folder; notable offenders include Gmail and Outlook. *Always* ensure to check your Spam or Junk folder. - -
- ## Non-code Contributions Alongside the other contribution methods listed up top, you can also choose to contribute through documentation, organization, or community guides. These can be done either through the code contribution methods described above, or created externally and shared via our Discord community. -If you have an external tool/page that you believe would be handy to integrate/link into Eden, please additionally email our developers at [`developers@eden-emu.dev`](mailto:developers@eden-emu.dev). **Do not submit vibe-coded or AI generated tools or applications**. +If you have an external tool/page that you believe would be handy to integrate/link into Eden, please additionally email our developers at [`developers@eden-emu.dev`](mailto:developers@eden-emu.dev). diff --git a/cpmfile.json b/cpmfile.json index a4d8f0b0e3..0032734712 100644 --- a/cpmfile.json +++ b/cpmfile.json @@ -243,27 +243,6 @@ "repo": "crosire/reshade", "version": "v6.8.0" }, - "scarletnx": { - "bundled": true, - "hash": "93f7e6810644253d6396239092d5769c44b005fa7a17cf39c6d773050a19de74fca1f96bb6eec7cb40384eb158b2595c9e1171ad592d8eb8326fd31f5632f505", - "options": [ - "SNX_BUILD_NX OFF", - "SNX_BUILD_GNUABI OFF", - "SNX_BUILD_3DS OFF", - "SNX_BUILD_VITA OFF", - "SNX_BUILD_AGB OFF", - "SNX_DEPLOY OFF", - "SNX_BUILD_TESTING OFF", - "SNX_BUILD_GUI OFF", - "SNX_MULTICALL OFF", - "SNX_TRACY OFF", - "SNX_RECOMPILER_SHARED OFF", - "SNX_LLVM_SHARED OFF" - ], - "package": "ScarletNX", - "repo": "scarletnx/ScarletNX", - "version": "f16810ce9c" - }, "sdl3": { "hash": "df5a323af7ac366661a3c0e887969c72584d232f3cc211419d59b0487b620b6b2859d4549c9e8df002ee489290062e466fcfddf7edc0872a37b1f2845e81c0f3", "min_version": "3.2.10", @@ -372,13 +351,13 @@ }, "zstd": { "find_args": "MODULE", - "hash": "cc5ad4b119a9c2ea57f0b71eeff01113bb506e0d17000159c5409cb8236d22e38c52d5e9e97e7947a4bf1b2dfc44b6c503ab2d9aedbd59458435c6a2849cb029", + "hash": "2b006c3c110c38bd2f5896b42d36a38931c8eae261f30552f9a9fd04458d218d8f1a1425f117fd12af545ed77042e4b2d1a1a9b6286c4d0b9d74ef79f20a8f08", "min_version": "1.5", "options": [ "ZSTD_BUILD_SHARED OFF" ], "repo": "facebook/zstd", "source_subdir": "build/cmake", - "version": "b8d6101fba" + "version": "33618c8" } } diff --git a/docs/Options.md b/docs/Options.md index 03527d3464..ece6bdeba6 100644 --- a/docs/Options.md +++ b/docs/Options.md @@ -78,9 +78,6 @@ The following options are desktop only. - `ENABLE_OPENGL` (ON) Enable the OpenGL graphics backend - Unavailable on Windows/ARM64 and on Android - You probably shouldn't turn this off. -- `ENABLE_SCARLETNX` (OFF) Enable the ScarletNX recompiler - - Requires Clang and LLVM's development files - - Needs multicore and fastmem ### Qt diff --git a/docs/policies/AI.md b/docs/policies/AI.md deleted file mode 100644 index dd4b4a7fb2..0000000000 --- a/docs/policies/AI.md +++ /dev/null @@ -1,203 +0,0 @@ -# AI Policy - -> [!WARNING] -> We are trialing a slightly relaxed AI policy to allow high-quality contributors to use AI *responsibly*. -> This policy is subject to change at any point in time, so check back frequently. -> -> This banner will be removed once we have finalized our revised policy. - -- [AI Policy](#ai-policy) - - [TL;DR](#tldr) - - [Definitions](#definitions) - - [Scope](#scope) - - [Introduction](#introduction) - - [Requirements](#requirements) - - [Sign-off, Authorship, and Copyright](#sign-off-authorship-and-copyright) - - [Permitted Usage](#permitted-usage) - - [Recommended Prior Discussion](#recommended-prior-discussion) - - [Forbidden Usage](#forbidden-usage) - - [Human Effort Denial of Service](#human-effort-denial-of-service) - - [Disclosure](#disclosure) - - [Codebase Ingestion](#codebase-ingestion) - - [Enforcement](#enforcement) - - [Addendum: AI Commit Messages](#addendum-ai-commit-messages) - - [References](#references) - -## TL;DR - -AI-assisted contributions are welcome, provided you take responsibility for them. - -- You **must** disclose any AI or LLM usage in your pull request description *and* commit message. -- You **must** be able to explain, defend, and revise every line of code you submit. -- Commit messages, PR descriptions, issues, and comments **must** be written by you. Verbatim AI output is not acceptable. -- Undisclosed AI usage will result in your contribution being rejected. -- Low-quality contributions, disclosed or not, will be reworked or rejected. -- Spamming automated PRs, comments, or issues is treated as a Denial of Service (DoS) attack on our reviewers; see [Human Effort Denial of Service](#human-effort-denial-of-service). - -## Definitions - -An *AI or LLM-based tool* is any piece of software that generates code, text, or other content through machine learning models. This includes, but is not limited to: chat assistants, agentic coding tools/harnesses, inline code completion suggestions you have accepted into the code, and AI-driven explanation/refactoring features. - -For the purposes of this policy, basic deterministic/static development tools (such as clang-tidy, git, compilers, etc.) are not considered AI tools. - -Terms such as "substantially", "large-scale", "majority", and "low-quality" are intentionally left vague, and will be interpreted at the maintainers' discretion. When in doubt, ask first. - -## Scope - -This policy applies to all code contributions, alongside issues and pull requests surrounding this codebase. This includes: - -- Commit messages -- PR descriptions -- Code review comments -- Issue/PR comments -- Patches submitted via e-mail - -Areas that are not directly related to code--such as our surrounding communities--are not covered by the terms of this policy. Additionally, spelling/grammar checking via AI or LLM-based tools is not covered under this document. - -## Introduction - -In the right hands, AI and LLM-based tools are incredibly useful for getting features and bugfixes out the door as fast as possible. At the same time, reviewer bandwidth is a scarce resource. We are therefore open to AI-assisted contributions, provided they are produced and disclosed properly. - -The goal of this document is *not* to set arbitrary restrictions, but rather to keep our development healthy and stable, while still allowing contributors to use the tools they find helpful. A contribution must be worth more to the project than the time it takes to review it. - -Throughout this document, **must** and **must not** are inflexible, binding requirements. Refusal to follow these is treated as a violation of the terms outlined in this document. - -## Requirements - -These apply to all contributions, whether AI-assisted or not. AI-assisted contributions also carry additional disclosure requirements as outlined in [Disclosure](#disclosure). - -1. *Disclosure*. Any and all usage of AI or LLM-based tools within any contribution **must** be explicitly marked and disclosed as appropriate. -2. *Responsibility*. The human submitter of any contribution is held wholly responsible for the accuracy, completeness, and intent of what is submitted. You **must** be able to explain, defend, and revise your code during review. -3. *Review*. All code **must** be thoroughly reviewed by the author before opening a PR or marking it ready for review. -4. *Testing*. All code changes **must** be tested directly by the human author. -5. *Quality*. All submitted code is held to a strict standard of excellence. Low-quality AI-generated content does not uphold this standard. -6. *Licensing*. All code **must** be compatible with the GPLv3 or later. You **must** warrant that you have the right to submit it under those terms, and that it was not copied from work published under an incompatible license. - -## Sign-off, Authorship, and Copyright - -AI agents **must not** add `Signed-off-by:` tags. The human submitter **must** sign off on all changes. - -Similarly, AI agents cannot hold authorship or copyright over code, and **must not** add `Co-authored-by:` or `Co-developed-by:` tags. The human submitter **must** author all changes. - -## Permitted Usage - -This is a non-exhaustive list of permitted uses of AI and LLM-based tools, provided they are disclosed properly. - -- Drafting issues and pull requests, provided the final text and submitted code are substantially your own. -- Debugging issues, bugs, glitches, and crashes, whether pre-existing or introduced by your own changes. -- Reviewing your own changes before submitting a PR. -- Research, brainstorming, and codebase searching/understanding. -- Applying tedious but not particularly large changes, such as mass-migrating specific classes to polyfills or abstractions. - -## Recommended Prior Discussion - -Some usage cases are not necessarily forbidden nor permitted. These should be discussed with maintainers in an issue or e-mail *before* submission. Maintainers and reviewers may also comment on this during the review/submission phase. - -- Generating boilerplate/scaffolding for new code. -- Writing unit tests. -- Large-scale refactors primarily driven by an AI agent. -- Any contribution where an AI agent or model was responsible for the majority of the implementation. - -## Forbidden Usage - -This is a non-exhaustive list of strictly forbidden uses of AI and LLM-based tools. - -- Vibecoding, or changes entirely authored by AI agents without human intervention or understanding. - - If you don't choose to put in the necessary effort to understand every line of code you submitted, neither will we. -- Fully AI-generated commit messages, PR descriptions, issues, or comments. See [Addendum: AI Commit Messages](#addendum-ai-commit-messages). -- Submission of AI-assisted work that is not disclosed per the [Disclosure](#disclosure) requirements. -- Low-quality submissions that fail to satisfy our standard of code excellence. -- Submitting secrets, credentials, private user data, or unpublished security reports to third-party AI services. - - This includes console keys, encryption material, and private vulnerability disclosures. -- Verbatim AI code reviews (e.g. Codex/Copilot reviews). - -## Human Effort Denial of Service - -Using automated tools and AI to submit PRs, issues, and comments that we have to carefully review and handle is considered a [Denial-of-service attack](https://en.wikipedia.org/wiki/Denial-of-service_attack) on our human effort. - -It takes very little effort on your side (an LLM prompt) to generate a large amount of effort on ours (careful and thorough code review). Please respect our time and don't do that. - -## Disclosure - -How you disclose AI usage is up to you. However, it **must** be explicit, easy to identify, and placed both within the commit message *and* the surrounding PR description. For patches submitted by e-mail, the cover letter counts as the PR description. We recommend an `Assisted-by:` footer: - -```txt -Assisted-by: () -``` - -For instance: - -```txt -Assisted-by: Claude Opus 5.5 (initial draft and code review) -``` - -Contributions without a commit message (e.g. issues/comments) should instead use a plain sentence in the text body such as `I used to help with `. PRs and commit messages may opt to use this format as well. - -Multi-commit series/squashes **must** maintain the trailer. Maintainers will handle this for merges to master, but you are responsible for ensuring the disclosure survives into the final commit/PR body of squashes within your branches. - -Regardless, what matters is that reviewers can tell *at a glance* where and how AI was involved in your contribution. - -## Codebase Ingestion - -Ingestion of this codebase by LLMs for any purpose--such as training or prompt context--is permitted *for now*. This may change at any point as litigation and court rulings around AI and code copyright proceed. - -AI models may have ingested AGPLv3 or otherwise copyleft-incompatible code and may regurgitate it. You remain responsible for ensuring that all code you submit is compatible with the GPLv3 or later. - -## Enforcement - -- If we suspect undisclosed AI usage in a contribution, we will ask for disclosure. -- Refusal to provide disclosure will result in rejection of the contribution, and your PR or issue will be closed. Maintainers may choose to rework it themselves if the change is worth having. -- Accounts that spam us with repeated automated PRs or comments will be blocked or permanently suspended from our Git instance. -- Maintainers reserve the right to close your issues and pull requests if you don't follow the guidelines and procedures outlined in this document, and refuse or fail to make changes as necessary to comply. -- Repeated failures to comply may result in loss of contribution privileges. - -## Addendum: AI Commit Messages - -The patchset for pull request [#3422](https://git.eden-emu.dev/eden-emu/eden/pulls/3422) was fed into a dedicated code AI model, and it produced the following: - -```txt -profile_manager: Refactor user deletion to use index instead of UUID - -This commit improves the user profile deletion flow in the following ways: - -1. **UI layer (ConfigureProfileManager)**: - - Replaces UUID-based deletion with index-based deletion - - Passes the selected row index from the tree view to the confirmation dialog - - Emits deleteUser(int index) signal from ConfigureProfileManagerDeleteDialog - - Removes lambda capture and std::function callback in favor of direct signal/slot connection - -2. **Delete dialog**: - - Changes SetInfo() signature to accept index instead of accept callback - - Stores index internally and emits it via deleteUser signal on acceptance - -3. **Core ProfileManager**: - - Refactors RemoveUser(UUID) → RemoveProfileAtIndex(index) - - Removes duplicated stable_partition + save logic from RemoveUser() - - Centralizes profile removal and save-file writing in RemoveProfileAtIndex() - -4. **Minor follow-up**: - - Adds profile_manager.ResetUserSaveFile() call in PopulateUserList() to ensure consistent state when refreshing the user list - -These changes eliminate redundant code, reduce the risk of UUID → index mismatch bugs, and make the deletion path more straightforward and maintainable. - -Fixes: potential inconsistency when user order changes between UI and backend -``` - -This: - -- Is needlessly verbose (nobody cares about most of these details) -- Doesn't address the actual purpose of the PR (fixing a double-deletion bug in the profile manager) -- Has unicode arrows (this is bad for command-line editors) -- Uses corporate and word-salad language that pads out the message to make it look like a lot changed - -A good commit message should be concise, descriptive, and convey the *intent* behind the change, such as the bug being fixed or the feature being added. It should *not* be a detailed changelog of every individual byte that's changed. - -## References - -This policy document has been adapted from the AI policies of numerous other free and open-source projects, whether restrictive or permissive. - -- [Gitea](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#ai-contribution-policy) -- The [Linux kernel](https://kernel.org/doc/html/next/process/coding-assistants.html) -- [Gentoo Linux](https://wiki.gentoo.org/wiki/Project:Council/AI_policy) -- [cURL](https://curl.se/dev/contribute.html#on-ai-use-in-curl) -- [FastAPI](https://tiangolo.com/open-source/contributing/#automated-code-and-ai) diff --git a/externals/CMakeLists.txt b/externals/CMakeLists.txt index 70b9fc5434..cb1deceb96 100644 --- a/externals/CMakeLists.txt +++ b/externals/CMakeLists.txt @@ -249,14 +249,6 @@ if (YUZU_TESTS OR DYNARMIC_TESTS) AddJsonPackage(catch2) endif() -# ScarletNX -if (ENABLE_SCARLETNX) - AddJsonPackage(scarletnx) - if (TARGET scarletnx-recompiler) - set_target_properties(scarletnx-recompiler PROPERTIES SYSTEM ON) - endif() -endif() - # getopt if (MSVC) add_subdirectory(getopt) diff --git a/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/BooleanSetting.kt b/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/BooleanSetting.kt index 0548004cf5..6c33fc7812 100644 --- a/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/BooleanSetting.kt +++ b/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/BooleanSetting.kt @@ -16,7 +16,6 @@ enum class BooleanSetting(override val key: String) : AbstractBooleanSetting { RENDERER_USE_SPEED_LIMIT("use_speed_limit"), USE_CUSTOM_CPU_TICKS("use_custom_cpu_ticks"), SKIP_CPU_INNER_INVALIDATION("skip_cpu_inner_invalidation"), - NCE_INVALIDATION_GPU_READBACK("nce_invalidation_gpu_readback"), FIX_BLOOM_EFFECTS("fix_bloom_effects"), EMULATE_BGR565("emulate_bgr565"), RESCALE_HACK("rescale_hack"), diff --git a/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/view/SettingsItem.kt b/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/view/SettingsItem.kt index 690c746628..f968a0ae18 100644 --- a/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/view/SettingsItem.kt +++ b/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/model/view/SettingsItem.kt @@ -859,13 +859,6 @@ abstract class SettingsItem( descriptionId = R.string.skip_cpu_inner_invalidation_description ) ) - put( - SwitchSetting( - BooleanSetting.NCE_INVALIDATION_GPU_READBACK, - titleId = R.string.nce_invalidation_gpu_readback, - descriptionId = R.string.nce_invalidation_gpu_readback_description - ) - ) put( SwitchSetting( BooleanSetting.FIX_BLOOM_EFFECTS, diff --git a/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/ui/SettingsFragmentPresenter.kt b/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/ui/SettingsFragmentPresenter.kt index f9185e39e6..824a54181e 100644 --- a/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/ui/SettingsFragmentPresenter.kt +++ b/src/android/app/src/main/java/org/yuzu/yuzu_emu/features/settings/ui/SettingsFragmentPresenter.kt @@ -554,7 +554,6 @@ class SettingsFragmentPresenter( add(HeaderSetting(R.string.hacks)) add(BooleanSetting.SKIP_CPU_INNER_INVALIDATION.key) - add(BooleanSetting.NCE_INVALIDATION_GPU_READBACK.key) add(BooleanSetting.FIX_BLOOM_EFFECTS.key) add(BooleanSetting.EMULATE_BGR565.key) add(BooleanSetting.RENDERER_ASYNCHRONOUS_SHADERS.key) diff --git a/src/android/app/src/main/java/org/yuzu/yuzu_emu/model/Game.kt b/src/android/app/src/main/java/org/yuzu/yuzu_emu/model/Game.kt index 6717894c93..f5f0cda0e9 100644 --- a/src/android/app/src/main/java/org/yuzu/yuzu_emu/model/Game.kt +++ b/src/android/app/src/main/java/org/yuzu/yuzu_emu/model/Game.kt @@ -100,7 +100,7 @@ class Game( companion object { val extensions: Set = HashSet( - listOf("xci", "nsp", "nca", "nro") + listOf("xci", "nsp", "nca", "nro", "nsz", "xcz") ) } } diff --git a/src/android/app/src/main/java/org/yuzu/yuzu_emu/utils/GameHelper.kt b/src/android/app/src/main/java/org/yuzu/yuzu_emu/utils/GameHelper.kt index 411b764e36..bb2337a310 100644 --- a/src/android/app/src/main/java/org/yuzu/yuzu_emu/utils/GameHelper.kt +++ b/src/android/app/src/main/java/org/yuzu/yuzu_emu/utils/GameHelper.kt @@ -109,7 +109,7 @@ object GameHelper { // File extensions considered as external content, buuut should // be done better imo. - private val externalContentExtensions = setOf("nsp", "xci") + private val externalContentExtensions = setOf("nsp", "xci", "nsz", "xcz") private fun scanContentContainersRecursive( files: Array, diff --git a/src/android/app/src/main/jni/game_metadata.cpp b/src/android/app/src/main/jni/game_metadata.cpp index 49d92bca75..9b6cd44e9f 100644 --- a/src/android/app/src/main/jni/game_metadata.cpp +++ b/src/android/app/src/main/jni/game_metadata.cpp @@ -76,7 +76,9 @@ jboolean Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIsValid(JNIEnv* env, jobj auto const file_type = loader->GetFileType(); if (file_type == Loader::FileType::Unknown || file_type == Loader::FileType::Error) return false; - if ((file_type == Loader::FileType::NSP || file_type == Loader::FileType::XCI) && !Loader::IsBootableGameContainer(file, file_type)) + if ((file_type == Loader::FileType::NSP || file_type == Loader::FileType::XCI || + file_type == Loader::FileType::NSZ || file_type == Loader::FileType::XCZ) && + !Loader::IsBootableGameContainer(file, file_type)) return false; u64 program_id = 0; return loader->ReadProgramId(program_id) == Loader::ResultStatus::Success; diff --git a/src/android/app/src/main/res/values-ar/strings.xml b/src/android/app/src/main/res/values-ar/strings.xml index 2fdff9847b..3be04d9186 100644 --- a/src/android/app/src/main/res/values-ar/strings.xml +++ b/src/android/app/src/main/res/values-ar/strings.xml @@ -587,8 +587,6 @@ يجعل اللعبة تعتقد أن عمل وحدة معالجة الرسومات ينتهي بشكل أسرع مما هو عليه في الواقع، لذلك تتوقف عن خفض الدقة ومسافة العرض لتتناسب مع سرعات جهاز Switch. تخطي إبطال صلاحية وحدة المعالجة المركزية الداخلية يتخطى بعض عمليات إبطال ذاكرة التخزين المؤقتة من جانب وحدة المعالجة المركزية أثناء تحديثات الذاكرة، مما يقلل من استخدام وحدة المعالجة المركزية ويحسن أداءها. قد يتسبب ذلك في حدوث أعطال أو تعطل في بعض الألعاب. - إعادة قراءة وحدة معالجة الرسومات قبل إبطال صلاحية NCE - إصلاح مؤقت. يعالج مشكلة تعطل لعبة Ender Magnolia (وغيرها من ألعاب UE). قد يؤدي إلى انخفاض في الأداء. يجري العمل على تحسينه. إصلاح تأثيرات التوهج يقلل من ضبابية التوهج في LA/EOW (Adreno A6XX - A7XX/ Turnip)، ويزيل التوهج في Burnout. تحذير: قد يسبب تشوهات رسومية في ألعاب أخرى. محاكاة BGR565 diff --git a/src/android/app/src/main/res/values-zh-rTW/strings.xml b/src/android/app/src/main/res/values-zh-rTW/strings.xml index 3f0c6d9215..1391f1bc6e 100644 --- a/src/android/app/src/main/res/values-zh-rTW/strings.xml +++ b/src/android/app/src/main/res/values-zh-rTW/strings.xml @@ -577,8 +577,6 @@ 讓遊戲誤以為 GPU 工作能更快完成，避免遊戲為了配合 Switch 的時脈而降低解析度與渲染距離 跳過 CPU 內部失效處理 在記憶體更新期間跳過某些 CPU 端快取的失效處理，減少 CPU 使用率並提高其性能。可能會導致某些遊戲出現故障或當機 - 在 NCE 無效化前進行 GPU 回讀 - （暫時性功能）修復 \"終結者瑪格諾莉亞：霧中綻放\" 及其它使用虛幻引擎之遊戲中的當機問題，可能會降低效能。此功能正在改善中 修正光暈特效 在 \"薩爾達傳說：織夢島\" 和 \"薩爾達傳說：智慧的再現\" 中減少光暈模糊的問題並移除 \"橫衝直撞：狂飆樂園重製版\" 中的光暈效果（適用於 Adreno A6XX - A7XX/ Turnip）。警告：可能會在其它遊戲中導致圖形異常 模擬 BGR565 diff --git a/src/android/app/src/main/res/values/strings.xml b/src/android/app/src/main/res/values/strings.xml index 191506c818..33e1eaa340 100644 --- a/src/android/app/src/main/res/values/strings.xml +++ b/src/android/app/src/main/res/values/strings.xml @@ -593,8 +593,6 @@ Makes the game believe GPU work finishes faster than it does, so it stops lowering resolution and render distance to fit the Switch\'s clocks. Skip CPU Inner Invalidation Skips certain CPU-side cache invalidations during memory updates, reducing CPU usage and improving it\'s performance. This may cause glitches or crashes on some games. - GPU Readback Before NCE Invalidation - Temporary. Fix Ender Magnolia (and other UE titles) crashes. May reduce performance. Improvement in progress. Fix Bloom Effects Reduces bloom blur in LA/EOW (Adreno A6XX - A7XX/ Turnip), removes bloom in Burnout. Warning: may cause graphical artifacts in other games. Emulate BGR565 diff --git a/src/common/settings.h b/src/common/settings.h index fad3b0ca78..b57bec5c12 100644 --- a/src/common/settings.h +++ b/src/common/settings.h @@ -256,9 +256,7 @@ struct Values { CpuBackend::Dynarmic, #endif CpuBackend::Dynarmic, -#if defined(HAS_SCARLETNX) - CpuBackend::ScarletNx, -#elif defined(HAS_NCE) +#ifdef HAS_NCE CpuBackend::Nce, #else CpuBackend::Dynarmic, @@ -641,8 +639,6 @@ struct Values { Specialization::Default, true, true}; - SwitchableSetting nce_invalidation_gpu_readback{ - linkage, false, "nce_invalidation_gpu_readback", Category::RendererHacks}; SwitchableSetting async_presentation{linkage, #ifdef __ANDROID__ true, diff --git a/src/common/settings_enums.h b/src/common/settings_enums.h index 1d7c465254..eaf3ec541d 100644 --- a/src/common/settings_enums.h +++ b/src/common/settings_enums.h @@ -138,7 +138,7 @@ ENUM(RendererBackend, OpenGL_GLSL, Vulkan, Null, OpenGL_GLASM, OpenGL_SPIRV); ENUM(GpuAccuracy, Low, High); ENUM(DmaAccuracy, Default, Unsafe, Safe); ENUM(GpuFenceBehavior, Default, Immediate, Balanced, Accurate); -ENUM(CpuBackend, Dynarmic, Nce, ScarletNx); +ENUM(CpuBackend, Dynarmic, Nce); ENUM(CpuAccuracy, Auto, Accurate, Unsafe, Paranoid, Debugging); ENUM(CpuClock, Normal, Boost, Overclock) ENUM(MemoryLayout, Memory_4Gb, Memory_6Gb, Memory_8Gb, Memory_10Gb, Memory_12Gb); diff --git a/src/common/zstd_compression.cpp b/src/common/zstd_compression.cpp index cb6ec171b5..7665482ee6 100644 --- a/src/common/zstd_compression.cpp +++ b/src/common/zstd_compression.cpp @@ -1,3 +1,6 @@ +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project +// SPDX-License-Identifier: GPL-3.0-or-later + // SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project // SPDX-License-Identifier: GPL-2.0-or-later @@ -46,4 +49,10 @@ std::vector DecompressDataZSTD(std::span compressed) { return decompressed; } +std::size_t DecompressDataZSTD(std::span compressed, std::span output) { + const std::size_t result = + ZSTD_decompress(output.data(), output.size(), compressed.data(), compressed.size()); + return ZSTD_isError(result) ? 0 : result; +} + } // namespace Common::Compression diff --git a/src/common/zstd_compression.h b/src/common/zstd_compression.h index a5ab2d05bd..819d7d201a 100644 --- a/src/common/zstd_compression.h +++ b/src/common/zstd_compression.h @@ -1,3 +1,6 @@ +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project +// SPDX-License-Identifier: GPL-3.0-or-later + // SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project // SPDX-License-Identifier: GPL-2.0-or-later @@ -42,4 +45,15 @@ namespace Common::Compression { */ [[nodiscard]] std::vector DecompressDataZSTD(std::span compressed); +/** + * Decompresses a single Zstandard frame into a caller-provided buffer of known size. + * + * @param compressed the compressed source memory region. + * @param output the destination buffer. Its size is the maximum number of bytes to write. + * + * @return the number of bytes written, or 0 if the frame is invalid or doesn't fit. An empty + * frame also returns 0. + */ +[[nodiscard]] std::size_t DecompressDataZSTD(std::span compressed, std::span output); + } // namespace Common::Compression diff --git a/src/core/CMakeLists.txt b/src/core/CMakeLists.txt index dce6bfa787..1b578f67ba 100644 --- a/src/core/CMakeLists.txt +++ b/src/core/CMakeLists.txt @@ -33,6 +33,8 @@ add_library(core STATIC crypto/encryption_layer.h crypto/key_manager.cpp crypto/key_manager.h + crypto/ncz_decompressor.cpp + crypto/ncz_decompressor.h crypto/partition_data_manager.cpp crypto/partition_data_manager.h crypto/xts_encryption_layer.cpp @@ -1248,18 +1250,6 @@ if (HAS_NCE) target_link_libraries(core PRIVATE merry::oaknut) endif() -if (HAS_SCARLETNX) - target_sources(core PRIVATE - arm/scarletnx/arm_scarletnx_32.cpp - arm/scarletnx/arm_scarletnx_32.h - arm/scarletnx/arm_scarletnx_64.cpp - arm/scarletnx/arm_scarletnx_64.h - arm/scarletnx/scarletnx_api.h - arm/scarletnx/scarletnx_process.cpp - arm/scarletnx/scarletnx_process.h) - target_link_libraries(core PRIVATE ScarletNX::recompiler) -endif() - if (ARCHITECTURE_x86_64 OR ARCHITECTURE_arm64 OR ARCHITECTURE_riscv64 OR ARCHITECTURE_loongarch64) target_sources(core PRIVATE arm/dynarmic/arm_dynarmic.h diff --git a/src/core/arm/nce/arm_nce.cpp b/src/core/arm/nce/arm_nce.cpp index 63d5638fce..438dd4d981 100644 --- a/src/core/arm/nce/arm_nce.cpp +++ b/src/core/arm/nce/arm_nce.cpp @@ -369,8 +369,8 @@ void ArmNce::SetContext(const Kernel::Svc::ThreadContext& ctx) { } void ArmNce::SignalInterrupt(Kernel::KThread* thread) { - // Add break loop condition. - m_guest_ctx.esr_el1.fetch_or(static_cast(HaltReason::BreakLoop)); + constexpr u64 break_loop = static_cast(HaltReason::BreakLoop); + if ((m_guest_ctx.esr_el1.fetch_or(break_loop) & break_loop) != 0) return; auto* params = &thread->GetNativeExecutionParameters(); LockThreadParameters(params); diff --git a/src/core/arm/scarletnx/arm_scarletnx_32.cpp b/src/core/arm/scarletnx/arm_scarletnx_32.cpp deleted file mode 100644 index b4a01d7499..0000000000 --- a/src/core/arm/scarletnx/arm_scarletnx_32.cpp +++ /dev/null @@ -1,227 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#include -#include -#include - -#include "common/logging.h" -#include "core/arm/scarletnx/arm_scarletnx_32.h" -#include "core/core.h" -#include "core/core_timing.h" -#include "core/hle/kernel/k_process.h" - -namespace Core { - -namespace { - -// The exit SignalInterrupt sets, which no block sets itself. -constexpr u32 InterruptExit = 0x1000; -constexpr u32 UserMode = 0x10; -constexpr u32 ModeMask = 0x1F; - -// FPSCR splits into FPSR and FPCR as in ArmDynarmic32. -std::pair FpscrToFpsrFpcr(u32 fpscr) { - const u32 fpsr = (fpscr & 0xf8000000) | (fpscr & 0x80) | (fpscr & 0x1f); - const u32 fpcr = (fpscr & 0x7ff8000) | (fpscr & 0x1f00); - return {fpsr, fpcr}; -} - -u32 FpsrFpcrToFpscr(u64 fpsr, u64 fpcr) { - const auto [s, c] = FpscrToFpsrFpcr(u32(fpsr | fpcr)); - return s | c; -} - -} // namespace - -ArmScarletNx32::ArmScarletNx32(std::shared_ptr process, std::size_t core_index) - : ArmInterface{true}, m_process{std::move(process)}, m_core_index{core_index} { - m_rt.coproc = &Coprocessor; - m_cpu.parent = this; - m_cpu.cpu.mem = m_process->m_arena; - m_cpu.cpu.mem_size = 1ULL << 32; - m_cpu.cpu.rt = &m_rt; - - m_fault_scope.memory = &m_process->m_process.GetMemory(); - m_fault_scope.arena = m_process->m_arena; - m_fault_scope.arena_size = ScarletNxProcess::ArenaSize; -} - -ArmScarletNx32::~ArmScarletNx32() { - snx_recompiled32_free(m_module); -} - -int ArmScarletNx32::Svc(void* context, u32 imm, u64 number, const u64 arguments[6]) { - auto& self = *static_cast(context); - self.m_svc = imm; - self.m_in_svc = true; - return 1; -} - -void ArmScarletNx32::Coprocessor(SnxArmCPU* cpu, u32 word) { - auto& self = *reinterpret_cast(cpu)->parent; - const u32 coproc = (word >> 8) & 0xF; - const u32 crm = word & 0xF; - - if (coproc == 15 && (word & 0x0F100010) == 0x0E000010) { - // MCR p15 - const u32 opc1 = (word >> 21) & 7; - const u32 crn = (word >> 16) & 0xF; - const u32 opc2 = (word >> 5) & 7; - if (opc1 == 0 && crn == 7 && crm == 5 && opc2 == 4) { - // CP15_FLUSH_PREFETCH_BUFFER - return; - } - if (opc1 == 0 && crn == 7 && crm == 10 && (opc2 == 4 || opc2 == 5)) { - // CP15_DATA_SYNC_BARRIER, CP15_DATA_MEMORY_BARRIER - std::atomic_thread_fence(std::memory_order_seq_cst); - return; - } - } else if (coproc == 15 && (word & 0x0FF00000) == 0x0C500000) { - // MRRC p15 - const u32 opc1 = (word >> 4) & 0xF; - const u32 rt = (word >> 12) & 0xF; - const u32 rt2 = (word >> 16) & 0xF; - if (opc1 == 0 && crm == 14) { - // CNTPCT - const u64 ticks = self.m_process->m_system.CoreTiming().GetClockTicks(); - cpu->r[rt] = u32(ticks); - cpu->r[rt2] = u32(ticks >> 32); - return; - } - } - - // TPIDRURW and TPIDRURO, and undefined for anything else. - snx_arm_coproc_user(cpu, word); - ASSERT(cpu->exit == SNX_ARM_OK && "unhandled coproc"); -} - -HaltReason ArmScarletNx32::RunThread(Kernel::KThread* thread) { - if (m_interrupt.exchange(false)) { - return HaltReason::BreakLoop; - } - - // The block cache only covers the code present when it was made. - const u64 generation = m_process->CodeGeneration(); - if (!m_module || generation != m_code_generation) { - snx_recompiled32_free(m_module); - m_module = m_process->MakeModule32(); - m_code_generation = generation; - } - - LoadCpu(); - m_in_svc = false; - m_fault_scope.last_fault = ~u64{}; - SetScarletNxFaultScope(&m_fault_scope); - if (sigsetjmp(m_fault_scope.env, 1) != 0) { - SetScarletNxFaultScope(nullptr); - LOG_CRITICAL(Core_ARM, "invalid memory access at {:#x}", - m_fault_scope.fault_address); - SaveCpu(); - return HaltReason::PrefetchAbort; - } - const int exit = - m_module ? snx_recompiled32_run(m_module, &m_cpu.cpu, &Svc, this) : SNX_ARM_BAD_ENTRY; - SetScarletNxFaultScope(nullptr); - - SaveCpu(); - return Halt(exit); -} - -HaltReason ArmScarletNx32::Halt(int exit) { - if (m_in_svc) { - return HaltReason::SupervisorCall; - } - u32 const pc = m_cpu.cpu.r[15]; - u32 const syndrome = m_cpu.cpu.syndrome; - switch (u32(exit)) { - case InterruptExit: - m_interrupt.store(false); - return HaltReason::BreakLoop; - case SNX_ARM_BREAKPOINT: - // As Dynarmic does without a debugger: report it and go on. - LOG_CRITICAL(Core_ARM, "BKPT #{:#x} at {:#x}", syndrome, pc); - return HaltReason::BreakLoop; - default: - LOG_CRITICAL(Core_ARM, "A32 exit {} at {:#x} (syndrome {:#x})", exit, pc, syndrome); - return HaltReason::PrefetchAbort; - } -} - -HaltReason ArmScarletNx32::StepThread(Kernel::KThread* thread) { - // Never reached: the debugger selects Dynarmic. - LOG_ERROR(Core_ARM, "ScarletNX cannot single step"); - return HaltReason::BreakLoop; -} - -void ArmScarletNx32::LoadCpu() { - auto& cpu = m_cpu.cpu; - static_assert(sizeof(cpu.d) <= sizeof(m_ctx.v)); - for (std::size_t i = 0; i < 16; i++) - cpu.r[i] = u32(m_ctx.r[i]); - cpu.cpsr = (m_ctx.pstate & ~ModeMask) | UserMode; - std::memcpy(cpu.d, m_ctx.v.data(), sizeof(cpu.d)); - cpu.fpscr = FpsrFpcrToFpscr(m_ctx.fpsr, m_ctx.fpcr); - cpu.tpidrurw = u32(m_ctx.tpidr); - cpu.tpidruro = m_tpidrro; - cpu.exclusive_valid = 0; -} - -void ArmScarletNx32::SaveCpu() { - const auto& cpu = m_cpu.cpu; - for (std::size_t i = 0; i < 16; i++) - m_ctx.r[i] = cpu.r[i]; - m_ctx.fp = cpu.r[11]; - m_ctx.sp = cpu.r[13]; - m_ctx.lr = cpu.r[14]; - m_ctx.pc = cpu.r[15]; - m_ctx.pstate = cpu.cpsr; - std::memcpy(m_ctx.v.data(), cpu.d, sizeof(cpu.d)); - const auto [fpsr, fpcr] = FpscrToFpsrFpcr(cpu.fpscr); - m_ctx.fpsr = fpsr; - m_ctx.fpcr = fpcr; - m_ctx.tpidr = cpu.tpidrurw; -} - -void ArmScarletNx32::GetContext(Kernel::Svc::ThreadContext& ctx) const { - ctx = m_ctx; -} - -void ArmScarletNx32::SetContext(const Kernel::Svc::ThreadContext& ctx) { - m_ctx = ctx; -} - -void ArmScarletNx32::SetTpidrroEl0(u64 value) { - m_tpidrro = u32(value); -} - -void ArmScarletNx32::GetSvcArguments(std::span args) const { - for (std::size_t i = 0; i < args.size(); i++) - args[i] = u32(m_ctx.r[i]); -} - -void ArmScarletNx32::SetSvcArguments(std::span args) { - for (std::size_t i = 0; i < args.size(); i++) - m_ctx.r[i] = u32(args[i]); -} - -u32 ArmScarletNx32::GetSvcNumber() const { - return m_svc; -} - -void ArmScarletNx32::SignalInterrupt(Kernel::KThread* thread) { - // The run loop checks the exit between blocks. A block's own exit takes precedence. - m_interrupt.store(true); - u32 expected = SNX_ARM_OK; - __atomic_compare_exchange_n(&m_cpu.cpu.exit, &expected, InterruptExit, false, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED); -} - -void ArmScarletNx32::ClearInstructionCache() { - m_process->ClearInstructionCache(); -} - -void ArmScarletNx32::InvalidateCacheRange(u64 addr, std::size_t size) { - m_process->InvalidateCacheRange(addr, size); -} - -} // namespace Core diff --git a/src/core/arm/scarletnx/arm_scarletnx_32.h b/src/core/arm/scarletnx/arm_scarletnx_32.h deleted file mode 100644 index ff82bef6ae..0000000000 --- a/src/core/arm/scarletnx/arm_scarletnx_32.h +++ /dev/null @@ -1,81 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#pragma once - -#include -#include - -#include "core/arm/arm_interface.h" -#include "core/arm/scarletnx/scarletnx_api.h" -#include "core/arm/scarletnx/scarletnx_process.h" - -namespace Core { - -/// Runs A32 and Thumb code through ScarletNX's block compiler. Blocks return to the run loop, so -/// an SVC ends the run with r15 past it. The block cache is not thread-safe, so each core has its -/// own, rebuilt when the process' code changes. -class ArmScarletNx32 final : public ArmInterface { -public: - ArmScarletNx32(std::shared_ptr process, std::size_t core_index); - ~ArmScarletNx32() override; - - Architecture GetArchitecture() const override { - return Architecture::AArch32; - } - - HaltReason RunThread(Kernel::KThread* thread) override; - HaltReason StepThread(Kernel::KThread* thread) override; - - void GetContext(Kernel::Svc::ThreadContext& ctx) const override; - void SetContext(const Kernel::Svc::ThreadContext& ctx) override; - void SetTpidrroEl0(u64 value) override; - - void GetSvcArguments(std::span args) const override; - void SetSvcArguments(std::span args) override; - u32 GetSvcNumber() const override; - - void SignalInterrupt(Kernel::KThread* thread) override; - void ClearInstructionCache() override; - void InvalidateCacheRange(u64 addr, std::size_t size) override; - -protected: - const Kernel::DebugWatchpoint* HaltedWatchpoint() const override { - return nullptr; - } - - void RewindBreakpointInstruction() override {} - -private: - // The CPU first, so the coprocessor hook can find its interface. - struct Cpu32 { - SnxArmCPU cpu; - ArmScarletNx32* parent; - }; - - static int Svc(void* context, u32 imm, u64 number, const u64 arguments[6]); - static void Coprocessor(SnxArmCPU* cpu, u32 word); - HaltReason Halt(int exit); - void LoadCpu(); - void SaveCpu(); - - std::shared_ptr m_process; - std::size_t m_core_index{}; - - snx_recompiled32* m_module{}; - u64 m_code_generation{~u64{}}; - - // The context the kernel reads and writes; m_cpu holds it while guest code runs. - Kernel::Svc::ThreadContext m_ctx{}; - u32 m_tpidrro{}; - Cpu32 m_cpu{}; - SnxArmRt m_rt{}; - - bool m_in_svc{}; - u32 m_svc{}; - std::atomic m_interrupt{}; - - ScarletNxFaultScope m_fault_scope{}; -}; - -} // namespace Core diff --git a/src/core/arm/scarletnx/arm_scarletnx_64.cpp b/src/core/arm/scarletnx/arm_scarletnx_64.cpp deleted file mode 100644 index 56c48e2959..0000000000 --- a/src/core/arm/scarletnx/arm_scarletnx_64.cpp +++ /dev/null @@ -1,261 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#include - -#include - -#include "common/assert.h" -#include "common/literals.h" -#include "common/logging.h" -#include "core/arm/scarletnx/arm_scarletnx_64.h" -#include "core/hle/kernel/k_thread.h" -#include "core/hle/kernel/k_process.h" - -namespace Core { - -namespace { - -using namespace Common::Literals; -namespace ctx = boost::context::detail; - -// Reserved, not committed: guest calls nest on it, and recompiling a region runs on it too. -constexpr std::size_t StackSize = 32_MiB; -constexpr std::size_t GuardSize = 64_KiB; - -bool SameContext(const Kernel::Svc::ThreadContext a, const Kernel::Svc::ThreadContext b, bool in_svc) { - // An SVC returns its results in x0-x7. - for (std::size_t i = in_svc ? 8 : 0; i < a.r.size(); i++) - if (a.r[i] != b.r[i]) - return false; - return a.fp == b.fp && a.lr == b.lr && a.sp == b.sp && a.pc == b.pc - && (a.pstate >> 28) == (b.pstate >> 28) && a.v == b.v && a.fpcr == b.fpcr - && a.fpsr == b.fpsr && a.tpidr == b.tpidr; -} - -} // namespace - -const RtAPI* ArmScarletNx64::Runtime() { - // this incurs a cost each time we call Runtime() -- but we only ever call it once (ideally) - static const RtAPI rt = [] { - RtAPI api; - snx_runtime_defaults(&api); - api.svc = &Svc; - api.undef = &Undefined; - api.brk = &Breakpoint; - api.poll = &Poll; - api.miss = &Miss; - api.code_changed = &CodeChanged; - api.ticks = &ScarletNxTicks; - return api; - }(); - return &rt; -} - -ArmScarletNx64::ArmScarletNx64(std::shared_ptr process, std::size_t core_index) - : ArmInterface{true} - , m_process{std::move(process)}, m_core_index{core_index} -{ - // can't use virtualbuffer... - void* const stack = mmap(nullptr, StackSize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - ASSERT_MSG(stack != MAP_FAILED, "cannot allocate a guest stack"); - m_stack = static_cast(stack); - mprotect(m_stack, GuardSize, PROT_NONE); - - m_cpu.mem = m_process->m_arena; - m_cpu.rt = Runtime(); - m_cpu.thread = this; - - m_fault_scope.memory = &m_process->m_process.GetMemory(); - m_fault_scope.arena = m_process->m_arena; - m_fault_scope.arena_size = ScarletNxProcess::ArenaSize; -} - -ArmScarletNx64::~ArmScarletNx64() { - munmap(m_stack, StackSize); -} - -void ArmScarletNx64::Svc(CPU* cpu, u32 imm) { - auto& self = From(cpu); - self.m_svc = imm; - self.Yield(HaltReason::SupervisorCall, true, true); -} - -void ArmScarletNx64::Undefined(CPU* cpu, u64 pc, u32 word) { - // A reserved encoding rejected from inside an instruction, which then goes on. - if (cpu->pc != pc) { - LOG_ERROR(Core_ARM, "reserved encoding {:08X} at {:#x}", word, pc); - return; - } - LOG_CRITICAL(Core_ARM, "cannot execute {:08X} at {:#x}", word, pc); - From(cpu).Yield(HaltReason::PrefetchAbort, false, false); - UNREACHABLE(); -} - -void ArmScarletNx64::Breakpoint(CPU* cpu, u64 pc, u32 imm) { - // As Dynarmic does without a debugger: report it and go on. - LOG_CRITICAL(Core_ARM, "BRK #{:#x} at {:#x}", imm, pc); - cpu->pc = pc + 4; -} - -void ArmScarletNx64::Poll(CPU* cpu) { - auto& self = From(cpu); - __atomic_store_n(&cpu->poll, 0, __ATOMIC_RELAXED); - if (self.m_interrupt.exchange(false)) - self.Yield(HaltReason::BreakLoop, true, false); -} - -void ArmScarletNx64::Miss(CPU* cpu, u64 pc) { - auto& process = *From(cpu).m_process; - const auto function = process.Lookup(pc); - if (!function) { - cpu->pc = pc; - Undefined(cpu, pc, 0); - return; - } - // The first region was recompiled by this lookup: dispatch through its table from now on. - if (snx_recompiled* const module = cpu->disp ? nullptr : process.FindModule(pc)) - snx_recompiled_bind(module, cpu); - function(cpu, pc); -} - -void ArmScarletNx64::CodeChanged(CPU* cpu, u64 addr) { - constexpr u64 ICacheLineSize = 64; - From(cpu).m_process->InvalidateCacheRange(addr & ~(ICacheLineSize - 1), ICacheLineSize); -} - -void ArmScarletNx64::Entry(ctx::transfer_t transfer) { - auto* const self = static_cast(transfer.data); - self->m_host = transfer.fctx; - self->Execute(); -} - -void ArmScarletNx64::Execute() { - if (sigsetjmp(m_fault_scope.env, 1) != 0) { - LOG_CRITICAL(Core_ARM, "invalid memory access at {:#x}", m_fault_scope.fault_address); - Yield(HaltReason::PrefetchAbort, false, false); - UNREACHABLE(); - } - - // Every guest return to below the first frame comes back here. - while (true) - snx_lookup(&m_cpu, m_cpu.pc)(&m_cpu, m_cpu.pc); -} - -void ArmScarletNx64::Yield(HaltReason reason, bool resumable, bool in_svc) { - m_halt = reason; - m_resumable = resumable; - m_suspended_in_svc = in_svc; - m_host = ctx::jump_fcontext(m_host, nullptr).fctx; -} - -void ArmScarletNx64::ResetGuest() { - m_guest = ctx::make_fcontext(m_stack + StackSize, StackSize - GuardSize, &Entry); - if (snx_recompiled* const module = m_process->FindModule(m_cpu.pc)) - snx_recompiled_bind(module, &m_cpu); -} - -HaltReason ArmScarletNx64::RunThread(Kernel::KThread* thread) { - if (m_interrupt.exchange(false)) - return HaltReason::BreakLoop; - - bool const resume = m_resumable && m_suspended_thread_id == thread->GetThreadId() && SameContext(m_ctx, m_suspended_ctx, m_suspended_in_svc); - LoadCpu(); - if (!resume) - ResetGuest(); - - m_resumable = false; - m_fault_scope.last_fault = ~u64{}; - SetScarletNxFaultScope(&m_fault_scope); - m_guest = ctx::jump_fcontext(m_guest, this).fctx; - SetScarletNxFaultScope(nullptr); - - SaveCpu(); - if (m_resumable) { - m_suspended_thread_id = thread->GetThreadId(); - m_suspended_ctx = m_ctx; - } - return m_halt; -} - -HaltReason ArmScarletNx64::StepThread(Kernel::KThread* thread) { - // Never reached: the debugger selects Dynarmic. - LOG_ERROR(Core_ARM, "ScarletNX cannot single step"); - return HaltReason::BreakLoop; -} - -void ArmScarletNx64::LoadCpu() { - static_assert(sizeof(m_cpu.v) == sizeof(m_ctx.v)); - for (std::size_t i = 0; i < m_ctx.r.size(); i++) - m_cpu.x[i] = m_ctx.r[i]; - m_cpu.x[29] = m_ctx.fp; - m_cpu.x[30] = m_ctx.lr; - m_cpu.x[31] = m_ctx.sp; - m_cpu.pc = m_ctx.pc; - set_nzcv(&m_cpu, m_ctx.pstate >> 28); - std::memcpy(m_cpu.v, m_ctx.v.data(), sizeof(m_cpu.v)); - m_cpu.fpcr = m_ctx.fpcr; - m_cpu.fpsr = m_ctx.fpsr; - m_cpu.tpidr = m_ctx.tpidr; - m_cpu.tpidrro = m_tpidrro; - m_cpu.excl_on = 0; -} - -void ArmScarletNx64::SaveCpu() { - for (std::size_t i = 0; i < m_ctx.r.size(); i++) { - m_ctx.r[i] = m_cpu.x[i]; - } - m_ctx.fp = m_cpu.x[29]; - m_ctx.lr = m_cpu.x[30]; - m_ctx.sp = m_cpu.x[31]; - m_ctx.pc = m_cpu.pc; - m_ctx.pstate = get_nzcv(&m_cpu); - std::memcpy(m_ctx.v.data(), m_cpu.v, sizeof(m_cpu.v)); - m_ctx.fpcr = m_cpu.fpcr; - m_ctx.fpsr = m_cpu.fpsr; - m_ctx.tpidr = m_cpu.tpidr; -} - -void ArmScarletNx64::GetContext(Kernel::Svc::ThreadContext& ctx) const { - ctx = m_ctx; -} - -void ArmScarletNx64::SetContext(const Kernel::Svc::ThreadContext& ctx) { - m_ctx = ctx; -} - -void ArmScarletNx64::SetTpidrroEl0(u64 value) { - m_tpidrro = value; -} - -void ArmScarletNx64::GetSvcArguments(std::span args) const { - for (std::size_t i = 0; i < args.size(); i++) { - args[i] = m_ctx.r[i]; - } -} - -void ArmScarletNx64::SetSvcArguments(std::span args) { - for (std::size_t i = 0; i < args.size(); i++) { - m_ctx.r[i] = args[i]; - } -} - -u32 ArmScarletNx64::GetSvcNumber() const { - return m_svc; -} - -void ArmScarletNx64::SignalInterrupt(Kernel::KThread* thread) { - // Generated code reads poll on backward branches and calls Poll, which yields. - m_interrupt.store(true); - __atomic_store_n(&m_cpu.poll, 1, __ATOMIC_RELAXED); -} - -void ArmScarletNx64::ClearInstructionCache() { - m_process->ClearInstructionCache(); -} - -void ArmScarletNx64::InvalidateCacheRange(u64 addr, std::size_t size) { - m_process->InvalidateCacheRange(addr, size); -} - -} // namespace Core diff --git a/src/core/arm/scarletnx/arm_scarletnx_64.h b/src/core/arm/scarletnx/arm_scarletnx_64.h deleted file mode 100644 index 322d072b03..0000000000 --- a/src/core/arm/scarletnx/arm_scarletnx_64.h +++ /dev/null @@ -1,100 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#pragma once - -#include -#include - -#include - -#include "core/arm/arm_interface.h" -#include "core/arm/scarletnx/scarletnx_api.h" -#include "core/arm/scarletnx/scarletnx_process.h" - -namespace Core { - -/// Runs A64 code recompiled by ScarletNX. -/// -/// A recompiled guest function is a host function that returns when the guest function returns, -/// so guest calls nest on the host stack and an SVC cannot return from the generated code. Guest -/// code therefore runs on a coroutine of this core, which the SVC and interrupt hooks suspend. -/// When the kernel next runs the same thread with the same context, the coroutine resumes; -/// otherwise its host frames are dropped and execution restarts at the new pc through dispatch. -class ArmScarletNx64 final : public ArmInterface { -public: - ArmScarletNx64(std::shared_ptr process, std::size_t core_index); - ~ArmScarletNx64() override; - - Architecture GetArchitecture() const override { - return Architecture::AArch64; - } - - HaltReason RunThread(Kernel::KThread* thread) override; - HaltReason StepThread(Kernel::KThread* thread) override; - - void GetContext(Kernel::Svc::ThreadContext& ctx) const override; - void SetContext(const Kernel::Svc::ThreadContext& ctx) override; - void SetTpidrroEl0(u64 value) override; - - void GetSvcArguments(std::span args) const override; - void SetSvcArguments(std::span args) override; - u32 GetSvcNumber() const override; - - void SignalInterrupt(Kernel::KThread* thread) override; - void ClearInstructionCache() override; - void InvalidateCacheRange(u64 addr, std::size_t size) override; - -protected: - const Kernel::DebugWatchpoint* HaltedWatchpoint() const override { - return nullptr; - } - - void RewindBreakpointInstruction() override {} - -private: - static ArmScarletNx64& From(CPU* cpu) { - return *static_cast(cpu->thread); - } - static const RtAPI* Runtime(); - - // Runtime hooks called by the generated code. - static void Svc(CPU* cpu, u32 imm); - static void Undefined(CPU* cpu, u64 pc, u32 word); - static void Breakpoint(CPU* cpu, u64 pc, u32 imm); - static void Poll(CPU* cpu); - static void Miss(CPU* cpu, u64 pc); - static void CodeChanged(CPU* cpu, u64 addr); - - static void Entry(boost::context::detail::transfer_t transfer); - [[noreturn]] void Execute(); - void Yield(HaltReason reason, bool resumable, bool in_svc); - void ResetGuest(); - void LoadCpu(); - void SaveCpu(); - - std::shared_ptr m_process; - std::size_t m_core_index{}; - - // The context the kernel reads and writes; m_cpu holds it while guest code runs. - Kernel::Svc::ThreadContext m_ctx{}; - u64 m_tpidrro{}; - CPU m_cpu{}; - - HaltReason m_halt{}; - u32 m_svc{}; - std::atomic m_interrupt{}; - - // The coroutine, and what it was suspended in, when it can be resumed. - boost::context::detail::fcontext_t m_guest{}; - boost::context::detail::fcontext_t m_host{}; - u8* m_stack{}; - bool m_resumable{}; - bool m_suspended_in_svc{}; - u64 m_suspended_thread_id{}; - Kernel::Svc::ThreadContext m_suspended_ctx{}; - - ScarletNxFaultScope m_fault_scope{}; -}; - -} // namespace Core diff --git a/src/core/arm/scarletnx/scarletnx_api.h b/src/core/arm/scarletnx/scarletnx_api.h deleted file mode 100644 index 234e59f350..0000000000 --- a/src/core/arm/scarletnx/scarletnx_api.h +++ /dev/null @@ -1,10 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#pragma once - -// snx_recompiler.h includes cpu.h and arm.h before its own extern "C" block, so the whole -// header needs C linkage for the runtime library's functions to resolve. -extern "C" { -#include -} diff --git a/src/core/arm/scarletnx/scarletnx_process.cpp b/src/core/arm/scarletnx/scarletnx_process.cpp deleted file mode 100644 index cd7697ed2a..0000000000 --- a/src/core/arm/scarletnx/scarletnx_process.cpp +++ /dev/null @@ -1,300 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#include -#include - -#include - -#include "common/fs/path_util.h" -#include "common/logging.h" -#include "common/settings.h" -#include "core/arm/scarletnx/arm_scarletnx_32.h" -#include "core/arm/scarletnx/arm_scarletnx_64.h" -#include "core/arm/scarletnx/scarletnx_api.h" -#include "core/arm/scarletnx/scarletnx_process.h" -#include "core/core.h" -#include "core/core_timing.h" -#include "core/device_memory.h" -#include "core/hle/kernel/k_process.h" -#include "core/hle/kernel/kernel.h" -#include "core/memory.h" - -namespace Core { - -namespace { - -// ScarletNX refuses dispatch tables spanning more than 1 GiB of guest code. -constexpr u64 MaxModuleSpan = 1ULL << 30; -constexpr u64 A32Limit = 1ULL << 32; - -std::atomic g_core_timing{}; - -// Needs be TLS because SNX expects like, more than 4 threads (host == guest threads) -// and well -- quite impervious to "emulated" threads... maybe one day this can be removed -thread_local ScarletNxFaultScope* g_fault_scope{}; -static struct { - struct sigaction old_sigsegv; - struct sigaction old_sigbus; - std::once_flag installed; -} g_snx_data; - -bool Overlaps(const ScarletNxProcess::CodeRange& range, u64 addr, u64 size) { - return addr < range.addr + range.size && range.addr < addr + size; -} - -void ChainSignal(int sig, siginfo_t* info, void* raw_context) { - const struct sigaction& old = sig == SIGSEGV ? g_snx_data.old_sigsegv : g_snx_data.old_sigbus; - if (old.sa_flags & SA_SIGINFO) { - old.sa_sigaction(sig, info, raw_context); - } else if (old.sa_handler == SIG_DFL) { - // The faulting instruction runs again and takes the default action. - signal(sig, SIG_DFL); - } else if (old.sa_handler != SIG_IGN) { - old.sa_handler(sig); - } -} - -void FaultHandler(int sig, siginfo_t* info, void* raw_context) { - ScarletNxFaultScope* const scope = g_fault_scope; - u8* const addr = static_cast(info->si_addr); - if (!scope || addr < scope->arena || addr >= scope->arena + scope->arena_size) { - ChainSignal(sig, info, raw_context); - return; - } - - // Pages the rasterizer protects fault on access: invalidate them and retry. A second fault at - // the same addr means retrying will not help. - const u64 guest_address = u64(addr - scope->arena); - if (guest_address != scope->last_fault) { - scope->last_fault = guest_address; - if (scope->memory->InvalidateNCE(guest_address & ~Memory::YUZU_PAGEMASK, Memory::YUZU_PAGESIZE)) { - return; - } - } - scope->fault_address = guest_address; - siglongjmp(scope->env, 1); -} - -} // namespace - -void InstallScarletNxFaultHandler() { - std::call_once(g_snx_data.installed, [] { - struct sigaction action {}; - action.sa_flags = SA_SIGINFO; - action.sa_sigaction = &FaultHandler; - sigemptyset(&action.sa_mask); - sigaction(SIGSEGV, &action, &g_snx_data.old_sigsegv); - sigaction(SIGBUS, &action, &g_snx_data.old_sigbus); - }); -} - -void SetScarletNxFaultScope(ScarletNxFaultScope* scope) { - g_fault_scope = scope; -} - -u64 ScarletNxTicks() { - return g_core_timing.load(std::memory_order_relaxed)->GetClockTicks(); -} - -std::unique_ptr MakeScarletNxInterface(std::shared_ptr process, bool is_64bit, std::size_t core_index) { - if (is_64bit) - return std::make_unique(std::move(process), core_index); - return std::make_unique(std::move(process), core_index); -} - -ScarletNxProcess::ScarletNxProcess(System& system, Kernel::KProcess& process) - : m_system{system} - , m_process{process} - , m_arena{system.DeviceMemory().buffer.VirtualBasePointer()} -{ - InstallScarletNxFaultHandler(); - g_core_timing.store(&system.CoreTiming(), std::memory_order_relaxed); - - m_recompiler = snx_recompiler_new(SNX_RECOMPILER_FLAG_LLVM); - ASSERT(m_recompiler && "cannot create the recompiler"); - - // Linked modules are kept by a hash of their code, so later launches skip the recompile. - const auto cache = Common::FS::GetEdenPath(Common::FS::EdenPath::CacheDir) / "scarletnx"; - if (snx_recompiler_set_cache(m_recompiler, Common::FS::PathToUTF8String(cache).c_str()) != 0) - LOG_WARNING(Core_ARM, "no code cache: {}", snx_recompiler_error(m_recompiler)); -} - -ScarletNxProcess::~ScarletNxProcess() { - for (auto& mod : m_modules) - snx_recompiled_free(mod.mod); - snx_recompiler_free(m_recompiler); -} - -bool ScarletNxProcess::ShouldUse(System& system, const Kernel::KProcess& process) { - if (Settings::values.cpu_backend.GetValue() != Settings::CpuBackend::ScarletNx) - return false; - auto const f = [](auto& s) { - LOG_WARNING(Core_ARM, "snx: {}", s); - return false; - }; - if (!process.IsApplication()) - return f("no qlaunch yet"); - if (!system.Kernel().IsMulticore()) - return f("single core mode needs cycle counting"); - if (system.DebuggerEnabled()) - return f("the debugger needs stepping and watchpoints"); - /// ScarletNX expects flat memory, so it will only work with fastmem - if (!Settings::IsFastmemEnabled() || !system.DeviceMemory().buffer.VirtualBasePointer()) - return f("guest memory must be mapped by fastmem"); - if (snx_recompiler_abi() != SNX_ABI) - return f("the library was built with a different ABI"); - if (!snx_recompiler_has(SNX_RECOMPILER_FLAG_LLVM)) - return f("LLVM cannot target this host"); - return true; -} - -std::vector ScarletNxProcess::GetCodeRanges() const { - std::vector r; - const auto& page_table = m_process.GetPageTable().GetBasePageTable(); - u64 addr = GetInteger(page_table.GetAddressSpaceStart()); - while (true) { - Kernel::KMemoryInfo info; - Kernel::Svc::PageInfo page_info; - if (page_table.QueryInfo(&info, &page_info, addr).IsError()) - break; - if (True(info.GetPermission() & Kernel::KMemoryPermission::UserExecute)) { - if (!r.empty() && r.back().addr + r.back().size == info.GetAddress()) { - r.back().size += info.GetSize(); - } else { - r.push_back({info.GetAddress(), info.GetSize()}); - } - } - // The block past the addr space ends at 0. - if (info.GetEndAddress() <= addr) - break; - addr = info.GetEndAddress(); - } - return r; -} - -const ScarletNxProcess::Module* ScarletNxProcess::FindModuleLocked(u64 pc) const { - for (const auto& mod : m_modules) { - if (mod.retired || pc < mod.lo || pc >= mod.hi) - continue; - for (const auto& range : mod.ranges) - if (pc >= range.addr && pc < range.addr + range.size) - return &mod; - } - return nullptr; -} - -snx_recompiled* ScarletNxProcess::RecompileRegionLocked(u64 pc) { - const auto r = GetCodeRanges(); - const auto it = std::ranges::find_if(r, [pc](const CodeRange& range) { - return pc >= range.addr && pc < range.addr + range.size; - }); - if (it == r.end()) - return nullptr; - - const auto is_covered = [this](const CodeRange& range) { - return std::ranges::any_of(m_modules, [&](const Module& mod) { - return !mod.retired && std::ranges::any_of(mod.ranges, [&](const CodeRange& cr) { - return Overlaps(cr, range.addr, range.size); - }); - }); - }; - - // Recompile as many neighbouring regions as one dispatch table can cover. - auto first = size_t(it - r.begin()); - auto last = first; - const auto end_of = [&](size_t i) { return r[i].addr + r[i].size; }; - while (first > 0 && !is_covered(r[first - 1]) && end_of(last) - r[first - 1].addr <= MaxModuleSpan) - --first; - while (last + 1 < r.size() && !is_covered(r[last + 1]) - && end_of(last + 1) - r[first].addr <= MaxModuleSpan) - ++last; - - Module mod{}; - mod.ranges.assign(r.begin() + first, r.begin() + last + 1); - mod.lo = r[first].addr; - mod.hi = end_of(last); - - std::vector code; - u64 code_size = 0; - for (const auto& range : mod.ranges) { - code.push_back({range.addr, m_arena + range.addr, range.size}); - code_size += range.size; - } - - LOG_INFO(Core_ARM, "recompiling {} KiB of code at {:#x}-{:#x}", code_size / 1024, mod.lo, mod.hi); - mod.mod = snx_recompile_ranges(m_recompiler, code.data(), code.size(), &pc, 1); - if (mod.mod) { - LOG_INFO(Core_ARM, "recompiled {} functions", snx_recompiled_functions(mod.mod)); - } else { - // Kept, so the region is not recompiled again on every miss. - LOG_ERROR(Core_ARM, "recompilation failed: {}", snx_recompiler_error(m_recompiler)); - } - - m_modules.push_back(std::move(mod)); - return m_modules.back().mod; -} - -ScarletNxProcess::SnxGuestFn ScarletNxProcess::Lookup(u64 pc) { - snx_recompiled* mod; - { - std::scoped_lock lk{m_mutex}; - const Module* const known = FindModuleLocked(pc); - mod = known ? known->mod : RecompileRegionLocked(pc); - } - if (!mod) - return nullptr; - // Waits for the mod's JIT when discovery missed the code at pc. - return snx_recompiled_lookup(mod, pc); -} - -snx_recompiled* ScarletNxProcess::FindModule(u64 pc) { - std::scoped_lock lk{m_mutex}; - const Module* const mod = FindModuleLocked(pc); - return mod ? mod->mod : nullptr; -} - -snx_recompiled32* ScarletNxProcess::MakeModule32() { - std::scoped_lock lk{m_mutex}; - std::vector code; - for (const auto& range : GetCodeRanges()) { - if (range.addr >= A32Limit) { - continue; - } - code.push_back({range.addr, nullptr, std::min(range.size, A32Limit - range.addr)}); - } - if (code.empty()) - return nullptr; - snx_recompiled32* const mod = snx_recompile32_ranges(m_recompiler, code.data(), code.size()); - if (!mod) - LOG_ERROR(Core_ARM, "cannot prepare A32 code: {}", snx_recompiler_error(m_recompiler)); - return mod; -} - -void ScarletNxProcess::InvalidateCacheRange(u64 addr, u64 size) { - std::scoped_lock lk{m_mutex}; - for (auto& mod : m_modules) { - if (mod.retired || !std::ranges::any_of(mod.ranges, [&](const CodeRange& range) { - return Overlaps(range, addr, size); - })) - continue; - // Running code diverts out of the retired functions, and the region is recompiled from - // the new code when it is next reached. - if (mod.mod) - snx_recompiled_invalidate(mod.mod, addr, size, nullptr, nullptr); - mod.retired = true; - } - m_code_generation.fetch_add(1, std::memory_order_release); -} - -void ScarletNxProcess::ClearInstructionCache() { - std::scoped_lock lk{m_mutex}; - for (auto& mod : m_modules) { - if (!mod.retired && mod.mod) - snx_recompiled_invalidate(mod.mod, mod.lo, mod.hi - mod.lo, nullptr, nullptr); - mod.retired = true; - } - m_code_generation.fetch_add(1, std::memory_order_release); -} - -} // namespace Core diff --git a/src/core/arm/scarletnx/scarletnx_process.h b/src/core/arm/scarletnx/scarletnx_process.h deleted file mode 100644 index cb08a2ae2b..0000000000 --- a/src/core/arm/scarletnx/scarletnx_process.h +++ /dev/null @@ -1,104 +0,0 @@ -// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project -// SPDX-License-Identifier: GPL-3.0-or-later - -#pragma once - -#include -#include -#include -#include -#include - -#include "common/common_types.h" - -struct CPU; -struct snx_recompiler; -struct snx_recompiled; -struct snx_recompiled32; - -namespace Kernel { -class KProcess; -} - -namespace Core::Memory { -class Memory; -} - -namespace Core { - -class ArmInterface; -class System; - -class ScarletNxProcess { -public: - using SnxGuestFn = void (*)(CPU*, u64); - - // DeviceMemory reserves 39 bits of address space for the fastmem arena. - static constexpr u64 ArenaSize = 1ULL << 39; - - struct CodeRange { - u64 addr; - u64 size; - }; - - ScarletNxProcess(System& system, Kernel::KProcess& process); - ~ScarletNxProcess(); - - static bool ShouldUse(System& system, const Kernel::KProcess& process); - SnxGuestFn Lookup(u64 pc); - snx_recompiled* FindModule(u64 pc); - snx_recompiled32* MakeModule32(); - - void InvalidateCacheRange(u64 addr, u64 size); - void ClearInstructionCache(); - - /// @brief general gate for when we are about to invalidate a given block aka. trigger regen - u64 CodeGeneration() const { - return m_code_generation.load(std::memory_order_acquire); - } - - struct Module { - std::vector ranges; - u64 lo; - u64 hi; - snx_recompiled* mod; - bool retired; - }; - - std::vector GetCodeRanges() const; - const Module* FindModuleLocked(u64 pc) const; - snx_recompiled* RecompileRegionLocked(u64 pc); - - std::vector m_modules; - std::atomic m_code_generation{}; - - // TODO: remove parent ref? - System& m_system; - Kernel::KProcess& m_process; - u8* m_arena{}; - snx_recompiler* m_recompiler{}; - - // ScarletNX's recompiler is used by one thread at a time. - std::mutex m_mutex; -}; - -/// Recovers from host faults in guest memory on the thread running guest code: a fault on a -/// page the rasterizer protects is resolved and retried, any other jumps to env. -struct ScarletNxFaultScope { - sigjmp_buf env; - Memory::Memory* memory{}; - u8* arena{}; - u64 arena_size{}; - u64 last_fault{~0ull}; - u64 fault_address{}; -}; - -void InstallScarletNxFaultHandler(); -void SetScarletNxFaultScope(ScarletNxFaultScope* scope); - -/// CNTPCT_EL0 of the emulated system. -u64 ScarletNxTicks(); - -std::unique_ptr MakeScarletNxInterface(std::shared_ptr process, bool is_64bit, std::size_t core_index); - -} // namespace Core diff --git a/src/core/core.cpp b/src/core/core.cpp index 046ee53346..06543b0833 100644 --- a/src/core/core.cpp +++ b/src/core/core.cpp @@ -4,7 +4,6 @@ #include #include #include -#include #include #include "game_settings.h" @@ -249,30 +248,12 @@ struct System::Impl { } } - void NotifyNVDECChannelOpen(u64 process_id) { - std::scoped_lock lock{nvdec_active_mutex}; - ++nvdec_active_channels[process_id]; - } - - void NotifyNVDECChannelClose(u64 process_id) { - std::scoped_lock lock{nvdec_active_mutex}; - const auto it = nvdec_active_channels.find(process_id); - if (it == nvdec_active_channels.end()) { - return; - } - if (--it->second == 0) { - nvdec_active_channels.erase(it); - } + void SetNVDECActive(bool is_nvdec_active) { + nvdec_active = is_nvdec_active; } bool GetNVDECActive() { - std::scoped_lock lock{nvdec_active_mutex}; - return !nvdec_active_channels.empty(); - } - - bool IsNVDECActiveForProcess(u64 process_id) { - std::scoped_lock lock{nvdec_active_mutex}; - return nvdec_active_channels.contains(process_id); + return nvdec_active; } void InitializeDebugger(System& system, u16 port) { @@ -528,8 +509,6 @@ struct System::Impl { mutable std::mutex suspend_guard; std::mutex general_channel_mutex; - std::mutex nvdec_active_mutex; - std::unordered_map nvdec_active_channels; std::atomic_bool is_paused{}; std::atomic_bool is_shutting_down{}; std::atomic_bool is_powered_on{}; @@ -537,6 +516,7 @@ struct System::Impl { bool extended_memory_layout : 1 = false; bool exit_locked : 1 = false; bool exit_requested : 1 = false; + bool nvdec_active : 1 = false; void EnsureGeneralChannelInitialized(System& system) { if (!general_channel_event) { @@ -600,22 +580,14 @@ void System::UnstallApplication() { impl->UnstallApplication(); } -void System::NotifyNVDECChannelOpen(u64 process_id) { - impl->NotifyNVDECChannelOpen(process_id); -} - -void System::NotifyNVDECChannelClose(u64 process_id) { - impl->NotifyNVDECChannelClose(process_id); +void System::SetNVDECActive(bool is_nvdec_active) { + impl->SetNVDECActive(is_nvdec_active); } bool System::GetNVDECActive() { return impl->GetNVDECActive(); } -bool System::IsNVDECActiveForProcess(u64 process_id) { - return impl->IsNVDECActiveForProcess(process_id); -} - void System::InitializeDebugger() { impl->InitializeDebugger(*this, Settings::values.gdbstub_port.GetValue()); } diff --git a/src/core/core.h b/src/core/core.h index e26093f260..d88ed64151 100644 --- a/src/core/core.h +++ b/src/core/core.h @@ -191,10 +191,8 @@ public: std::unique_lock StallApplication(); void UnstallApplication(); - void NotifyNVDECChannelOpen(u64 process_id); - void NotifyNVDECChannelClose(u64 process_id); + void SetNVDECActive(bool is_nvdec_active); [[nodiscard]] bool GetNVDECActive(); - [[nodiscard]] bool IsNVDECActiveForProcess(u64 process_id); /** * Initialize the debugger. diff --git a/src/core/crypto/ncz_decompressor.cpp b/src/core/crypto/ncz_decompressor.cpp new file mode 100644 index 0000000000..5f56e55375 --- /dev/null +++ b/src/core/crypto/ncz_decompressor.cpp @@ -0,0 +1,430 @@ +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project +// SPDX-License-Identifier: GPL-3.0-or-later + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "common/common_types.h" +#include "common/logging.h" +#include "common/zstd_compression.h" +#include "core/crypto/aes_util.h" +#include "core/crypto/key_manager.h" +#include "core/crypto/ncz_decompressor.h" +#include "core/file_sys/vfs/vfs.h" + +namespace Core::Crypto { + +namespace { + +constexpr u64 HeaderSize = 0x4000; +constexpr std::size_t SectionWire = 0x40; +constexpr std::size_t BlockFixed = 0x18; +constexpr std::size_t Chunk = 0x100000; +// Upper bound on NCZ section entries (64 bytes each, so the table is at most 4 MiB). A base NCA +// has a handful of sections, but nsz stores an update's patch (BKTR) data one entry per +// subsection, which is thousands of them. +constexpr u64 MaxSectionCount = 0x10000; +constexpr std::array SectionsMagic{'N', 'C', 'Z', 'S', 'E', 'C', 'T', 'N'}; +constexpr std::array BlockMagic{'N', 'C', 'Z', 'B', 'L', 'O', 'C', 'K'}; + +struct Section { + s64 offset, size; + u64 crypto_type; + Key128 key, counter; + bool Ctr() const { return crypto_type == 3 || crypto_type == 4; } +}; + +struct BlockHeader { + u32 exponent; + u64 decompressed_size; + std::vector sizes; +}; + +u64 Le(const u8* p, std::size_t n) { + u64 v = 0; + for (std::size_t i = n; i-- > 0;) v = (v << 8) | p[i]; + return v; +} + +// Mirrors nsz AESCTR.seek(): upper 8 bytes = stored counter, lower 8 = absolute NCA offset / 16. +void SetIv(Key128& iv, const Key128& base, u64 off) { + iv = base; + u64 blk = off >> 4; + for (std::size_t i = 0; i < 8; ++i) { iv[15 - i] = static_cast(blk & 0xFF); blk >>= 8; } +} + +// Parse header + section table + optional block header. Returns payload_start. +std::optional, std::optional>> ParseHeader( + const FileSys::VirtualFile& f, u64& payload_start) { + const u64 fsize = f->GetSize(); + if (fsize < HeaderSize + 16) return std::nullopt; + + const auto head = f->ReadBytes(16, HeaderSize); + if (head.size() != 16 || !std::equal(head.begin(), head.begin() + 8, SectionsMagic.begin())) + return std::nullopt; + const u64 count = Le(head.data() + 8, 8); + if (count == 0 || count > MaxSectionCount || count * SectionWire > fsize - HeaderSize - 16) + return std::nullopt; + + const auto table = f->ReadBytes(count * SectionWire, HeaderSize + 16); + if (table.size() != count * SectionWire) return std::nullopt; + + std::vector
sections; + sections.reserve(count + 1); + for (u64 i = 0; i < count; ++i) { + const u8* e = table.data() + i * SectionWire; + Section s{}; + s.offset = static_cast(Le(e, 8)); + s.size = static_cast(Le(e + 8, 8)); + s.crypto_type = Le(e + 16, 8); + std::memcpy(s.key.data(), e + 32, 16); + std::memcpy(s.counter.data(), e + 48, 16); + sections.push_back(s); + } + + const u64 block_start = HeaderSize + 16 + count * SectionWire; + const auto fixed = f->ReadBytes(BlockFixed, block_start); + std::optional bh; + if (fixed.size() == BlockFixed && + std::equal(fixed.begin(), fixed.begin() + 8, BlockMagic.begin())) { + BlockHeader b{}; + b.exponent = fixed[11]; + const s32 n = static_cast(Le(fixed.data() + 12, 4)); + if (n <= 0) return std::nullopt; + b.decompressed_size = Le(fixed.data() + 16, 8); + const auto sizes = f->ReadBytes(static_cast(n) * 4, block_start + BlockFixed); + if (sizes.size() != static_cast(n) * 4) return std::nullopt; + b.sizes.resize(n); + for (s32 i = 0; i < n; ++i) b.sizes[i] = static_cast(Le(sizes.data() + i * 4, 4)); + payload_start = block_start + BlockFixed + static_cast(n) * 4; + bh = std::move(b); + } else { + payload_start = block_start; + } + + if (!sections.empty() && sections[0].offset > static_cast(HeaderSize)) { + Section fake{}; + fake.offset = static_cast(HeaderSize); + fake.size = sections[0].offset - static_cast(HeaderSize); + sections.insert(sections.begin(), fake); + } + return std::make_pair(std::move(sections), std::move(bh)); +} + +// Sequential reader over independently-compressed blocks, one block cached (like nsz_rs). +class BlockReader { +public: + BlockReader(FileSys::VirtualFile f, const BlockHeader& bh, u64 payload_start) + : source{std::move(f)}, block_size{1ULL << bh.exponent}, sizes{bh.sizes}, + decompressed_size{bh.decompressed_size} { + if (bh.exponent < 14 || bh.exponent > 32 || sizes.empty()) + throw std::runtime_error("bad NCZBLOCK header"); + u64 off = payload_start; + offsets.reserve(sizes.size()); + for (u32 s : sizes) { offsets.push_back(off); off += s; } + } + + // Fills `out` from `position`; returns bytes written. + std::size_t ReadAt(u64 position, std::span out) { + std::size_t done = 0; + while (done < out.size() && position < decompressed_size) { + const u64 id = position / block_size; + if (id >= sizes.size()) break; + if (cache_id != id && !Load(id)) break; + const u64 in = position - id * block_size; + const std::size_t n = static_cast( + std::min(out.size() - done, cache.size() - in)); + std::memcpy(out.data() + done, cache.data() + in, n); + done += n; + position += n; + } + return done; + } + +private: + bool Load(u64 id) { + u64 want = block_size; + if (id + 1 == sizes.size()) { + const u64 rem = decompressed_size % block_size; + if (rem > 0) want = rem; // avoid the 0-size bug from the original nsz + } + try { cache.resize(static_cast(want)); } + catch (const std::bad_alloc&) { return false; } + const u32 csize = sizes[id]; + if (csize < want) { + const auto comp = source->ReadBytes(csize, offsets[id]); + if (comp.size() != csize || + Common::Compression::DecompressDataZSTD(comp, cache) != want) return false; + } else if (source->Read(cache.data(), cache.size(), offsets[id]) != cache.size()) { + return false; + } + cache_id = id; + return true; + } + + FileSys::VirtualFile source; + u64 block_size; + std::vector offsets; + std::vector sizes; + u64 decompressed_size; + u64 cache_id = ~0ULL; + std::vector cache; +}; + +class BlockNczFile final : public FileSys::VfsFile { +public: + BlockNczFile(std::string name_, std::vector hdr_, std::vector
sections_, + std::unique_ptr body_) + : name{std::move(name_)}, hdr{std::move(hdr_)}, sections{std::move(sections_)}, + body{std::move(body_)} { + nca_size = HeaderSize; + for (const auto& s : sections) + nca_size = std::max(nca_size, static_cast(s.offset + s.size)); + ciphers.reserve(sections.size()); + for (const auto& s : sections) + ciphers.push_back(s.Ctr() ? std::make_unique>(s.key, Mode::CTR) + : nullptr); + } + + std::string GetName() const override { return name; } + std::size_t GetSize() const override { return static_cast(nca_size); } + bool Resize(std::size_t) override { return false; } + FileSys::VirtualDir GetContainingDirectory() const override { return nullptr; } + bool IsWritable() const override { return false; } + bool IsReadable() const override { return true; } + + std::size_t Read(u8* data, std::size_t len, std::size_t off) const override { + if (off >= nca_size) return 0; + len = static_cast(std::min(len, nca_size - off)); + const u64 end = off + len; + u64 pos = off; + if (pos < HeaderSize) { + const u64 n = std::min(end, HeaderSize) - pos; + std::memcpy(data, hdr.data() + pos, static_cast(n)); + pos += n; + } + std::scoped_lock lock{mutex}; + for (std::size_t i = 0; i < sections.size() && pos < end; ++i) { + const auto& s = sections[i]; + const u64 begin = std::max(static_cast(s.offset), HeaderSize); + const u64 stop = std::max(static_cast(s.offset + s.size), begin); + if (stop <= pos) continue; + if (pos < begin) { + const u64 g = std::min(end, begin); + std::memset(data + (pos - off), 0, static_cast(g - pos)); + pos = g; + if (pos >= end) break; + } + const u64 pe = std::min(end, stop); + const std::span piece(data + (pos - off), static_cast(pe - pos)); + if (body->ReadAt(pos - HeaderSize, piece) != piece.size()) + return static_cast(pos - off); + if (ciphers[i]) { + Key128 iv{}; + SetIv(iv, s.counter, pos); + ciphers[i]->SetIV(iv); + ciphers[i]->Transcode(piece.data(), piece.size(), piece.data(), Op::Encrypt); + } + pos = pe; + } + return len; + } + + std::size_t Write(const u8*, std::size_t, std::size_t) override { return 0; } + bool Rename(std::string_view) override { return false; } + +private: + const std::string name; + const std::vector hdr; + const std::vector
sections; + const std::unique_ptr body; + std::vector>> ciphers; + mutable std::mutex mutex; + u64 nca_size{}; +}; + +// Solid NCZ: one zstd stream over the whole body. Decoded incrementally into memory, up to the +// requested offset, so sequential install reads are O(n) instead of O(n^2). +class SolidNczFile final : public FileSys::VfsFile { +public: + SolidNczFile(std::string name_, FileSys::VirtualFile source_, + std::vector
sections_, u64 payload_start_) + : name{std::move(name_)}, source{std::move(source_)}, sections{std::move(sections_)}, + payload_start{payload_start_} { + const auto raw_header = source->ReadBytes(HeaderSize, 0); + data.assign(raw_header.begin(), raw_header.end()); + frontier = HeaderSize; + } + + ~SolidNczFile() override { + if (stream != nullptr) ZSTD_freeDStream(stream); + } + + std::string GetName() const override { return name; } + std::size_t GetSize() const override { + u64 size = HeaderSize; + for (const auto& s : sections) + size = std::max(size, static_cast(s.offset + s.size)); + return static_cast(size); + } + bool Resize(std::size_t) override { return false; } + FileSys::VirtualDir GetContainingDirectory() const override { return nullptr; } + bool IsWritable() const override { return false; } + bool IsReadable() const override { return true; } + + std::size_t Read(u8* out, std::size_t len, std::size_t off) const override { + std::scoped_lock lock{mutex}; + const u64 nca_size = GetSize(); + if (off >= nca_size) return 0; + len = static_cast(std::min(len, nca_size - off)); + const u64 needed = off + len; + if (needed > frontier && !DecodeTo(needed)) return 0; + std::memcpy(out, data.data() + off, len); + return len; + } + + std::size_t Write(const u8*, std::size_t, std::size_t) override { return 0; } + bool Rename(std::string_view) override { return false; } + +private: + bool DecodeTo(u64 needed) const { + if (stream == nullptr) { + stream = ZSTD_createDStream(); + if (stream == nullptr) return false; + ZSTD_DCtx_setParameter(stream, ZSTD_d_windowLogMax, 31); + read_pos = payload_start; + } + + const u64 file_size = source->GetSize(); + std::vector out_buf(Chunk); + + std::size_t index = 0; + for (; index < sections.size(); ++index) { + const auto& s = sections[index]; + const u64 begin = std::max(static_cast(s.offset), HeaderSize); + const u64 end = std::max(static_cast(s.offset) + static_cast(s.size), + begin); + if (end > frontier) break; + } + + for (; index < sections.size() && frontier < needed; ++index) { + const auto& s = sections[index]; + const u64 begin = std::max(static_cast(s.offset), HeaderSize); + const u64 end = std::max(static_cast(s.offset) + static_cast(s.size), + begin); + if (end <= frontier) continue; + u64 pos = std::max(begin, frontier); + + std::optional> cipher; + if (s.Ctr()) cipher.emplace(s.key, Mode::CTR); + + while (pos < end && frontier < needed) { + const u64 want = std::min(end - pos, out_buf.size()); + std::size_t produced = 0; + while (produced < want) { + if (in.pos >= in.size) { + const u64 remaining = file_size - read_pos; + const std::size_t chunk = static_cast( + std::min(Chunk, remaining)); + if (chunk == 0) { LOG_ERROR(Crypto, "Solid NCZ: truncated"); return false; } + const auto bytes = source->ReadBytes(chunk, read_pos); + if (bytes.empty()) { LOG_ERROR(Crypto, "Solid NCZ: truncated"); return false; } + read_pos += bytes.size(); + in_buf.assign(bytes.begin(), bytes.end()); + in.src = in_buf.data(); + in.size = in_buf.size(); + in.pos = 0; + } + ZSTD_outBuffer outb{out_buf.data() + produced, want - produced, 0}; + const std::size_t res = ZSTD_decompressStream(stream, &outb, &in); + if (ZSTD_isError(res)) { + LOG_ERROR(Crypto, "Solid NCZ: zstd error"); + return false; + } + produced += outb.pos; + } + if (cipher) { + Key128 iv{}; + SetIv(iv, s.counter, pos); + cipher->SetIV(iv); + cipher->Transcode(out_buf.data(), produced, out_buf.data(), Op::Encrypt); + } + data.insert(data.end(), out_buf.begin(), out_buf.begin() + produced); + pos += produced; + frontier = pos; + } + } + return frontier >= needed; + } + + const std::string name; + const FileSys::VirtualFile source; + const std::vector
sections; + const u64 payload_start; + mutable std::mutex mutex; + mutable std::vector data; + mutable ZSTD_DStream* stream = nullptr; + mutable u64 read_pos = 0; + mutable u64 frontier = 0; + mutable std::vector in_buf; + mutable ZSTD_inBuffer in{nullptr, 0, 0}; +}; + +// Shared opener: block path always; solid only when allow_solid is true. +FileSys::VirtualFile OpenNcz(const FileSys::VirtualFile& f, std::string_view name, + bool allow_solid) { + if (!f) return nullptr; + u64 ps = 0; + auto parsed = ParseHeader(f, ps); + if (!parsed) return nullptr; + auto& [sections, block] = *parsed; + + if (block) { + auto hdr = f->ReadBytes(HeaderSize, 0); + if (hdr.size() != HeaderSize) return nullptr; + std::unique_ptr body; + try { body = std::make_unique(f, *block, ps); } + catch (const std::exception& e) { + LOG_ERROR(Crypto, "Failed to open NCZBLOCK: {}", e.what()); + return nullptr; + } + return std::make_shared(std::string{name}, std::move(hdr), + std::move(sections), std::move(body)); + } + + if (allow_solid) { + return std::make_shared(std::string{name}, f, std::move(sections), ps); + } + + LOG_DEBUG(Crypto, "Solid NCZ {} skipped (only NCZBLOCK is supported for play)", name); + return nullptr; +} + +} // Anonymous namespace + +bool IsBlockCompressedNCZ(const FileSys::VirtualFile& f) { + if (!f) return false; + u64 ps = 0; + const auto p = ParseHeader(f, ps); + return p && p->second.has_value(); +} + +FileSys::VirtualFile DecompressNCZ(const FileSys::VirtualFile& f, std::string_view name) { + return OpenNcz(f, name, /*allow_solid=*/false); +} + +FileSys::VirtualFile DecompressNCZForInstall(const FileSys::VirtualFile& f, + std::string_view name) { + return OpenNcz(f, name, /*allow_solid=*/true); +} + +} // namespace Core::Crypto \ No newline at end of file diff --git a/src/core/crypto/ncz_decompressor.h b/src/core/crypto/ncz_decompressor.h new file mode 100644 index 0000000000..9557f8c517 --- /dev/null +++ b/src/core/crypto/ncz_decompressor.h @@ -0,0 +1,35 @@ +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project +// SPDX-License-Identifier: GPL-3.0-or-later + +#pragma once + +#include + +#include "core/file_sys/vfs/vfs_types.h" + +namespace Core::Crypto { + +/** + * Opens an NCZBLOCK-compressed NCA (nsz -B / .xcz block format) as a VirtualFile backed by + * on-demand block decoding. Nothing is written to disk; each read decodes only the blocks it + * covers and re-encrypts CTR sections. Solid NCZ is not supported. + * + * @param ncz_file The raw NCZ payload (e.g. ".ncz" entry from an NSZ/XCZ PFS0). + * @param output_name Name to give the returned VirtualFile (e.g. ".nca"). + * + * @return a VirtualFile for the NCA, or nullptr if the NCZ is malformed or not NCZBLOCK. + */ +[[nodiscard]] FileSys::VirtualFile DecompressNCZ(const FileSys::VirtualFile& ncz_file, + std::string_view output_name); + +/** + * Returns true if the given file is an NCZ payload in block-compressed (NCZBLOCK) form, + * i.e. one that DecompressNCZ() can open. Solid NCZ returns false. + */ +[[nodiscard]] bool IsBlockCompressedNCZ(const FileSys::VirtualFile& ncz_file); + +// Install to NAND: same as DecompressNCZ, but solid NCZ is decoded on demand too. +[[nodiscard]] FileSys::VirtualFile DecompressNCZForInstall( + const FileSys::VirtualFile& ncz_file, std::string_view output_name); + +} // namespace Core::Crypto \ No newline at end of file diff --git a/src/core/file_sys/content_archive.cpp b/src/core/file_sys/content_archive.cpp index b47b9437d0..1ba7bd3de3 100644 --- a/src/core/file_sys/content_archive.cpp +++ b/src/core/file_sys/content_archive.cpp @@ -45,6 +45,7 @@ NCA::NCA(VirtualFile file_, const NCA* base_nca, bool allow_missing_base) // Ensure we have the proper key area keys to continue. const u8 master_key_id = MasterKeyIdForKeyGeneration(reader->GetKeyGeneration()); if (!keys.HasKey(Core::Crypto::S128KeyType::KeyArea, master_key_id, reader->GetKeyIndex())) { + LOG_ERROR(Loader, "Cannot decrypt {}: keys for gen 0x{:02X}+ required.", file->GetName(), reader->GetKeyGeneration()); status = Loader::ResultStatus::ErrorMissingKeyAreaKey; return; } diff --git a/src/core/file_sys/registered_cache.cpp b/src/core/file_sys/registered_cache.cpp index 1f430434f7..26e73d260a 100644 --- a/src/core/file_sys/registered_cache.cpp +++ b/src/core/file_sys/registered_cache.cpp @@ -16,6 +16,7 @@ #include "common/random.h" #include "common/string_util.h" #include "core/crypto/key_manager.h" +#include "core/crypto/ncz_decompressor.h" #include "core/file_sys/card_image.h" #include "core/file_sys/common_funcs.h" #include "core/file_sys/content_archive.h" @@ -158,6 +159,25 @@ static std::shared_ptr OpenContainerAsNsp(const VirtualFile& file, Loader:: return nullptr; } +// Checks that every .ncz entry in the container is NCZBLOCK. Solid NCZ has no random access, +// and the whole point of this build is to reject it at scan time so such files never appear +// in the game list. Containers without any .ncz (plain NSP/XCI) pass through. +static bool ContainerIsFullyBlockCompressed(const std::shared_ptr& nsp) { + if (!nsp) { + return false; + } + for (const auto& entry : nsp->GetFiles()) { + const auto name = entry->GetName(); + if (name.size() < 4 || name.substr(name.size() - 4) != ".ncz") { + continue; + } + if (!Core::Crypto::IsBlockCompressedNCZ(entry)) { + return false; // solid NCZ -> reject the whole container + } + } + return true; // no .ncz at all, or all block-compressed +} + template bool ForEachContainerEntry(const std::shared_ptr& nsp, bool only_content, std::optional base_program_id, Callback&& on_entry) { @@ -811,7 +831,14 @@ std::vector RegisteredCache::ListEntriesFilter( } static std::shared_ptr GetNCAFromNSPForID(const NSP& nsp, const NcaID& id) { - auto file = nsp.GetFile(fmt::format("{}.nca", Common::HexToString(id, false))); + const auto id_string = Common::HexToString(id, false); + auto file = nsp.GetFile(fmt::format("{}.nca", id_string)); + if (file == nullptr) { + if (auto ncz_file = nsp.GetFile(fmt::format("{}.ncz", id_string))) { + file = Core::Crypto::DecompressNCZForInstall( + ncz_file, fmt::format("{}.nca", id_string)); + } + } if (file == nullptr) { return nullptr; } @@ -1273,6 +1300,12 @@ bool ManualContentProvider::AddEntriesFromContainer(VirtualFile file, bool only_ return false; } + // Reject solid NCZ (.nsz without -B): only block-compressed NCZ is supported. + if (!ContainerIsFullyBlockCompressed(nsp)) { + LOG_DEBUG(Service_FS, "Skipping solid-compressed container: {}", file->GetName()); + return false; + } + return ForEachContainerEntry( nsp, only_content, base_program_id, [this](TitleType title_type, ContentRecordType content_type, u64 title_id, @@ -1415,8 +1448,9 @@ void ExternalContentProvider::ScanDirectory(const VirtualDir& dir) { } const auto extension = Common::ToLower(filename.substr(dot_pos + 1)); - - if (extension == "nsp") { + // .xcz is an XCI container, but its updates only work through the NSP/PFS0 path + // (DecompressNCZ), not ProcessXCI. So route .xcz here on purpose. + if (extension == "nsp" || extension == "nsz" || extension == "xcz") { ProcessNSP(file); } else if (extension == "xci") { ProcessXCI(file); @@ -1434,6 +1468,12 @@ void ExternalContentProvider::ProcessNSP(const VirtualFile& file) { return; } + // Reject solid NCZ (.nsz without -B): Eden only supports block-compressed NCZ. + if (!ContainerIsFullyBlockCompressed(nsp)) { + LOG_DEBUG(Service_FS, "Skipping solid-compressed container: {}", file->GetName()); + return; + } + LOG_DEBUG(Service_FS, "Processing NSP file: {}", file->GetName()); AddExternalEntriesFromContainer(nsp, entries, versions, multi_version_entries); } diff --git a/src/core/file_sys/submission_package.cpp b/src/core/file_sys/submission_package.cpp index 0ea323492b..1a39d819f6 100644 --- a/src/core/file_sys/submission_package.cpp +++ b/src/core/file_sys/submission_package.cpp @@ -12,6 +12,7 @@ #include "common/hex_util.h" #include "common/logging.h" #include "core/crypto/key_manager.h" +#include "core/crypto/ncz_decompressor.h" #include "core/file_sys/common_funcs.h" #include "core/file_sys/content_archive.h" #include "core/file_sys/nca_metadata.h" @@ -246,6 +247,17 @@ void NSP::ReadNCAs(const std::vector& files) { const auto id_string = Common::HexToString(rec.nca_id, false); auto next_file = pfs->GetFile(fmt::format("{}.nca", id_string)); + if (next_file == nullptr) { + auto ncz_file = pfs->GetFile(fmt::format("{}.ncz", id_string)); + if (ncz_file != nullptr) { + next_file = Core::Crypto::DecompressNCZ(ncz_file, fmt::format("{}.nca", id_string)); + if (next_file == nullptr) { + LOG_DEBUG(Service_FS, "Skipping solid or unsupported NCZ content {}.ncz", id_string); + continue; + } + } + } + if (next_file == nullptr) { if (rec.type != ContentRecordType::DeltaFragment) { LOG_WARNING(Service_FS, diff --git a/src/core/hle/kernel/k_process.cpp b/src/core/hle/kernel/k_process.cpp index 942bc9f52e..27d1b1bfc4 100644 --- a/src/core/hle/kernel/k_process.cpp +++ b/src/core/hle/kernel/k_process.cpp @@ -23,9 +23,6 @@ #ifdef HAS_NCE #include "core/arm/nce/arm_nce.h" #endif -#ifdef HAS_SCARLETNX -#include "core/arm/scarletnx/scarletnx_process.h" -#endif namespace Kernel { @@ -1314,13 +1311,6 @@ void KProcess::InitializeInterfaces(KernelCore& kernel) { for (size_t i = 0; i < Core::Hardware::NUM_CPU_CORES; i++) m_arm_interfaces[i] = std::make_unique(kernel.System(), true, i); } else -#endif -#ifdef HAS_SCARLETNX - if (Core::ScarletNxProcess::ShouldUse(kernel.System(), *this)) { - auto scarletnx = std::make_shared(kernel.System(), *this); - for (size_t i = 0; i < Core::Hardware::NUM_CPU_CORES; i++) - m_arm_interfaces[i] = Core::MakeScarletNxInterface(scarletnx, this->Is64Bit(), i); - } else #endif if (this->Is64Bit()) { for (size_t i = 0; i < Core::Hardware::NUM_CPU_CORES; i++) { diff --git a/src/core/hle/service/filesystem/fsp/fs_i_storage.cpp b/src/core/hle/service/filesystem/fsp/fs_i_storage.cpp index 50b7d1e765..8541c17817 100644 --- a/src/core/hle/service/filesystem/fsp/fs_i_storage.cpp +++ b/src/core/hle/service/filesystem/fsp/fs_i_storage.cpp @@ -4,6 +4,10 @@ // SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project // SPDX-License-Identifier: GPL-2.0-or-later +#include +#include +#include + #include "core/file_sys/errors.h" #include "core/hle/service/cmif_serialization.h" #include "core/hle/service/filesystem/fsp/fs_i_storage.h" @@ -31,6 +35,14 @@ Result IStorage::Read( R_UNLESS(length >= 0, FileSys::ResultInvalidSize); R_UNLESS(offset >= 0, FileSys::ResultInvalidOffset); + const auto now = std::chrono::steady_clock::now(); + if (now < next_read_tick) { + std::this_thread::sleep_for(next_read_tick - now); + } + next_read_tick = std::chrono::steady_clock::now() + + std::chrono::nanoseconds{std::min(length, MaxChargedReadSize) * + 1'000'000'000 / ReadBytesPerSecond}; + // Read the data from the Storage backend backend->Read(out_bytes.data(), length, offset); diff --git a/src/core/hle/service/filesystem/fsp/fs_i_storage.h b/src/core/hle/service/filesystem/fsp/fs_i_storage.h index e676c52220..bac72d73c7 100644 --- a/src/core/hle/service/filesystem/fsp/fs_i_storage.h +++ b/src/core/hle/service/filesystem/fsp/fs_i_storage.h @@ -6,6 +6,8 @@ #pragma once +#include + #include "core/file_sys/vfs/vfs.h" #include "core/hle/service/cmif_types.h" #include "core/hle/service/filesystem/filesystem.h" @@ -20,6 +22,10 @@ public: private: FileSys::VirtualFile backend; + static constexpr s64 ReadBytesPerSecond = 40'000'000; + static constexpr s64 MaxChargedReadSize = 64 * 1024; + std::chrono::steady_clock::time_point next_read_tick{}; + Result Read( OutBuffer out_bytes, s64 offset, s64 length); diff --git a/src/core/hle/service/nvdrv/devices/nvhost_nvdec.cpp b/src/core/hle/service/nvdrv/devices/nvhost_nvdec.cpp index f47369adb8..7ac3dfaa46 100644 --- a/src/core/hle/service/nvdrv/devices/nvhost_nvdec.cpp +++ b/src/core/hle/service/nvdrv/devices/nvhost_nvdec.cpp @@ -8,7 +8,6 @@ #include "common/assert.h" #include "common/logging.h" #include "core/core.h" -#include "core/hle/kernel/k_process.h" #include "core/hle/service/nvdrv/core/container.h" #include "core/hle/service/nvdrv/devices/ioctl_serialization.h" #include "core/hle/service/nvdrv/devices/nvhost_nvdec.h" @@ -72,23 +71,17 @@ NvResult nvhost_nvdec::Ioctl3(DeviceFD fd, Ioctl command, std::span in void nvhost_nvdec::OnOpen(NvCore::SessionId session_id, DeviceFD fd) { LOG_INFO(Service_NVDRV, "NVDEC video stream started"); + system.SetNVDECActive(true); sessions[fd] = session_id; - if (const auto* session = core.GetSession(session_id); - session != nullptr && session->process != nullptr) { - system.NotifyNVDECChannelOpen(session->process->GetId()); - } host1x.StartDevice(fd, Tegra::Host1x::ChannelType::NvDec, channel_syncpoint); } void nvhost_nvdec::OnClose(DeviceFD fd) { LOG_INFO(Service_NVDRV, "NVDEC video stream ended"); host1x.StopDevice(fd, Tegra::Host1x::ChannelType::NvDec); + system.SetNVDECActive(false); auto it = sessions.find(fd); if (it != sessions.end()) { - if (const auto* session = core.GetSession(it->second); - session != nullptr && session->process != nullptr) { - system.NotifyNVDECChannelClose(session->process->GetId()); - } sessions.erase(it); } } diff --git a/src/core/hle/service/service.cpp b/src/core/hle/service/service.cpp index 225b92a783..b4fe2b3ed5 100644 --- a/src/core/hle/service/service.cpp +++ b/src/core/hle/service/service.cpp @@ -4,16 +4,12 @@ // SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project // SPDX-License-Identifier: GPL-2.0-or-later -#include #include -#include -#include #include "common/assert.h" #include "common/logging.h" #include "common/settings.h" #include "core/core.h" #include "core/hle/ipc.h" -#include "core/hle/kernel/k_process.h" #include "core/hle/kernel/kernel.h" #include "core/hle/service/ipc_helpers.h" #include "core/hle/service/service.h" @@ -37,7 +33,6 @@ ServiceFrameworkBase::ServiceFrameworkBase(Core::System& system_, const char* se : SessionRequestHandler(system_.Kernel(), service_name_) , system{system_} , service_name{service_name_} - , is_i_storage{std::string_view{service_name_} == "IStorage"} , handler_invoker{handler_invoker_} , max_sessions{max_sessions_} {} @@ -82,22 +77,13 @@ void ServiceFrameworkBase::ReportUnimplementedFunction(HLERequestContext& ctx, } void ServiceFrameworkBase::InvokeRequest(HLERequestContext& ctx) { - const auto command = ctx.GetCommand(); - auto it = handlers.find(command); - const bool is_cmd_read = command == 0; + auto it = handlers.find(ctx.GetCommand()); FunctionInfoBase const* info = it == handlers.end() ? nullptr : &it->second; if (info == nullptr || info->handler_callback == nullptr) return ReportUnimplementedFunction(ctx, info); LOG_TRACE(Service, "{}", MakeFunctionString(info->name, GetServiceName(), ctx.CommandBuffer())); handler_invoker(this, info->handler_callback, ctx); - - if (is_i_storage && is_cmd_read) { - const auto* const process = ctx.GetThread().GetOwnerProcess(); - if (process != nullptr && system.IsNVDECActiveForProcess(process->GetId())) { - std::this_thread::sleep_for(std::chrono::microseconds{600}); - } - } } void ServiceFrameworkBase::InvokeRequestTipc(HLERequestContext& ctx) { diff --git a/src/core/hle/service/service.h b/src/core/hle/service/service.h index ea98897e79..68b0d636d8 100644 --- a/src/core/hle/service/service.h +++ b/src/core/hle/service/service.h @@ -107,8 +107,6 @@ protected: Core::System& system; /// Identifier string used to connect to the service. const char* service_name; - /// Whether this is the IStorage service. - const bool is_i_storage; /// Function used to safely up-cast pointers to the derived class before invoking a handler. InvokerFn* handler_invoker; /// Maximum number of concurrent sessions that this service can handle. diff --git a/src/core/loader/loader.cpp b/src/core/loader/loader.cpp index 90e23ccc11..92e2e2f602 100644 --- a/src/core/loader/loader.cpp +++ b/src/core/loader/loader.cpp @@ -82,23 +82,27 @@ std::shared_ptr OpenContainerAsNsp(FileSys::VirtualFile file, File return nullptr; } - if (type == FileType::NSP) { + if (type == FileType::NSP || type == FileType::NSZ) { auto nsp = std::make_shared(file, program_id, program_index); return nsp->GetStatus() == ResultStatus::Success ? nsp : nullptr; } - if (type == FileType::XCI) { + if (type == FileType::XCI || type == FileType::XCZ) { FileSys::XCI xci{file, program_id, program_index}; - if (xci.GetStatus() != ResultStatus::Success) { - return nullptr; + if (xci.GetStatus() == ResultStatus::Success) { + auto secure_nsp = xci.GetSecurePartitionNSP(); + if (secure_nsp != nullptr && secure_nsp->GetStatus() == ResultStatus::Success) { + return secure_nsp; + } } - auto secure_nsp = xci.GetSecurePartitionNSP(); - if (secure_nsp == nullptr || secure_nsp->GetStatus() != ResultStatus::Success) { - return nullptr; + // PFS0-shaped .xcz (NSZ-style container) does not parse as XCI; fall back to NSP. + if (type == FileType::XCZ) { + auto nsp = std::make_shared(file, program_id, program_index); + return nsp->GetStatus() == ResultStatus::Success ? nsp : nullptr; } - return secure_nsp; + return nullptr; } return nullptr; @@ -144,7 +148,8 @@ FileType IdentifyFile(FileSys::VirtualFile file) { } bool IsContainerType(FileType type) { - return type == FileType::NSP || type == FileType::XCI; + return type == FileType::NSP || type == FileType::NSZ || type == FileType::XCI || + type == FileType::XCZ; } bool IsBootableGameContainer(FileSys::VirtualFile file, FileType type, u64 program_id, @@ -180,8 +185,12 @@ FileType GuessFromFilename(const std::string& name) { return FileType::NCA; else if (extension == "xci") return FileType::XCI; + else if (extension == "xcz") + return FileType::XCZ; else if (extension == "nsp") return FileType::NSP; + else if (extension == "nsz") + return FileType::NSZ; else if (extension == "kip") return FileType::KIP; return FileType::Unknown; @@ -197,10 +206,14 @@ std::string GetFileTypeString(FileType type) { return "NCA"; case FileType::XCI: return "XCI"; + case FileType::XCZ: + return "XCZ"; case FileType::NAX: return "NAX"; case FileType::NSP: return "NSP"; + case FileType::NSZ: + return "NSZ"; case FileType::KIP: return "KIP"; case FileType::DeconstructedRomDirectory: @@ -326,12 +339,29 @@ static std::unique_ptr GetFileLoader(Core::System& system, FileSys::V system.GetContentProvider(), program_id, program_index); + // NCZ-compressed XCI container. XCI-shaped .xcz goes to the XCI loader, while + // PFS0-shaped (NSZ-style) .xcz containers fall back to the NSP loader. + case FileType::XCZ: { + FileSys::XCI probe{file, program_id, program_index}; + if (probe.GetStatus() == ResultStatus::Success && + probe.GetSecurePartitionNSP() != nullptr) { + return std::make_unique(std::move(file), + system.GetFileSystemController(), + system.GetContentProvider(), program_id, + program_index); + } + return std::make_unique(std::move(file), system.GetFileSystemController(), + system.GetContentProvider(), program_id, + program_index); + } + // NX NAX (NintendoAesXts) file format. case FileType::NAX: return std::make_unique(std::move(file)); // NX NSP (Nintendo Submission Package) file format case FileType::NSP: + case FileType::NSZ: return std::make_unique(std::move(file), system.GetFileSystemController(), system.GetContentProvider(), program_id, program_index); diff --git a/src/core/loader/loader.h b/src/core/loader/loader.h index 95ce638da0..d3c0230cb0 100644 --- a/src/core/loader/loader.h +++ b/src/core/loader/loader.h @@ -42,7 +42,9 @@ enum class FileType { NRO, NCA, NSP, + NSZ, XCI, + XCZ, NAX, KIP, DeconstructedRomDirectory, diff --git a/src/core/loader/nsp.cpp b/src/core/loader/nsp.cpp index cfc908b4ef..8b2c79394d 100644 --- a/src/core/loader/nsp.cpp +++ b/src/core/loader/nsp.cpp @@ -56,6 +56,17 @@ AppLoader_NSP::AppLoader_NSP(FileSys::VirtualFile file_, AppLoader_NSP::~AppLoader_NSP() = default; FileType AppLoader_NSP::IdentifyType(const FileSys::VirtualFile& nsp_file) { + // .nsz is NSP-compatible; return NSZ so the game list shows the right type. + if (nsp_file->GetName().ends_with(".nsz")) { + return FileType::NSZ; + } + + // PFS0-shaped .xcz containers are handled by this loader too; report XCZ so the game + // list does not show them as NSP. XCI-shaped .xcz is identified by AppLoader_XCI. + if (nsp_file->GetName().ends_with(".xcz")) { + return FileType::XCZ; + } + const FileSys::NSP nsp(nsp_file); if (nsp.GetStatus() != ResultStatus::Success) { diff --git a/src/frontend_common/content_manager.h b/src/frontend_common/content_manager.h index 4f3112e3fb..6d79098efa 100644 --- a/src/frontend_common/content_manager.h +++ b/src/frontend_common/content_manager.h @@ -1,4 +1,4 @@ -// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project // SPDX-License-Identifier: GPL-3.0-or-later // SPDX-FileCopyrightText: 2024 yuzu Emulator Project @@ -163,7 +163,9 @@ inline InstallResult InstallNSP(Core::System& system, FileSys::VfsFilesystem& vf std::shared_ptr nsp; FileSys::VirtualFile file = vfs.OpenFile(filename, FileSys::OpenMode::Read); - if (boost::to_lower_copy(file->GetName()).ends_with("nsp")) { + const auto lower_name = boost::to_lower_copy(file->GetName()); + + if (lower_name.ends_with("nsp") || lower_name.ends_with("nsz")) { nsp = std::make_shared(file); if (nsp->IsExtractedType()) { return InstallResult::Failure; diff --git a/src/qt_common/config/shared_translation.cpp b/src/qt_common/config/shared_translation.cpp index e43fe8037a..bb92d896f8 100644 --- a/src/qt_common/config/shared_translation.cpp +++ b/src/qt_common/config/shared_translation.cpp @@ -194,10 +194,6 @@ std::unique_ptr InitializeTranslations(QObject* parent) { INSERT(Settings, skip_cpu_inner_invalidation, tr("Skip CPU Inner Invalidation"), tr("Skips certain cache invalidations during memory updates, reducing CPU usage and " "improving latency. This may cause soft-crashes.")); - INSERT(Settings, nce_invalidation_gpu_readback, - tr("GPU Readback Before NCE Invalidation"), - tr("Flushes GPU buffer data back to guest memory before NCE invalidates the whole " - "guest page, preserving it while the CPU store proceeds.")); INSERT(Settings, vsync_mode, tr("VSync Mode:"), tr("FIFO (VSync) does not drop frames or exhibit tearing but is limited by the screen " "refresh rate.\nFIFO Relaxed allows tearing as it recovers from a slow down.\n" @@ -449,12 +445,7 @@ std::unique_ptr ComboboxEnumeration(QObject* parent) { translations->insert({Settings::EnumMetadata::Index(), { PAIR(CpuBackend, Dynarmic, tr("Dynarmic")), -#ifdef HAS_NCE PAIR(CpuBackend, Nce, tr("NCE")), -#endif -#ifdef HAS_SCARLETNX - PAIR(CpuBackend, ScarletNx, tr("ScarletNX")), -#endif }}); translations->insert({Settings::EnumMetadata::Index(), { diff --git a/src/qt_common/game_list/worker.cpp b/src/qt_common/game_list/worker.cpp index 586a26e8b9..6b6f93447d 100644 --- a/src/qt_common/game_list/worker.cpp +++ b/src/qt_common/game_list/worker.cpp @@ -360,7 +360,8 @@ void GameListWorker::ScanFileSystem(ScanTarget target, const std::string& dir_pa } if (target == ScanTarget::PopulateGameList && - (file_type == Loader::FileType::XCI || file_type == Loader::FileType::NSP) && + (file_type == Loader::FileType::XCI || file_type == Loader::FileType::XCZ || + file_type == Loader::FileType::NSP || file_type == Loader::FileType::NSZ) && !Loader::IsBootableGameContainer(file, file_type)) { return true; } @@ -375,7 +376,9 @@ void GameListWorker::ScanFileSystem(ScanTarget target, const std::string& dir_pa program_id, file); } else if (Settings::values.ext_content_from_game_dirs.GetValue() && (file_type == Loader::FileType::XCI || - file_type == Loader::FileType::NSP)) { + file_type == Loader::FileType::XCZ || + file_type == Loader::FileType::NSP || + file_type == Loader::FileType::NSZ)) { void(provider->AddEntriesFromContainer(file)); } } else { @@ -402,7 +405,8 @@ void GameListWorker::ScanFileSystem(ScanTarget target, const std::string& dir_pa }; if (res2 == Loader::ResultStatus::Success && program_ids.size() > 1 && - (file_type == Loader::FileType::XCI || file_type == Loader::FileType::NSP)) { + (file_type == Loader::FileType::XCI || file_type == Loader::FileType::XCZ || + file_type == Loader::FileType::NSP || file_type == Loader::FileType::NSZ)) { for (const auto id : program_ids) { // dravee suggested this, only viable way to // not show sub-games in qlaunch for now. diff --git a/src/qt_common/qt_common.cpp b/src/qt_common/qt_common.cpp index b9b8caf6e8..794aeaabb1 100644 --- a/src/qt_common/qt_common.cpp +++ b/src/qt_common/qt_common.cpp @@ -57,7 +57,8 @@ std::unique_ptr emu_thread = nullptr; const QStringList supported_file_extensions = {QStringLiteral("nro"), QStringLiteral("nso"), QStringLiteral("nca"), QStringLiteral("xci"), - QStringLiteral("nsp"), QStringLiteral("kip")}; + QStringLiteral("xcz"), QStringLiteral("nsp"), + QStringLiteral("nsz"), QStringLiteral("kip")}; Core::Frontend::WindowSystemType GetWindowSystemType() { // Determine WSI type based on Qt platform. diff --git a/src/shader_recompiler/frontend/ir/breadth_first_search.h b/src/shader_recompiler/frontend/ir/breadth_first_search.h index ec55425549..afe153f180 100644 --- a/src/shader_recompiler/frontend/ir/breadth_first_search.h +++ b/src/shader_recompiler/frontend/ir/breadth_first_search.h @@ -1,8 +1,12 @@ +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project +// SPDX-License-Identifier: GPL-3.0-or-later + // SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project // SPDX-License-Identifier: GPL-2.0-or-later #pragma once +#include #include #include #include @@ -13,9 +17,10 @@ namespace Shader::IR { -template -auto BreadthFirstSearch(const Value& value, Pred&& pred) +template +auto BreadthFirstSearch(Value value, Pred&& pred, Resolve&& resolve = {}) -> std::invoke_result_t { + value = resolve(value); if (value.IsImmediate()) { // Nothing to do with immediates return std::nullopt; @@ -36,7 +41,7 @@ auto BreadthFirstSearch(const Value& value, Pred&& pred) } // Visit the right most arguments first for (size_t arg = inst->NumArgs(); arg--;) { - const Value arg_value{inst->Arg(arg)}; + const Value arg_value{resolve(inst->Arg(arg))}; if (arg_value.IsImmediate()) { continue; } diff --git a/src/shader_recompiler/ir_opt/texture_pass.cpp b/src/shader_recompiler/ir_opt/texture_pass.cpp index fe34a67555..fae69ab91b 100644 --- a/src/shader_recompiler/ir_opt/texture_pass.cpp +++ b/src/shader_recompiler/ir_opt/texture_pass.cpp @@ -31,6 +31,60 @@ struct TextureInst { using TextureInstVector = boost::container::small_vector; +const IR::Block* ConditionalHeader(const IR::Block* body) { + const auto predecessors{body->ImmPredecessors()}; + const auto successors{body->ImmSuccessors()}; + if (predecessors.size() != 1 || successors.size() != 1) return nullptr; + const IR::Block* const header{predecessors[0]}; + const auto targets{header->ImmSuccessors()}; + if (targets.size() != 2 || targets[0] != body || targets[1] != successors[0] || + targets[1]->ImmPredecessors().size() != 2 || header->empty() || + header->back().GetOpcode() != IR::Opcode::ConditionRef) { + return nullptr; + } + return header; +} + +std::pair Condition(IR::Value value) { + bool negated{}; + while (const IR::Inst* inst = value.TryInstRecursive()) { + if (inst->GetOpcode() == IR::Opcode::LogicalNot) negated = !negated; + else if (inst->GetOpcode() != IR::Opcode::ConditionRef) break; + value = inst->Arg(0); + } + return {value.Resolve(), negated}; +} + +IR::Value ResolveConditionalPhi(IR::Value value, const IR::Block* block) { + if (!value.Resolve().IsPhi() || !(block = ConditionalHeader(block))) return value; + const auto [condition, negated]{Condition(block->back().Arg(0))}; + boost::container::small_vector visited; + while (value.Resolve().IsPhi() && std::ranges::find(visited, block) == visited.end()) { + visited.push_back(block); + const IR::Inst* const phi{value.InstRecursive()}; + const auto predecessors{block->ImmPredecessors()}; + if (predecessors.size() == 1) { + block = predecessors[0]; + continue; + } + if (predecessors.size() != 2 || phi->NumArgs() != 2) break; + const IR::Block* body{predecessors[0]}; + const IR::Block* header{ConditionalHeader(body)}; + if (header != predecessors[1]) { + body = predecessors[1]; + header = ConditionalHeader(body); + if (header != predecessors[0]) break; + } + const auto [other_condition, other_negated]{Condition(header->back().Arg(0))}; + if (condition == other_condition && (phi->PhiBlock(0) == header || phi->PhiBlock(0) == body)) { + const IR::Block* const incoming{negated == other_negated ? body : header}; + value = phi->Arg(phi->PhiBlock(0) == incoming ? 0 : 1); + } + block = header; + } + return value; +} + constexpr u32 DESCRIPTOR_SIZE = 8; constexpr u32 DESCRIPTOR_SIZE_SHIFT = u32(std::countr_zero(DESCRIPTOR_SIZE)); constexpr u32 DESCRIPTOR_MAX_COUNT = 1024; @@ -299,23 +353,23 @@ static inline bool IsTexturePixelFormatIntegerCached(Environment& env, } -std::optional Track(const IR::Value& value, Environment& env, const HostTranslateInfo& host_info); -static inline std::optional TrackCached(const IR::Value& v, Environment& env, const HostTranslateInfo& host_info) { - if (const IR::Inst* key = v.InstRecursive()) { +std::optional Track(const IR::Value& value, Environment& env, const HostTranslateInfo& host_info, const IR::Block* block); +static inline std::optional TrackCached(const IR::Value& v, Environment& env, const HostTranslateInfo& host_info, const IR::Block* block) { + if (const IR::Inst* key = v.TryInstRecursive()) { if (auto it = env.track_cache.find(key); it != env.track_cache.end()) return it->second; - auto found = Track(v, env, host_info); + auto found = Track(v, env, host_info, block); if (found) env.track_cache.emplace(key, *found); return found; } - return Track(v, env, host_info); + return Track(v, env, host_info, block); } -std::optional TryGetConstBuffer(const IR::Inst* inst, Environment& env, const HostTranslateInfo& host_info); +std::optional TryGetConstBuffer(const IR::Inst* inst, Environment& env, const HostTranslateInfo& host_info, const IR::Block* block); -std::optional Track(const IR::Value& value, Environment& env, const HostTranslateInfo& host_info) { - return IR::BreadthFirstSearch(value, [&env, &host_info](const IR::Inst* inst) { - return TryGetConstBuffer(inst, env, host_info); - }); +std::optional Track(const IR::Value& value, Environment& env, const HostTranslateInfo& host_info, const IR::Block* block) { + return IR::BreadthFirstSearch(value, [&env, &host_info, block](const IR::Inst* inst) { + return TryGetConstBuffer(inst, env, host_info, block); + }, [&](IR::Value operand) { return ResolveConditionalPhi(operand, block); }); } std::optional TryGetConstant(IR::Value& value, Environment& env) { @@ -339,13 +393,13 @@ std::optional TryGetConstant(IR::Value& value, Environment& env) { return ReadCbufCached(env, index_number, offset_number); } -std::optional TryGetConstBuffer(const IR::Inst* inst, Environment& env, const HostTranslateInfo& host_info) { +std::optional TryGetConstBuffer(const IR::Inst* inst, Environment& env, const HostTranslateInfo& host_info, const IR::Block* block) { switch (inst->GetOpcode()) { default: return std::nullopt; case IR::Opcode::BitwiseOr32: { - std::optional lhs{TrackCached(inst->Arg(0), env, host_info)}; - std::optional rhs{TrackCached(inst->Arg(1), env, host_info)}; + std::optional lhs{TrackCached(inst->Arg(0), env, host_info, block)}; + std::optional rhs{TrackCached(inst->Arg(1), env, host_info, block)}; if (!lhs || !rhs) { return std::nullopt; } @@ -375,7 +429,7 @@ std::optional TryGetConstBuffer(const IR::Inst* inst, Environme if (!shift.IsImmediate()) { return std::nullopt; } - std::optional lhs{TrackCached(inst->Arg(0), env, host_info)}; + std::optional lhs{TrackCached(inst->Arg(0), env, host_info, block)}; if (lhs) { lhs->shift_left = shift.U32(); } @@ -403,7 +457,7 @@ std::optional TryGetConstBuffer(const IR::Inst* inst, Environme return std::nullopt; } while (false); } - std::optional lhs{TrackCached(op1, env, host_info)}; + std::optional lhs{TrackCached(op1, env, host_info, block)}; if (lhs) { lhs->shift_left = static_cast(std::countr_zero(op2.U32())); } @@ -469,7 +523,7 @@ std::optional TryGetConstBuffer(const IR::Inst* inst, Environme TextureInst MakeInst(Environment& env, IR::Block* block, IR::Inst& inst, const HostTranslateInfo& host_info) { ConstBufferAddr addr; if (IsBindless(inst)) { - const std::optional track_addr{TrackCached(inst.Arg(0), env, host_info)}; + const std::optional track_addr{TrackCached(inst.Arg(0), env, host_info, block)}; if (!track_addr) { throw NotImplementedException("Failed to track bindless texture constant buffer"); @@ -671,6 +725,7 @@ void TexturePass(Environment& env, IR::Program& program, const HostTranslateInfo env.track_cache.clear(); TextureInstVector to_replace; for (IR::Block* const block : program.post_order_blocks) { + env.track_cache.clear(); for (IR::Inst& inst : block->Instructions()) { if (!IsTextureInstruction(inst)) { continue; diff --git a/src/video_core/buffer_cache/buffer_cache.h b/src/video_core/buffer_cache/buffer_cache.h index 53addb5dec..f30c85911f 100644 --- a/src/video_core/buffer_cache/buffer_cache.h +++ b/src/video_core/buffer_cache/buffer_cache.h @@ -1738,36 +1738,64 @@ void BufferCache

::TouchBuffer(Buffer& buffer, BufferId buffer_id) noexcept { template bool BufferCache

::SynchronizeBuffer(Buffer& buffer, DAddr device_addr, u32 size) { - upload_copies.clear(); - u64 total_size_bytes = 0; - u64 largest_copy = 0; - const DAddr buffer_start = buffer.cpu_addr_cached; - const auto add_upload = [&](DAddr start, DAddr end) { - if (start == end) return; - const u64 range_size = end - start; - upload_copies.push_back(BufferCopy{ - .src_offset = total_size_bytes, - .dst_offset = start - buffer_start, - .size = range_size, +#ifdef __ANDROID__ + const bool use_optimized_vertex_buffers = Settings::values.use_optimized_vertex_buffers.GetValue(); +#else + constexpr bool use_optimized_vertex_buffers = true; +#endif + + if (use_optimized_vertex_buffers) { + upload_copies.clear(); + u64 total_size_bytes = 0; + u64 largest_copy = 0; + const DAddr buffer_start = buffer.cpu_addr_cached; + const auto add_upload = [&](DAddr start, DAddr end) { + if (start == end) return; + const u64 range_size = end - start; + upload_copies.push_back(BufferCopy{ + .src_offset = total_size_bytes, + .dst_offset = start - buffer_start, + .size = range_size, + }); + total_size_bytes += range_size; + largest_copy = (std::max)(largest_copy, range_size); + }; + memory_tracker.ForEachUploadRange(device_addr, size, [&](u64 device_addr_out, u64 range_size) { + DAddr upload_start = device_addr_out; + gpu_modified_ranges.ForEachInRange(device_addr_out, range_size, [&](DAddr gpu_start, DAddr gpu_end) { + add_upload(upload_start, gpu_start); + upload_start = gpu_end; + }); + add_upload(upload_start, device_addr_out + range_size); }); - total_size_bytes += range_size; - largest_copy = (std::max)(largest_copy, range_size); - }; - memory_tracker.ForEachUploadRange(device_addr, size, [&](u64 device_addr_out, u64 range_size) { - DAddr upload_start = device_addr_out; - gpu_modified_ranges.ForEachInRange(device_addr_out, range_size, [&](DAddr gpu_start, DAddr gpu_end) { - add_upload(upload_start, gpu_start); - upload_start = gpu_end; + if (total_size_bytes == 0) { + return true; + } + const std::span copies_span(upload_copies.data(), upload_copies.size()); + UploadMemory(buffer, total_size_bytes, largest_copy, copies_span); + any_buffer_uploaded = true; + return false; + } else { + boost::container::small_vector copies; + u64 total_size_bytes = 0; + u64 largest_copy = 0; + DAddr buffer_start = buffer.CpuAddr(); + memory_tracker.ForEachUploadRange(device_addr, size, [&](u64 device_addr_out, u64 range_size) { + copies.push_back(BufferCopy{ + .src_offset = total_size_bytes, + .dst_offset = device_addr_out - buffer_start, + .size = range_size, + }); + total_size_bytes += range_size; + largest_copy = (std::max)(largest_copy, range_size); }); - add_upload(upload_start, device_addr_out + range_size); - }); - if (total_size_bytes == 0) { - return true; + if (total_size_bytes == 0) { + return true; + } + const std::span copies_span(copies.data(), copies.size()); + UploadMemory(buffer, total_size_bytes, largest_copy, copies_span); + return false; } - const std::span copies_span(upload_copies.data(), upload_copies.size()); - UploadMemory(buffer, total_size_bytes, largest_copy, copies_span); - any_buffer_uploaded = true; - return false; } template diff --git a/src/video_core/gpu.cpp b/src/video_core/gpu.cpp index 4e36d086e6..1621b24125 100644 --- a/src/video_core/gpu.cpp +++ b/src/video_core/gpu.cpp @@ -227,17 +227,17 @@ struct GPU::Impl { /// Notify rasterizer that any caches of the specified region should be invalidated void InvalidateRegion(DAddr addr, u64 size, bool preserve_gpu_writes) { - if (Settings::values.nce_invalidation_gpu_readback.GetValue()) { + if (preserve_gpu_writes) { VideoCore::RasterizerInterface* rasterizer = renderer->ReadRasterizer(); - if (preserve_gpu_writes && rasterizer->MustFlushRegion(addr, size, VideoCommon::CacheType::BufferCache)) { + if (!rasterizer->TryCacheInvalidation(addr, size)) { const u64 fence = RequestSyncOperation([rasterizer, addr, size] { rasterizer->FlushRegion(addr, size, VideoCommon::CacheType::BufferCache); rasterizer->OnCacheInvalidation(addr, size); }); gpu_thread.TickGPU(is_async); WaitForSyncOperation(fence); - return; } + return; } gpu_thread.InvalidateRegion(addr, size); } diff --git a/src/video_core/rasterizer_interface.h b/src/video_core/rasterizer_interface.h index 481efbf53b..b527c0e2a9 100644 --- a/src/video_core/rasterizer_interface.h +++ b/src/video_core/rasterizer_interface.h @@ -1,3 +1,6 @@ +// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project +// SPDX-License-Identifier: GPL-3.0-or-later + // SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project // SPDX-License-Identifier: GPL-2.0-or-later @@ -111,6 +114,12 @@ public: /// Notify rasterizer that any caches of the specified region are desync with guest virtual void OnCacheInvalidation(PAddr addr, u64 size) = 0; + virtual bool TryCacheInvalidation(DAddr addr, u64 size) { + if (MustFlushRegion(addr, size, VideoCommon::CacheType::BufferCache)) return false; + OnCacheInvalidation(addr, size); + return true; + } + virtual bool OnCPUWrite(PAddr addr, u64 size) = 0; /// Sync memory between guest and host. diff --git a/src/video_core/renderer_opengl/gl_shader_cache.cpp b/src/video_core/renderer_opengl/gl_shader_cache.cpp index df69eb97f5..277cf222fa 100644 --- a/src/video_core/renderer_opengl/gl_shader_cache.cpp +++ b/src/video_core/renderer_opengl/gl_shader_cache.cpp @@ -414,6 +414,11 @@ GraphicsPipeline* ShaderCache::BuiltPipeline(GraphicsPipeline* pipeline) const n if (!use_asynchronous_shaders) { return pipeline; } + // If something is using depth, we can assume that games are not rendering anything which + // will be used one time. + if (maxwell3d->regs.zeta_enable) { + return nullptr; + } // If games are using a small index count, we can assume these are full screen quads. // Usually these shaders are only used once for building textures so we can assume they // can't be built async diff --git a/src/video_core/renderer_vulkan/vk_pipeline_cache.cpp b/src/video_core/renderer_vulkan/vk_pipeline_cache.cpp index a85e2811a3..d4ffcd8317 100644 --- a/src/video_core/renderer_vulkan/vk_pipeline_cache.cpp +++ b/src/video_core/renderer_vulkan/vk_pipeline_cache.cpp @@ -447,7 +447,9 @@ PipelineCache::PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_, driver_id == VK_DRIVER_ID_AMD_OPEN_SOURCE || driver_id == VK_DRIVER_ID_MESA_RADV || driver_id == VK_DRIVER_ID_INTEL_PROPRIETARY_WINDOWS || - driver_id == VK_DRIVER_ID_INTEL_OPEN_SOURCE_MESA, + driver_id == VK_DRIVER_ID_INTEL_OPEN_SOURCE_MESA || + driver_id == VK_DRIVER_ID_QUALCOMM_PROPRIETARY || + driver_id == VK_DRIVER_ID_MESA_TURNIP, .has_broken_spirv_clamp = driver_id == VK_DRIVER_ID_INTEL_PROPRIETARY_WINDOWS, .has_broken_spirv_position_input = driver_id == VK_DRIVER_ID_QUALCOMM_PROPRIETARY, @@ -787,6 +789,11 @@ GraphicsPipeline* PipelineCache::BuiltPipeline(GraphicsPipeline* pipeline) const if (!use_asynchronous_shaders) { return pipeline; } + // If something is using depth, we can assume that games are not rendering anything which + // will be used one time. + if (maxwell3d->regs.zeta_enable) { + return nullptr; + } // If games are using a small index count, we can assume these are full screen quads. // Usually these shaders are only used once for building textures so we can assume they // can't be built async diff --git a/src/video_core/renderer_vulkan/vk_rasterizer.cpp b/src/video_core/renderer_vulkan/vk_rasterizer.cpp index f43bea1220..264ffb28b5 100644 --- a/src/video_core/renderer_vulkan/vk_rasterizer.cpp +++ b/src/video_core/renderer_vulkan/vk_rasterizer.cpp @@ -798,6 +798,18 @@ void RasterizerVulkan::OnCacheInvalidation(DAddr addr, u64 size) { pipeline_cache.InvalidateRegion(addr, size); } +bool RasterizerVulkan::TryCacheInvalidation(DAddr addr, u64 size) { + if (addr == 0 || size == 0) return true; + { + std::scoped_lock lock{buffer_cache.mutex, texture_cache.mutex}; + if (buffer_cache.IsRegionGpuModified(addr, size)) return false; + texture_cache.WriteMemory(addr, size); + buffer_cache.WriteMemory(addr, size); + } + pipeline_cache.InvalidateRegion(addr, size); + return true; +} + void RasterizerVulkan::InvalidateGPUCache() { gpu.InvalidateGPUCache(); } diff --git a/src/video_core/renderer_vulkan/vk_rasterizer.h b/src/video_core/renderer_vulkan/vk_rasterizer.h index 7470aa4f14..ae9929ca0b 100644 --- a/src/video_core/renderer_vulkan/vk_rasterizer.h +++ b/src/video_core/renderer_vulkan/vk_rasterizer.h @@ -106,6 +106,7 @@ public: VideoCommon::CacheType which = VideoCommon::CacheType::All) override; void InnerInvalidation(std::span> sequences) override; void OnCacheInvalidation(DAddr addr, u64 size) override; + bool TryCacheInvalidation(DAddr addr, u64 size) override; bool OnCPUWrite(DAddr addr, u64 size) override; void InvalidateGPUCache() override; void UnmapMemory(DAddr addr, u64 size) override; diff --git a/src/yuzu/configuration/configure_cpu.cpp b/src/yuzu/configuration/configure_cpu.cpp index 3834816bab..8c7b15e4e0 100644 --- a/src/yuzu/configuration/configure_cpu.cpp +++ b/src/yuzu/configuration/configure_cpu.cpp @@ -34,7 +34,7 @@ ConfigureCpu::ConfigureCpu(const Core::System& system_, connect(backend_combobox, qOverload(&QComboBox::currentIndexChanged), this, &ConfigureCpu::UpdateGroup); -#if defined(HAS_NCE) || defined(HAS_SCARLETNX) +#ifdef HAS_NCE ui->backend_group->setVisible(true); #endif } diff --git a/src/yuzu/main_window.cpp b/src/yuzu/main_window.cpp index 6e21265f9e..ad76e5b2db 100644 --- a/src/yuzu/main_window.cpp +++ b/src/yuzu/main_window.cpp @@ -2797,9 +2797,9 @@ void MainWindow::IncrementInstallProgress() { void MainWindow::OnMenuInstallToNAND() { const QString file_filter = - tr("Installable Switch File (*.nca *.nsp *.xci);;Nintendo Content Archive " - "(*.nca);;Nintendo Submission Package (*.nsp);;NX Cartridge " - "Image (*.xci)"); + tr("Installable Switch File (*.nca *.nsp *.nsz *.xci *.xcz);;Nintendo Content Archive " + "(*.nca);;Nintendo Submission Package (*.nsp *.nsz);;NX Cartridge " + "Image (*.xci *.xcz)"); QStringList filenames = QFileDialog::getOpenFileNames( this, tr("Install Files"), QString::fromStdString(UISettings::values.roms_path), @@ -2856,7 +2856,10 @@ void MainWindow::OnMenuInstallToNAND() { QFuture future; ContentManager::InstallResult result; - if (file.endsWith(QStringLiteral("nsp"), Qt::CaseInsensitive)) { + if (file.endsWith(QStringLiteral("nsp"), Qt::CaseInsensitive) || + file.endsWith(QStringLiteral("nsz"), Qt::CaseInsensitive) || + file.endsWith(QStringLiteral("xci"), Qt::CaseInsensitive) || + file.endsWith(QStringLiteral("xcz"), Qt::CaseInsensitive)) { const auto progress_callback = [this](size_t size, size_t progress) { emit UpdateInstallProgress(); if (install_progress->wasCanceled()) {