Configure DLSS for window and offscreen rendering

agent_context/MAP.yaml

@@ -9,7 +9,8 @@ topics:
 - id: simulation-system
   title: Simulation System
   aliases: [simulation system, simulation manager, motion module, sim motion, 运动能力模块, 仿真系统, 仿真管理器]
-  keywords: [SimulationManager, SimulationManagerCfg, explicit physics stepping, arena, DexSim, ArticulationJointKinematics,
+  keywords: [SimulationManager, SimulationManagerCfg, RenderCfg, DLSSCfg, DLSS, frame_time_delta_ms,
+    offscreen_dlss_enabled, explicit physics stepping, arena, DexSim, ArticulationJointKinematics,
     get_parent_joint_chain, enable_gravity]
   paths: [topics/simulation-system/simulation-system.md]
   source_of_truth:

agent_context/topics/simulation-system/simulation-system.md

@@ -141,8 +141,88 @@ asset registry and environment count, then coordinates creation and attachment.
 `qpos_joint_names`. Stochastic surface sampling and Atomic Action geometry keys
 do not belong to the simulation object; use
 `atomic_actions.sample_initial_articulation_geometry()` for that adaptation.
+
 Entity/IK gizmo configuration is owned by [native gizmos](../sim-visualization/native-gizmos.md).
 
+## Configuration Flow
+
+### Gizmo ownership
+
+Native entity manipulation belongs to DexSim 0.5.0. The first successful native
+window open enables its world-owned `EntityGizmoManipulator` by default, after
+the scene and default plane are ready. The manager registers the default plane
+as a static external target. `SimulationManagerCfg.enable_entity_gizmo=False`
+opts out; Gym deployments accept the same top-level JSON/YAML field through
+`gym.utils.gym_utils.config_to_cfg()`.
+
+`sim.enable_entity_gizmo(config)` explicitly enables/configures the controller;
+`sim.disable_entity_gizmo()` also cancels pending automatic enablement before
+the first window. These explicit calls take precedence over the startup default.
+Query through `sim.get_world().get_entity_gizmo()`. DexSim owns window
+detach/reopen and controller state; reopening never reapplies the default or
+overwrites an explicit native disable. Pure headless and Viser runs do not
+automatically create a native entity controller.
+
+`SimulationManagerCfg.robot_ik_gizmo` defaults to `GizmoCfg()`. During normal
+updates the manager registers robot control parts with complete solver chain/TCP
+metadata in single-environment interactive runs. Pure headless, read-only Viser, and
+multi-environment runs do not register automatic controls. The first native I
+press creates DexSim's `IKGizmoController` by default;
+`GizmoCfg(ik_start_enabled=True)` opts into activation on the first update with
+an open window. The startup attempt is consumed once, including on failure;
+later key presses can retry. Viser constructs IK on its first drag.
+Registration never writes drive targets. `Gizmo` owns managed native input and
+target-node cleanup, detaches input on window close, and reattaches the same
+controller on reopen. Robot removal releases all its managed controls.
+
+Set `robot_ik_gizmo=None` to opt out or supply `GizmoCfg` overrides; Gym
+JSON/YAML accepts the same mapping/null. `enable_gizmo()` can override one part,
+and `disable_gizmo()` prevents automatic recreation (all parts when omitted).
+The explicit `create_robot_ik_gizmo_controller()` factory still returns
+caller-owned controllers; a weak registry prevents automatic duplicates.
+Both robot paths
+default to native Newton IK; `GizmoCfg(ik_solver="embodichain")` adapts the
+control part's existing solver, such as PinkSolver. Both support one environment
+and write only selected non-mimic joint drive targets through `Robot`.
+
+`SimulationManagerCfg` owns window size, headless mode, rendering, GPU/CPU
+selection, arena count and spacing, physics timestep, physics and GPU-memory
+settings, recording, profiling, and browser visualization.
+
+`EnvCfg` embeds `SimulationManagerCfg` and supplies the control-to-physics
+step ratio. CLI and task config loaders may override runtime fields before
+constructing the environment. Trace those overrides through the caller rather
+than changing a default in the manager blindly.
+
+`RenderCfg.apply_to_dexsim_config()` owns renderer, sampling, tone mapping,
+and `DLSSCfg` conversion into `WorldConfig`. DLSS settings apply to `hybrid`,
+`fast-rt`, and `rt`, after automatic renderer resolution. Defaults enable
+window and offscreen DLSS, with independent RR/SR switches,
+Balanced quality, and zero render dimensions for engine-derived scaling.
+Always forward the master switch, including `False`. Headless initialization
+must retain DLSS settings because offscreen cameras or a later window can use
+them. The actual window/camera owns output size; compatibility target fields
+must not resize it. Explicit internal dimensions and `upsample_ratio` only
+affect FastRT/OfflineRT windows; hybrid and offscreen cameras derive internal
+size from their own output and quality. DexSim initializes DLSS lazily on a
+rendered frame, so config tests do not qualify GPU/NGX support.
+
+`gym/utils/gym_utils.py:config_to_cfg()` decodes task `render_cfg.dlss`
+mappings into `DLSSCfg` before constructing `RenderCfg`. DLSS switches require
+booleans; ratio/exposure settings require real numbers, excluding booleans.
+Malformed scalar types raise a field-specific `ValueError` during construction
+and are rechecked before native conversion after mutable config edits. Focused
+coverage lives in `tests/sim/test_cfg.py`, `tests/sim/test_sim_manager.py`, and
+`tests/gym/utils/test_gym_utils.py`.
+
+Object-specific configuration belongs in `lab/sim/cfg.py` or the
+corresponding robot/sensor module. Scene composition belongs in
+`EmbodiedEnv` or a task config, not in `SimulationManagerCfg`.
+
+For mesh collision decomposition, `MeshCfg.acd_method` defaults to `"visacd"`
+with DexSim 0.5.0; it requires CUDA support. `"coacd"` and `"vhacd"` remain
+supported explicit options.
+
 ## Where to Make Changes
 
 | Change | Primary location |

docs/source/api_reference/embodichain/embodichain.lab.sim.rst

@@ -145,3 +145,16 @@ Utility
    :maxdepth: 1
 
    embodichain.lab.sim.utility
+
+DLSS Configuration
+------------------
+
+.. currentmodule:: embodichain.lab.sim
+
+Configure window and offscreen Ray Reconstruction and Super Resolution through
+``SimulationManagerCfg.render_cfg.dlss``. Output resolution remains owned by the
+window or camera configuration.
+
+.. autoclass:: DLSSCfg
+   :members:
+   :undoc-members:

docs/source/overview/sim/sim_manager.md

@@ -62,58 +62,21 @@ PCM and TGS remain enabled, enhanced determinism remains disabled, and friction
 is evaluated on every solver iteration. These solver implementation details use
 fixed defaults and are not exposed by `PhysicsCfg`.
 
-### Render Configuration
+### Rendering
 
-The {class}`~cfg.RenderCfg` class controls the rendering backend and quality settings.
+Rendering configuration and advanced renderer features live in the dedicated
+{doc}`sim_manager/rendering/index` section. Start with
+{doc}`sim_manager/rendering/configuration` for renderer selection and common
+image-quality settings, then see {doc}`sim_manager/rendering/dlss` for DLSS
+behavior, quality modes, frame timing, and
+availability/fallback notes.
 
-| Parameter | Type | Default | Description |
-| :--- | :--- | :--- | :--- |
-| `renderer` | `str` | `"auto"` | Renderer backend to use. Options are `'auto'` (pick a default based on the detected GPU), `'hybrid'` (ray tracing for shadows/reflections + rasterization), `'fast-rt'` (full ray tracing), and `'rt'` (offline ray-traced renderer for maximum visual fidelity). |
-| `spp` | `int` | `1` | Samples per pixel for ray-traced rendering. Must be at least 1. |
-| `tone_mapping_enabled` | `bool` | `False` | Whether to map HDR RGB output with the modified Reinhard curve. |
-| `tone_mapping_exposure` | `float` | `1.0` | Non-negative fixed linear exposure multiplier applied before tone mapping. |
-
-Ray-traced output always uses DexSim's default OptiX denoiser. Tone mapping
-affects RGB output only; depth, segmentation masks, normals, and position
-buffers remain unchanged.
-
-#### Automatic Renderer Selection
-
-By default (`renderer="auto"`), EmbodiChain selects the renderer based on the GPU detected at the configured `gpu_id` when the {class}`SimulationManager` is constructed:
+```{toctree}
+:maxdepth: 2
 
-| GPU class | Examples | Selected renderer |
-| :--- | :--- | :--- |
-| RTX-series (consumer/workstation) | RTX 4090, RTX 6000 Ada | `hybrid` |
-| Datacenter accelerators | A100, A800, H100, H800, H200, H20 | `fast-rt` |
-| No CUDA device / unknown GPU | — | `hybrid` (fallback) |
-
-You can override the global default at runtime — useful for forcing a renderer across all simulations regardless of hardware:
-
-```python
-from embodichain.lab.sim import SimulationManager
-
-# Resolve the default from the current GPU, or force a specific backend.
-SimulationManager.set_default_renderer("auto")       # auto-detect from GPU
-SimulationManager.set_default_renderer("fast-rt")    # force full ray tracing
+sim_manager/rendering/index
 ```
 
-Setting `render_cfg.renderer` explicitly always takes precedence over auto-selection:
-
-```python
-from embodichain.lab.sim import SimulationManagerCfg
-from embodichain.lab.sim.cfg import RenderCfg
-
-sim_config = SimulationManagerCfg(
-    render_cfg=RenderCfg(
-        renderer="fast-rt",         # Override automatic renderer selection
-        spp=4,                      # Render four samples per pixel
-        tone_mapping_enabled=True,  # Convert HDR RGB to display-referred RGB
-        tone_mapping_exposure=1.0,  # Fixed exposure for reproducible frames
-    )
-)
-```
-
-
 ## Initialization
 
 Initialize the manager with the configuration object:

docs/source/overview/sim/sim_manager/rendering/configuration.md

@@ -0,0 +1,59 @@
+# Rendering Configuration
+
+The {class}`~embodichain.lab.sim.cfg.RenderCfg` class controls the renderer,
+ray-tracing sample count, tone mapping, and DLSS settings used by
+{class}`~embodichain.lab.sim.sim_manager.SimulationManager`.
+
+## Core options
+
+| Parameter | Type | Default | Description |
+| :--- | :--- | :--- | :--- |
+| `renderer` | `str` | `"auto"` | Renderer backend: `auto`, `hybrid`, `fast-rt`, or `rt`. |
+| `spp` | `int` | `1` | Samples per pixel for ray-traced rendering. Must be at least `1`. |
+| `tone_mapping_enabled` | `bool` | `False` | Apply modified Reinhard tone mapping to RGB output. |
+| `tone_mapping_exposure` | `float` | `1.0` | Fixed linear exposure multiplier used before tone mapping. |
+| `dlss` | `DLSSCfg` | `DLSSCfg()` | NVIDIA DLSS settings. See {doc}`dlss`. |
+
+Ray-traced output uses DexSim's OptiX denoiser. Tone mapping affects RGB
+output only; depth, segmentation masks, normals, and position buffers remain
+unchanged.
+
+## Renderer selection
+
+With `renderer="auto"`, EmbodiChain selects a backend from the GPU detected at
+the configured `gpu_id` when the simulation manager is constructed:
+
+| GPU class | Examples | Selected renderer |
+| :--- | :--- | :--- |
+| RTX-series consumer/workstation GPUs | RTX 4090, RTX 6000 Ada | `hybrid` |
+| Datacenter accelerators | A100, A800, H100, H800, H200, H20 | `fast-rt` |
+| No CUDA device or unknown GPU | — | `hybrid` |
+
+An explicit renderer always takes precedence over automatic selection. The
+process-wide default can also be changed through
+`SimulationManager.set_default_renderer()`:
+
+```python
+from embodichain.lab.sim import SimulationManager
+
+SimulationManager.set_default_renderer("auto")
+SimulationManager.set_default_renderer("fast-rt")
+```
+
+## Configuration example
+
+```python
+from embodichain.lab.sim import SimulationManagerCfg
+from embodichain.lab.sim.cfg import RenderCfg
+
+sim_config = SimulationManagerCfg(
+    render_cfg=RenderCfg(
+        renderer="fast-rt",
+        spp=4,
+        tone_mapping_enabled=True,
+        tone_mapping_exposure=1.0,
+    )
+)
+```
+
+For DLSS-specific settings, see {doc}`dlss`.

docs/source/overview/sim/sim_manager/rendering/dlss.md

@@ -0,0 +1,139 @@
+# NVIDIA DLSS
+
+EmbodiChain exposes the core NVIDIA DLSS controls through
+{class}`~embodichain.lab.sim.cfg.DLSSCfg` and passes them to DexSim when the
+{class}`~embodichain.lab.sim.sim_manager.SimulationManager` creates its world.
+The integration applies to the `hybrid`, `fast-rt`, and `rt` renderers. DexSim
+owns DLSS feature detection, initialization, temporal history, and fallback;
+constructing a configuration object does not by itself initialize DLSS.
+
+## Processing model
+
+DLSS has two selectable processing modes in the current integration:
+
+- **Ray Reconstruction (RR)** denoises ray-traced input and reconstructs the
+  output image.
+- **Super Resolution (SR)** upscales the image when RR is disabled.
+
+RR and SR are exposed as separate switches, but they are not chained in one
+frame. RR takes precedence when it is enabled:
+
+| RR | SR | Effective path |
+| :---: | :---: | :--- |
+| Enabled | Enabled | RR performs denoising and reconstruction. |
+| Enabled | Disabled | RR runs without a separate SR stage. |
+| Disabled | Enabled | Standalone SR performs the upscale. |
+| Disabled | Disabled | The standard OptiX denoiser/rendering path is used. |
+
+## EmbodiChain configuration
+
+The following fields are available under `RenderCfg.dlss`:
+
+| Parameter | Default | Description |
+| :--- | :---: | :--- |
+| `dlss_enabled` | `True` | Master switch for DLSS on window and offscreen targets. |
+| `offscreen_dlss_enabled` | `True` | Enables DLSS for offscreen camera outputs, including headless simulations. |
+| `rayreconstruction_enabled` | `True` | Enables RR denoising and reconstruction. |
+| `upscale_enabled` | `True` | Enables standalone SR when RR is disabled. |
+| `dlss_quality` | `2` | Quality preset: `-1` auto, `0` ultra performance, `1` performance, `2` balanced, `3` quality, `4` ultra quality, or `5` DLAA. |
+| `render_width`, `render_height` | `0` | Optional internal dimensions for FastRT/OfflineRT windows. Zero derives the dimensions from the quality preset. |
+| `target_width`, `target_height` | `0` | Compatibility fields. Set the actual output size on the window or camera instead. |
+| `upsample_ratio` | `None` | Optional FastRT/OfflineRT ratio used to derive unset internal dimensions. |
+| `exposure_compensation` | `1.0` | Positive exposure multiplier used by the RR bridge. |
+| `frame_time_delta_ms` | `0.0` | Render-frame interval in milliseconds. Zero selects DexSim's automatic measurement; a positive value supplies a fixed interval. |
+
+`frame_time_delta_ms` is deliberately defaulted to `0.0`, matching DexSim's
+native `DLSSConfig` default. DexSim measures the elapsed time between rendered
+frames and uses it in the temporal path. This value describes render cadence,
+not the physics or control timestep, so it should normally remain `0.0`.
+Specify a positive value only when the application intentionally renders at a
+known fixed cadence.
+
+EmbodiChain currently mirrors the core DLSS controls only. DexSim's advanced
+multi-camera tiled controls are intentionally not duplicated in
+`DLSSCfg`; their native defaults remain in effect while the engine manages
+camera-group rendering.
+
+## Quality and resolution
+
+The quality preset determines the internal render resolution relative to the
+requested output:
+
+| Value | Mode | Approximate internal size |
+| :---: | :--- | :---: |
+| `-1` | Auto | Balanced-safe scale followed by NGX mode selection |
+| `0` | Ultra Performance | 33% of output |
+| `1` | Performance | 50% of output |
+| `2` | Balanced | 58% of output |
+| `3` | Quality | 67% of output |
+| `4` | Ultra Quality | 77% of output |
+| `5` | DLAA | 100% of output |
+
+Set the window output size with `SimulationManagerCfg.width` and
+`SimulationManagerCfg.height`, or set the output resolution in the camera
+configuration. FastRT/OfflineRT windows may additionally use explicit
+`render_width`/`render_height` values or `upsample_ratio`. Hybrid and offscreen
+targets derive their internal resolution from the target output and quality
+preset.
+
+## Examples
+
+Enable DLSS for offscreen camera observations in a headless simulation:
+
+```python
+from embodichain.lab.sim import DLSSCfg, SimulationManagerCfg
+from embodichain.lab.sim.cfg import RenderCfg
+
+sim_config = SimulationManagerCfg(
+    headless=True,
+    render_cfg=RenderCfg(
+        renderer="hybrid",
+        dlss=DLSSCfg(
+            dlss_enabled=True,
+            offscreen_dlss_enabled=True,
+            dlss_quality=3,
+        ),
+    ),
+)
+```
+
+Use a fixed 60 FPS render interval only when the rendering cadence is known
+and intentionally fixed:
+
+```python
+render_cfg = RenderCfg(
+    renderer="hybrid",
+    dlss=DLSSCfg(frame_time_delta_ms=16.667),
+)
+```
+
+Configure an OfflineRT window with an explicit internal resolution:
+
+```python
+sim_config = SimulationManagerCfg(
+    width=1920,
+    height=1080,
+    render_cfg=RenderCfg(
+        renderer="rt",
+        dlss=DLSSCfg(render_width=1280, render_height=720),
+    ),
+)
+```
+
+The same settings are available in task JSON/YAML under
+`render_cfg.dlss` (decoded into `env_cfg.sim_cfg.render_cfg.dlss`). Set
+`dlss_enabled: false` explicitly when a task must use the standard renderer
+path.
+
+## Availability and fallback
+
+DLSS requires a Vulkan render device, a compatible NVIDIA GPU and driver, and
+a DexSim build that includes the NGX runtime libraries. DexSim checks support
+during renderer startup. If DLSS is unavailable, it falls back to the OptiX
+denoiser and reports initialization or fallback in the engine log.
+
+For offscreen rendering, each enabled camera group may allocate temporal
+history and Vulkan exchange resources. Increasing the number or resolution of
+camera groups therefore increases GPU memory usage. Validate DLSS by rendering
+an eligible frame and checking the engine log; configuration conversion or
+world construction alone is not sufficient evidence that DLSS initialized.