diff --git a/agent_context/MAP.yaml b/agent_context/MAP.yaml index b670e0448..2745240de 100644 --- a/agent_context/MAP.yaml +++ b/agent_context/MAP.yaml @@ -9,7 +9,8 @@ topics: - id: simulation-system title: Simulation System aliases: [simulation system, simulation manager, motion module, sim motion, 运动能力模块, 仿真系统, 仿真管理器] - keywords: [SimulationManager, SimulationManagerCfg, explicit physics stepping, arena, DexSim, ArticulationJointKinematics, + keywords: [SimulationManager, SimulationManagerCfg, RenderCfg, DLSSCfg, DLSS, frame_time_delta_ms, + offscreen_dlss_enabled, explicit physics stepping, arena, DexSim, ArticulationJointKinematics, get_parent_joint_chain, enable_gravity] paths: [topics/simulation-system/simulation-system.md] source_of_truth: diff --git a/agent_context/topics/simulation-system/simulation-system.md b/agent_context/topics/simulation-system/simulation-system.md index c73d6262a..f62c19bdd 100644 --- a/agent_context/topics/simulation-system/simulation-system.md +++ b/agent_context/topics/simulation-system/simulation-system.md @@ -141,8 +141,88 @@ asset registry and environment count, then coordinates creation and attachment. `qpos_joint_names`. Stochastic surface sampling and Atomic Action geometry keys do not belong to the simulation object; use `atomic_actions.sample_initial_articulation_geometry()` for that adaptation. + Entity/IK gizmo configuration is owned by [native gizmos](../sim-visualization/native-gizmos.md). +## Configuration Flow + +### Gizmo ownership + +Native entity manipulation belongs to DexSim 0.5.0. The first successful native +window open enables its world-owned `EntityGizmoManipulator` by default, after +the scene and default plane are ready. The manager registers the default plane +as a static external target. `SimulationManagerCfg.enable_entity_gizmo=False` +opts out; Gym deployments accept the same top-level JSON/YAML field through +`gym.utils.gym_utils.config_to_cfg()`. + +`sim.enable_entity_gizmo(config)` explicitly enables/configures the controller; +`sim.disable_entity_gizmo()` also cancels pending automatic enablement before +the first window. These explicit calls take precedence over the startup default. +Query through `sim.get_world().get_entity_gizmo()`. DexSim owns window +detach/reopen and controller state; reopening never reapplies the default or +overwrites an explicit native disable. Pure headless and Viser runs do not +automatically create a native entity controller. + +`SimulationManagerCfg.robot_ik_gizmo` defaults to `GizmoCfg()`. During normal +updates the manager registers robot control parts with complete solver chain/TCP +metadata in single-environment interactive runs. Pure headless, read-only Viser, and +multi-environment runs do not register automatic controls. The first native I +press creates DexSim's `IKGizmoController` by default; +`GizmoCfg(ik_start_enabled=True)` opts into activation on the first update with +an open window. The startup attempt is consumed once, including on failure; +later key presses can retry. Viser constructs IK on its first drag. +Registration never writes drive targets. `Gizmo` owns managed native input and +target-node cleanup, detaches input on window close, and reattaches the same +controller on reopen. Robot removal releases all its managed controls. + +Set `robot_ik_gizmo=None` to opt out or supply `GizmoCfg` overrides; Gym +JSON/YAML accepts the same mapping/null. `enable_gizmo()` can override one part, +and `disable_gizmo()` prevents automatic recreation (all parts when omitted). +The explicit `create_robot_ik_gizmo_controller()` factory still returns +caller-owned controllers; a weak registry prevents automatic duplicates. +Both robot paths +default to native Newton IK; `GizmoCfg(ik_solver="embodichain")` adapts the +control part's existing solver, such as PinkSolver. Both support one environment +and write only selected non-mimic joint drive targets through `Robot`. + +`SimulationManagerCfg` owns window size, headless mode, rendering, GPU/CPU +selection, arena count and spacing, physics timestep, physics and GPU-memory +settings, recording, profiling, and browser visualization. + +`EnvCfg` embeds `SimulationManagerCfg` and supplies the control-to-physics +step ratio. CLI and task config loaders may override runtime fields before +constructing the environment. Trace those overrides through the caller rather +than changing a default in the manager blindly. + +`RenderCfg.apply_to_dexsim_config()` owns renderer, sampling, tone mapping, +and `DLSSCfg` conversion into `WorldConfig`. DLSS settings apply to `hybrid`, +`fast-rt`, and `rt`, after automatic renderer resolution. Defaults enable +window and offscreen DLSS, with independent RR/SR switches, +Balanced quality, and zero render dimensions for engine-derived scaling. +Always forward the master switch, including `False`. Headless initialization +must retain DLSS settings because offscreen cameras or a later window can use +them. The actual window/camera owns output size; compatibility target fields +must not resize it. Explicit internal dimensions and `upsample_ratio` only +affect FastRT/OfflineRT windows; hybrid and offscreen cameras derive internal +size from their own output and quality. DexSim initializes DLSS lazily on a +rendered frame, so config tests do not qualify GPU/NGX support. + +`gym/utils/gym_utils.py:config_to_cfg()` decodes task `render_cfg.dlss` +mappings into `DLSSCfg` before constructing `RenderCfg`. DLSS switches require +booleans; ratio/exposure settings require real numbers, excluding booleans. +Malformed scalar types raise a field-specific `ValueError` during construction +and are rechecked before native conversion after mutable config edits. Focused +coverage lives in `tests/sim/test_cfg.py`, `tests/sim/test_sim_manager.py`, and +`tests/gym/utils/test_gym_utils.py`. + +Object-specific configuration belongs in `lab/sim/cfg.py` or the +corresponding robot/sensor module. Scene composition belongs in +`EmbodiedEnv` or a task config, not in `SimulationManagerCfg`. + +For mesh collision decomposition, `MeshCfg.acd_method` defaults to `"visacd"` +with DexSim 0.5.0; it requires CUDA support. `"coacd"` and `"vhacd"` remain +supported explicit options. + ## Where to Make Changes | Change | Primary location | diff --git a/docs/source/api_reference/embodichain/embodichain.lab.sim.rst b/docs/source/api_reference/embodichain/embodichain.lab.sim.rst index a08de3cfb..2d0874cfb 100644 --- a/docs/source/api_reference/embodichain/embodichain.lab.sim.rst +++ b/docs/source/api_reference/embodichain/embodichain.lab.sim.rst @@ -145,3 +145,16 @@ Utility :maxdepth: 1 embodichain.lab.sim.utility + +DLSS Configuration +------------------ + +.. currentmodule:: embodichain.lab.sim + +Configure window and offscreen Ray Reconstruction and Super Resolution through +``SimulationManagerCfg.render_cfg.dlss``. Output resolution remains owned by the +window or camera configuration. + +.. autoclass:: DLSSCfg + :members: + :undoc-members: diff --git a/docs/source/overview/sim/sim_manager.md b/docs/source/overview/sim/sim_manager.md index 7fd2ec9ad..02399a5a4 100644 --- a/docs/source/overview/sim/sim_manager.md +++ b/docs/source/overview/sim/sim_manager.md @@ -62,58 +62,21 @@ PCM and TGS remain enabled, enhanced determinism remains disabled, and friction is evaluated on every solver iteration. These solver implementation details use fixed defaults and are not exposed by `PhysicsCfg`. -### Render Configuration +### Rendering -The {class}`~cfg.RenderCfg` class controls the rendering backend and quality settings. +Rendering configuration and advanced renderer features live in the dedicated +{doc}`sim_manager/rendering/index` section. Start with +{doc}`sim_manager/rendering/configuration` for renderer selection and common +image-quality settings, then see {doc}`sim_manager/rendering/dlss` for DLSS +behavior, quality modes, frame timing, and +availability/fallback notes. -| Parameter | Type | Default | Description | -| :--- | :--- | :--- | :--- | -| `renderer` | `str` | `"auto"` | Renderer backend to use. Options are `'auto'` (pick a default based on the detected GPU), `'hybrid'` (ray tracing for shadows/reflections + rasterization), `'fast-rt'` (full ray tracing), and `'rt'` (offline ray-traced renderer for maximum visual fidelity). | -| `spp` | `int` | `1` | Samples per pixel for ray-traced rendering. Must be at least 1. | -| `tone_mapping_enabled` | `bool` | `False` | Whether to map HDR RGB output with the modified Reinhard curve. | -| `tone_mapping_exposure` | `float` | `1.0` | Non-negative fixed linear exposure multiplier applied before tone mapping. | - -Ray-traced output always uses DexSim's default OptiX denoiser. Tone mapping -affects RGB output only; depth, segmentation masks, normals, and position -buffers remain unchanged. - -#### Automatic Renderer Selection - -By default (`renderer="auto"`), EmbodiChain selects the renderer based on the GPU detected at the configured `gpu_id` when the {class}`SimulationManager` is constructed: +```{toctree} +:maxdepth: 2 -| GPU class | Examples | Selected renderer | -| :--- | :--- | :--- | -| RTX-series (consumer/workstation) | RTX 4090, RTX 6000 Ada | `hybrid` | -| Datacenter accelerators | A100, A800, H100, H800, H200, H20 | `fast-rt` | -| No CUDA device / unknown GPU | — | `hybrid` (fallback) | - -You can override the global default at runtime — useful for forcing a renderer across all simulations regardless of hardware: - -```python -from embodichain.lab.sim import SimulationManager - -# Resolve the default from the current GPU, or force a specific backend. -SimulationManager.set_default_renderer("auto") # auto-detect from GPU -SimulationManager.set_default_renderer("fast-rt") # force full ray tracing +sim_manager/rendering/index ``` -Setting `render_cfg.renderer` explicitly always takes precedence over auto-selection: - -```python -from embodichain.lab.sim import SimulationManagerCfg -from embodichain.lab.sim.cfg import RenderCfg - -sim_config = SimulationManagerCfg( - render_cfg=RenderCfg( - renderer="fast-rt", # Override automatic renderer selection - spp=4, # Render four samples per pixel - tone_mapping_enabled=True, # Convert HDR RGB to display-referred RGB - tone_mapping_exposure=1.0, # Fixed exposure for reproducible frames - ) -) -``` - - ## Initialization Initialize the manager with the configuration object: diff --git a/docs/source/overview/sim/sim_manager/rendering/configuration.md b/docs/source/overview/sim/sim_manager/rendering/configuration.md new file mode 100644 index 000000000..776dba9ef --- /dev/null +++ b/docs/source/overview/sim/sim_manager/rendering/configuration.md @@ -0,0 +1,59 @@ +# Rendering Configuration + +The {class}`~embodichain.lab.sim.cfg.RenderCfg` class controls the renderer, +ray-tracing sample count, tone mapping, and DLSS settings used by +{class}`~embodichain.lab.sim.sim_manager.SimulationManager`. + +## Core options + +| Parameter | Type | Default | Description | +| :--- | :--- | :--- | :--- | +| `renderer` | `str` | `"auto"` | Renderer backend: `auto`, `hybrid`, `fast-rt`, or `rt`. | +| `spp` | `int` | `1` | Samples per pixel for ray-traced rendering. Must be at least `1`. | +| `tone_mapping_enabled` | `bool` | `False` | Apply modified Reinhard tone mapping to RGB output. | +| `tone_mapping_exposure` | `float` | `1.0` | Fixed linear exposure multiplier used before tone mapping. | +| `dlss` | `DLSSCfg` | `DLSSCfg()` | NVIDIA DLSS settings. See {doc}`dlss`. | + +Ray-traced output uses DexSim's OptiX denoiser. Tone mapping affects RGB +output only; depth, segmentation masks, normals, and position buffers remain +unchanged. + +## Renderer selection + +With `renderer="auto"`, EmbodiChain selects a backend from the GPU detected at +the configured `gpu_id` when the simulation manager is constructed: + +| GPU class | Examples | Selected renderer | +| :--- | :--- | :--- | +| RTX-series consumer/workstation GPUs | RTX 4090, RTX 6000 Ada | `hybrid` | +| Datacenter accelerators | A100, A800, H100, H800, H200, H20 | `fast-rt` | +| No CUDA device or unknown GPU | — | `hybrid` | + +An explicit renderer always takes precedence over automatic selection. The +process-wide default can also be changed through +`SimulationManager.set_default_renderer()`: + +```python +from embodichain.lab.sim import SimulationManager + +SimulationManager.set_default_renderer("auto") +SimulationManager.set_default_renderer("fast-rt") +``` + +## Configuration example + +```python +from embodichain.lab.sim import SimulationManagerCfg +from embodichain.lab.sim.cfg import RenderCfg + +sim_config = SimulationManagerCfg( + render_cfg=RenderCfg( + renderer="fast-rt", + spp=4, + tone_mapping_enabled=True, + tone_mapping_exposure=1.0, + ) +) +``` + +For DLSS-specific settings, see {doc}`dlss`. diff --git a/docs/source/overview/sim/sim_manager/rendering/dlss.md b/docs/source/overview/sim/sim_manager/rendering/dlss.md new file mode 100644 index 000000000..5aaa9eee2 --- /dev/null +++ b/docs/source/overview/sim/sim_manager/rendering/dlss.md @@ -0,0 +1,139 @@ +# NVIDIA DLSS + +EmbodiChain exposes the core NVIDIA DLSS controls through +{class}`~embodichain.lab.sim.cfg.DLSSCfg` and passes them to DexSim when the +{class}`~embodichain.lab.sim.sim_manager.SimulationManager` creates its world. +The integration applies to the `hybrid`, `fast-rt`, and `rt` renderers. DexSim +owns DLSS feature detection, initialization, temporal history, and fallback; +constructing a configuration object does not by itself initialize DLSS. + +## Processing model + +DLSS has two selectable processing modes in the current integration: + +- **Ray Reconstruction (RR)** denoises ray-traced input and reconstructs the + output image. +- **Super Resolution (SR)** upscales the image when RR is disabled. + +RR and SR are exposed as separate switches, but they are not chained in one +frame. RR takes precedence when it is enabled: + +| RR | SR | Effective path | +| :---: | :---: | :--- | +| Enabled | Enabled | RR performs denoising and reconstruction. | +| Enabled | Disabled | RR runs without a separate SR stage. | +| Disabled | Enabled | Standalone SR performs the upscale. | +| Disabled | Disabled | The standard OptiX denoiser/rendering path is used. | + +## EmbodiChain configuration + +The following fields are available under `RenderCfg.dlss`: + +| Parameter | Default | Description | +| :--- | :---: | :--- | +| `dlss_enabled` | `True` | Master switch for DLSS on window and offscreen targets. | +| `offscreen_dlss_enabled` | `True` | Enables DLSS for offscreen camera outputs, including headless simulations. | +| `rayreconstruction_enabled` | `True` | Enables RR denoising and reconstruction. | +| `upscale_enabled` | `True` | Enables standalone SR when RR is disabled. | +| `dlss_quality` | `2` | Quality preset: `-1` auto, `0` ultra performance, `1` performance, `2` balanced, `3` quality, `4` ultra quality, or `5` DLAA. | +| `render_width`, `render_height` | `0` | Optional internal dimensions for FastRT/OfflineRT windows. Zero derives the dimensions from the quality preset. | +| `target_width`, `target_height` | `0` | Compatibility fields. Set the actual output size on the window or camera instead. | +| `upsample_ratio` | `None` | Optional FastRT/OfflineRT ratio used to derive unset internal dimensions. | +| `exposure_compensation` | `1.0` | Positive exposure multiplier used by the RR bridge. | +| `frame_time_delta_ms` | `0.0` | Render-frame interval in milliseconds. Zero selects DexSim's automatic measurement; a positive value supplies a fixed interval. | + +`frame_time_delta_ms` is deliberately defaulted to `0.0`, matching DexSim's +native `DLSSConfig` default. DexSim measures the elapsed time between rendered +frames and uses it in the temporal path. This value describes render cadence, +not the physics or control timestep, so it should normally remain `0.0`. +Specify a positive value only when the application intentionally renders at a +known fixed cadence. + +EmbodiChain currently mirrors the core DLSS controls only. DexSim's advanced +multi-camera tiled controls are intentionally not duplicated in +`DLSSCfg`; their native defaults remain in effect while the engine manages +camera-group rendering. + +## Quality and resolution + +The quality preset determines the internal render resolution relative to the +requested output: + +| Value | Mode | Approximate internal size | +| :---: | :--- | :---: | +| `-1` | Auto | Balanced-safe scale followed by NGX mode selection | +| `0` | Ultra Performance | 33% of output | +| `1` | Performance | 50% of output | +| `2` | Balanced | 58% of output | +| `3` | Quality | 67% of output | +| `4` | Ultra Quality | 77% of output | +| `5` | DLAA | 100% of output | + +Set the window output size with `SimulationManagerCfg.width` and +`SimulationManagerCfg.height`, or set the output resolution in the camera +configuration. FastRT/OfflineRT windows may additionally use explicit +`render_width`/`render_height` values or `upsample_ratio`. Hybrid and offscreen +targets derive their internal resolution from the target output and quality +preset. + +## Examples + +Enable DLSS for offscreen camera observations in a headless simulation: + +```python +from embodichain.lab.sim import DLSSCfg, SimulationManagerCfg +from embodichain.lab.sim.cfg import RenderCfg + +sim_config = SimulationManagerCfg( + headless=True, + render_cfg=RenderCfg( + renderer="hybrid", + dlss=DLSSCfg( + dlss_enabled=True, + offscreen_dlss_enabled=True, + dlss_quality=3, + ), + ), +) +``` + +Use a fixed 60 FPS render interval only when the rendering cadence is known +and intentionally fixed: + +```python +render_cfg = RenderCfg( + renderer="hybrid", + dlss=DLSSCfg(frame_time_delta_ms=16.667), +) +``` + +Configure an OfflineRT window with an explicit internal resolution: + +```python +sim_config = SimulationManagerCfg( + width=1920, + height=1080, + render_cfg=RenderCfg( + renderer="rt", + dlss=DLSSCfg(render_width=1280, render_height=720), + ), +) +``` + +The same settings are available in task JSON/YAML under +`render_cfg.dlss` (decoded into `env_cfg.sim_cfg.render_cfg.dlss`). Set +`dlss_enabled: false` explicitly when a task must use the standard renderer +path. + +## Availability and fallback + +DLSS requires a Vulkan render device, a compatible NVIDIA GPU and driver, and +a DexSim build that includes the NGX runtime libraries. DexSim checks support +during renderer startup. If DLSS is unavailable, it falls back to the OptiX +denoiser and reports initialization or fallback in the engine log. + +For offscreen rendering, each enabled camera group may allocate temporal +history and Vulkan exchange resources. Increasing the number or resolution of +camera groups therefore increases GPU memory usage. Validate DLSS by rendering +an eligible frame and checking the engine log; configuration conversion or +world construction alone is not sufficient evidence that DLSS initialized. diff --git a/docs/source/overview/sim/sim_manager/rendering/index.rst b/docs/source/overview/sim/sim_manager/rendering/index.rst new file mode 100644 index 000000000..38d9eb42f --- /dev/null +++ b/docs/source/overview/sim/sim_manager/rendering/index.rst @@ -0,0 +1,19 @@ +Rendering +================================= + +This section documents EmbodiChain's rendering configuration and the advanced +features provided by the DexSim renderer. The simulation manager owns the +rendering lifecycle, while these pages describe renderer selection, image +quality, DLSS, and related performance trade-offs. + +Start with :doc:`configuration` for common ``RenderCfg`` options. Use +:doc:`dlss` for NVIDIA DLSS behavior and temporal-rendering settings. + +See also +-------- + +.. toctree:: + :maxdepth: 1 + + configuration.md + dlss.md diff --git a/embodichain/lab/gym/utils/gym_utils.py b/embodichain/lab/gym/utils/gym_utils.py index 15c714737..f6365201d 100644 --- a/embodichain/lab/gym/utils/gym_utils.py +++ b/embodichain/lab/gym/utils/gym_utils.py @@ -440,6 +440,7 @@ def config_to_cfg( ArticulationCfg, LightCfg, PhysicsCfg, + DLSSCfg, RenderCfg, ) from embodichain.lab.sim import SimulationManagerCfg @@ -603,6 +604,8 @@ class ComponentCfg: physics_config["gravity"] = np.asarray(physics_config["gravity"]) render_config = deepcopy(config.get("render_cfg", {})) + if isinstance(render_config.get("dlss"), dict): + render_config["dlss"] = DLSSCfg(**render_config["dlss"]) if "renderer" in config: # Keep the existing flat renderer option as the command-line override. render_config["renderer"] = config["renderer"] diff --git a/embodichain/lab/sim/__init__.py b/embodichain/lab/sim/__init__.py index 496832f17..cec451a87 100644 --- a/embodichain/lab/sim/__init__.py +++ b/embodichain/lab/sim/__init__.py @@ -27,6 +27,7 @@ VisualMaterialInst, ReuseSegmentState, ) +from .cfg import DLSSCfg from .common import BatchEntity from .profiler import Profiler, ProfilerCfg @@ -42,6 +43,7 @@ "ProfilerCfg", "SimulationManager", "SimulationManagerCfg", + "DLSSCfg", "SIM_CACHE_DIR", "MATERIAL_CACHE_DIR", "CONVEX_DECOMP_DIR", diff --git a/embodichain/lab/sim/cfg.py b/embodichain/lab/sim/cfg.py index a434af9da..05781b49f 100644 --- a/embodichain/lab/sim/cfg.py +++ b/embodichain/lab/sim/cfg.py @@ -18,6 +18,7 @@ import enum import json +import math import os import dexsim @@ -26,6 +27,7 @@ from typing import Sequence, Dict, Literal, List, Any, Optional from dataclasses import field, MISSING +from numbers import Real from dexsim.types import ( DenoiserType, @@ -49,6 +51,35 @@ from .shapes import ShapeCfg, MeshCfg from .motion.workspace.cfg import RobotWorkspaceCfg +__all__ = [ + "DEFAULT_RENDERER", + "DLSSCfg", + "RenderCfg", + "PhysicsCfg", + "MarkerCfg", + "WindowRecordCfg", + "WindowCameraPoseCfg", + "GPUMemoryCfg", + "RigidBodyAttributesCfg", + "RigidBodyAttributesOverrideCfg", + "LinkPhysicsOverrideCfg", + "link_attrs_from_dict", + "SoftbodyVoxelAttributesCfg", + "SoftbodyPhysicalAttributesCfg", + "ClothPhysicalAttributesCfg", + "JointDrivePropertiesCfg", + "ObjectBaseCfg", + "LightCfg", + "RigidObjectCfg", + "SoftObjectCfg", + "ClothObjectCfg", + "RigidObjectGroupCfg", + "RigidConstraintCfg", + "URDFCfg", + "ArticulationCfg", + "RobotCfg", +] + # Global default renderer settings for simulation. # # The sentinel value ``"auto"`` defers the choice to GPU-based auto-selection @@ -59,6 +90,141 @@ DEFAULT_RENDERER: Literal["auto", "hybrid", "fast-rt", "rt"] = "auto" +@configclass +class DLSSCfg: + """DexSim DLSS configuration for window and offscreen rendering. + + Ray Reconstruction (RR) and Super Resolution (SR) are independently + configurable on the ``"hybrid"``, ``"fast-rt"``, and ``"rt"`` renderers. + DLSS is enabled by default for both windows and offscreen cameras. + Offscreen DLSS also requires the master switch to remain enabled. + + .. attention:: + DLSS requires a Vulkan render device, a compatible NVIDIA GPU/driver, + and a DexSim build with the NGX runtime. Initialization is deferred + until rendering; configuration conversion alone cannot verify support. + Each enabled offscreen camera needs its own temporal history and + Vulkan exchange images, increasing GPU memory use. + """ + + dlss_enabled: bool = True + """Master switch for DLSS. False retains the standard rendering path.""" + + offscreen_dlss_enabled: bool = True + """Enable DLSS for offscreen cameras, including in headless simulations.""" + + rayreconstruction_enabled: bool = True + """Enable RR denoising. Can be used without SR at the target resolution.""" + + upscale_enabled: bool = True + """Enable SR upscaling. Can be used independently of RR.""" + + dlss_quality: int = 2 + """Quality mode and derived internal scale: ``-1`` auto (58%), ``0`` Ultra + Performance (~33%), ``1`` Performance (50%), ``2`` Balanced (58%), + ``3`` Quality (~67%), ``4`` Ultra Quality (77%), ``5`` DLAA (100%).""" + + upsample_ratio: float | None = None + """Optional window target/render ratio, at least 1.0. None leaves zero + render dimensions for DexSim to derive from quality. When specified, + computes each unset render dimension from the actual window size. Only + FastRT/OfflineRT windows honor these overrides; hybrid and offscreen + targets derive their internal resolution from quality.""" + + render_width: int = 0 + """Internal FastRT/OfflineRT window width; zero derives it from quality.""" + + render_height: int = 0 + """Internal FastRT/OfflineRT window height; zero derives it from quality.""" + + target_width: int = 0 + """DexSim compatibility field. Set the actual window or camera width instead.""" + + target_height: int = 0 + """DexSim compatibility field. Set the actual window or camera height instead.""" + + exposure_compensation: float = 1.0 + """Positive, finite exposure multiplier used by the RR bridge.""" + + frame_time_delta_ms: float = 0.0 + """Frame interval in milliseconds passed to DexSim's DLSS temporal path. + + The default ``0.0`` intentionally matches ``dexsim.DLSSConfig``: DexSim + measures the actual render interval automatically. Set a positive value + only for a fixed render cadence; this is a render-frame interval, not a + physics or control timestep. + """ + + def __post_init__(self) -> None: + """Validate scalar types and the ranges of numeric settings.""" + for name in ( + "dlss_enabled", + "offscreen_dlss_enabled", + "rayreconstruction_enabled", + "upscale_enabled", + ): + if not isinstance(getattr(self, name), bool): + raise ValueError(f"DLSSCfg.{name} must be a boolean.") + if type(self.dlss_quality) is not int or not -1 <= self.dlss_quality <= 5: + raise ValueError("DLSSCfg.dlss_quality must be an integer from -1 to 5.") + for name in ("render_width", "render_height", "target_width", "target_height"): + value = getattr(self, name) + if type(value) is not int or value < 0: + raise ValueError(f"DLSSCfg.{name} must be a non-negative integer.") + if self.upsample_ratio is not None and ( + isinstance(self.upsample_ratio, bool) + or not isinstance(self.upsample_ratio, Real) + or not math.isfinite(self.upsample_ratio) + or self.upsample_ratio < 1.0 + ): + raise ValueError( + "DLSSCfg.upsample_ratio must be a finite number of at least 1.0." + ) + if ( + isinstance(self.exposure_compensation, bool) + or not isinstance(self.exposure_compensation, Real) + or not math.isfinite(self.exposure_compensation) + or self.exposure_compensation <= 0.0 + ): + raise ValueError( + "DLSSCfg.exposure_compensation must be a positive, finite number." + ) + + def to_dexsim_cfg(self, window_width: int, window_height: int) -> dexsim.DLSSConfig: + """Convert settings without changing the window or camera output size. + + Args: + window_width: Window width in pixels. + window_height: Window height in pixels. + + Returns: + Populated :class:`dexsim.DLSSConfig` instance ready to assign to + ``world_config.dlss_config``. + + Raises: + ValueError: If the configuration contains invalid values. + """ + self.__post_init__() + dlss = dexsim.DLSSConfig() + dlss.dlss_enabled = self.dlss_enabled + dlss.offscreen_dlss_enabled = self.offscreen_dlss_enabled + dlss.rayreconstruction_enabled = self.rayreconstruction_enabled + dlss.upscale_enabled = self.upscale_enabled + dlss.dlss_quality = self.dlss_quality + dlss.render_width = self.render_width + dlss.render_height = self.render_height + if self.upsample_ratio is not None: + if self.render_width == 0: + dlss.render_width = max(1, int(window_width / self.upsample_ratio)) + if self.render_height == 0: + dlss.render_height = max(1, int(window_height / self.upsample_ratio)) + dlss.target_width = self.target_width + dlss.target_height = self.target_height + dlss.exposure_compensation = self.exposure_compensation + dlss.frame_time_delta_ms = self.frame_time_delta_ms + return dlss + + @configclass class RenderCfg: renderer: Literal["auto", "hybrid", "fast-rt", "rt"] = "auto" @@ -77,6 +243,9 @@ class RenderCfg: spp: int = 1 """Samples per pixel for ray tracing rendering. This parameter is only valid when renderer is 'hybrid', 'fast-rt' or 'rt'.""" + dlss: DLSSCfg = field(default_factory=DLSSCfg) + """DLSS settings for hybrid, fast-rt, and rt windows and offscreen cameras.""" + tone_mapping_enabled: bool = False """Whether to map HDR RGB output with the modified Reinhard curve.""" @@ -120,6 +289,10 @@ def apply_to_dexsim_config(self, world_config: dexsim.WorldConfig) -> None: world_config: DexSim world configuration to update in place. """ world_config.renderer = self.to_dexsim_flags() + world_config.dlss_config = self.dlss.to_dexsim_cfg( + window_width=world_config.win_config.width, + window_height=world_config.win_config.height, + ) world_config.raytrace_config.render_iterations_per_frame = self.spp world_config.raytrace_config.open_denoise = True world_config.raytrace_config.denoiser_type = DenoiserType.OPTIX diff --git a/tests/gym/utils/test_gym_utils.py b/tests/gym/utils/test_gym_utils.py index a5be09d5b..6fc8d18e8 100644 --- a/tests/gym/utils/test_gym_utils.py +++ b/tests/gym/utils/test_gym_utils.py @@ -1385,6 +1385,44 @@ def test_task_integration_rejects_removed_version_field( source_path=tmp_path / "env.yaml", ) + @pytest.mark.parametrize("extension", ["json", "yaml"]) + @pytest.mark.parametrize( + ("field_name", "invalid_value"), + [ + ("dlss_enabled", "false"), + ("offscreen_dlss_enabled", 0), + ("rayreconstruction_enabled", 1), + ("upscale_enabled", None), + ("upsample_ratio", "2.0"), + ("exposure_compensation", "1.0"), + ], + ) + def test_gym_config_rejects_wrongly_typed_dlss_scalars( + self, + tmp_path: Path, + extension: str, + field_name: str, + invalid_value: object, + ) -> None: + """Malformed file values fail at DLSS decoding with the offending field name.""" + config = { + "id": "EmbodiedEnv-v1", + "env": {}, + "robot": { + "class_type": "URRobot", + "robot_type": "ur5", + "uid": "TestUR5", + }, + "render_cfg": {"dlss": {field_name: invalid_value}}, + } + config_path = tmp_path / f"gym_config.{extension}" + save_config(config_path, config) + + with pytest.raises(ValueError, match=field_name): + config_to_cfg( + load_config(config_path), manager_modules=DEFAULT_MANAGER_MODULES + ) + @pytest.mark.parametrize("suffix", ["yaml", "json"]) @pytest.mark.parametrize("enabled", [None, False, True]) def test_gym_config_preserves_entity_gizmo_startup_preference( @@ -1453,6 +1491,13 @@ def test_yaml_gym_config_parses_to_cfg(self, tmp_path): "spp": 4, "tone_mapping_enabled": True, "tone_mapping_exposure": 1.25, + "dlss": { + "dlss_enabled": True, + "offscreen_dlss_enabled": True, + "rayreconstruction_enabled": False, + "upscale_enabled": True, + "dlss_quality": 1, + }, }, "visualization": { "backend": "viser", @@ -1517,6 +1562,17 @@ def test_yaml_gym_config_parses_to_cfg(self, tmp_path): assert cfg.sim_cfg.render_cfg.spp == 4 assert cfg.sim_cfg.render_cfg.tone_mapping_enabled is True assert cfg.sim_cfg.render_cfg.tone_mapping_exposure == 1.25 + from embodichain.lab.sim import DLSSCfg + import dexsim + + assert isinstance(cfg.sim_cfg.render_cfg.dlss, DLSSCfg) + world_config = dexsim.WorldConfig() + cfg.sim_cfg.render_cfg.apply_to_dexsim_config(world_config) + assert world_config.dlss_config.dlss_enabled is True + assert world_config.dlss_config.offscreen_dlss_enabled is True + assert world_config.dlss_config.rayreconstruction_enabled is False + assert world_config.dlss_config.upscale_enabled is True + assert world_config.dlss_config.dlss_quality == 1 assert cfg.sim_cfg.visualization.backend == "viser" assert cfg.sim_cfg.visualization.scene_fps == 12.5 assert cfg.sim_cfg.visualization.viser_server.host == "0.0.0.0" diff --git a/tests/sim/test_cfg.py b/tests/sim/test_cfg.py index 3bc44ec53..9b379fa28 100644 --- a/tests/sim/test_cfg.py +++ b/tests/sim/test_cfg.py @@ -21,7 +21,13 @@ from dexsim.types import DenoiserType, Renderer, ToneMappingType -from embodichain.lab.sim.cfg import ArticulationCfg, PhysicsCfg, RenderCfg, RobotCfg +from embodichain.lab.sim.cfg import ( + ArticulationCfg, + DLSSCfg, + PhysicsCfg, + RenderCfg, + RobotCfg, +) def test_articulation_cfg_defaults_to_no_joint_drive() -> None: @@ -145,3 +151,167 @@ def test_render_cfg_rejects_invalid_image_processing_settings( """Invalid image-processing values fail at configuration construction.""" with pytest.raises(ValueError): RenderCfg(**{field_name: invalid_value}) + + +def test_dlss_defaults_enable_offscreen_and_preserve_quality_resolution() -> None: + """Offscreen DLSS defaults on while quality and resolution retain native defaults.""" + native = dexsim.DLSSConfig() + converted = DLSSCfg().to_dexsim_cfg(1920, 1080) + + for name in DLSSCfg().to_dict(): + if name not in ("upsample_ratio", "offscreen_dlss_enabled"): + assert getattr(converted, name) == getattr(native, name), name + assert converted.render_width == converted.render_height == 0 + assert converted.dlss_quality == 2 + assert converted.offscreen_dlss_enabled is True + + +@pytest.mark.parametrize("rr_enabled", [False, True]) +@pytest.mark.parametrize("sr_enabled", [False, True]) +def test_dlss_features_are_independent(rr_enabled: bool, sr_enabled: bool) -> None: + """RR and SR selections survive conversion independently.""" + converted = DLSSCfg( + rayreconstruction_enabled=rr_enabled, + upscale_enabled=sr_enabled, + ).to_dexsim_cfg(1920, 1080) + + assert converted.rayreconstruction_enabled is rr_enabled + assert converted.upscale_enabled is sr_enabled + + +@pytest.mark.parametrize("quality", range(-1, 6)) +def test_dlss_quality_presets_keep_automatic_render_dimensions(quality: int) -> None: + """EmbodiChain forwards quality instead of hard-coding an internal scale.""" + converted = DLSSCfg(dlss_quality=quality).to_dexsim_cfg(1920, 1080) + + assert converted.dlss_quality == quality + assert converted.render_width == converted.render_height == 0 + + +@pytest.mark.parametrize( + ("render_size", "expected_size"), + [ + ((0, 0), (960, 540)), + ((1280, 0), (1280, 540)), + ((0, 720), (960, 720)), + ((1280, 720), (1280, 720)), + ], +) +def test_dlss_ratio_uses_window_size_and_preserves_explicit_dimensions( + render_size: tuple[int, int], expected_size: tuple[int, int] +) -> None: + """The ratio fills only zero dimensions using the actual window target.""" + converted = DLSSCfg( + upsample_ratio=2.0, + render_width=render_size[0], + render_height=render_size[1], + target_width=3840, + target_height=2160, + exposure_compensation=1.5, + ).to_dexsim_cfg(1920, 1080) + + assert (converted.render_width, converted.render_height) == expected_size + assert (converted.target_width, converted.target_height) == (3840, 2160) + assert converted.exposure_compensation == pytest.approx(1.5) + + +def test_dlss_ratio_clamps_small_internal_dimensions() -> None: + """An explicit ratio cannot produce an invalid zero-pixel render target.""" + converted = DLSSCfg(upsample_ratio=8.0).to_dexsim_cfg(4, 2) + + assert converted.render_width == converted.render_height == 1 + + +def test_render_cfg_instances_do_not_share_dlss_settings() -> None: + """Changing one rendering configuration does not alter another.""" + first, second = RenderCfg(), RenderCfg() + first.dlss.dlss_enabled = False + + assert second.dlss.dlss_enabled is True + + +@pytest.mark.parametrize( + ("field_name", "invalid_value"), + [ + ("dlss_quality", -2), + ("dlss_quality", 6), + ("dlss_quality", 2.5), + ("render_width", -1), + ("render_height", -1), + ("target_width", -1), + ("target_height", -1), + ("render_width", 1.5), + ("upsample_ratio", 0.5), + ("upsample_ratio", float("inf")), + ("upsample_ratio", float("nan")), + ("upsample_ratio", "2.0"), + ("upsample_ratio", True), + ("upsample_ratio", []), + ("upsample_ratio", {}), + ("exposure_compensation", 0.0), + ("exposure_compensation", -1.0), + ("exposure_compensation", float("inf")), + ("exposure_compensation", float("nan")), + ("exposure_compensation", "1.0"), + ("exposure_compensation", True), + ("exposure_compensation", None), + ("exposure_compensation", []), + ("exposure_compensation", {}), + ], +) +def test_dlss_rejects_invalid_settings(field_name: str, invalid_value: object) -> None: + """Invalid DLSS values fail before native configuration or rendering.""" + with pytest.raises(ValueError, match=field_name): + DLSSCfg(**{field_name: invalid_value}) + + +@pytest.mark.parametrize( + "field_name", + [ + "dlss_enabled", + "offscreen_dlss_enabled", + "rayreconstruction_enabled", + "upscale_enabled", + ], +) +@pytest.mark.parametrize("invalid_value", ["false", 0, 1, None]) +def test_dlss_rejects_non_boolean_switches( + field_name: str, invalid_value: object +) -> None: + """Switches require booleans instead of silently coercing task config values.""" + with pytest.raises(ValueError, match=field_name): + DLSSCfg(**{field_name: invalid_value}) + + +@pytest.mark.parametrize("value", [2, 2.0]) +def test_dlss_accepts_integer_and_float_numeric_settings(value: int | float) -> None: + """Both JSON/YAML numeric representations remain valid for real-valued settings.""" + converted = DLSSCfg( + upsample_ratio=value, exposure_compensation=value + ).to_dexsim_cfg(1920, 1080) + + assert (converted.render_width, converted.render_height) == (960, 540) + assert converted.exposure_compensation == pytest.approx(2.0) + + +@pytest.mark.parametrize( + ("field_name", "invalid_value"), + [ + ("upsample_ratio", 0.0), + ("upsample_ratio", "2.0"), + ("exposure_compensation", "1.0"), + ("dlss_enabled", "false"), + ("offscreen_dlss_enabled", 0), + ("rayreconstruction_enabled", 1), + ("upscale_enabled", None), + ], +) +def test_dlss_conversion_revalidates_mutated_settings( + field_name: str, invalid_value: object +) -> None: + """Mutable config edits are checked before entering the native binding.""" + config = DLSSCfg() + setattr(config, field_name, invalid_value) + + with pytest.raises(ValueError, match=field_name): + config.to_dexsim_cfg(1920, 1080) diff --git a/tests/sim/test_sim_manager.py b/tests/sim/test_sim_manager.py index fd20ebfcd..fd4af648f 100644 --- a/tests/sim/test_sim_manager.py +++ b/tests/sim/test_sim_manager.py @@ -19,6 +19,8 @@ import gc import queue +from pathlib import Path + from types import SimpleNamespace from unittest.mock import MagicMock @@ -30,6 +32,7 @@ import embodichain.lab.sim.sim_manager as sim_manager_module from embodichain.lab.sim.cfg import MarkerCfg from embodichain.lab.sim.profiler import Profiler +from embodichain.lab.sim.cfg import DLSSCfg, RenderCfg from embodichain.lab.sim.sim_manager import ( SimulationManager, SimulationManagerCfg, @@ -53,6 +56,52 @@ pytestmark = pytest.mark.no_sim +@pytest.mark.parametrize("renderer", ["hybrid", "fast-rt", "rt", "auto"]) +@pytest.mark.parametrize("headless", [False, True]) +@pytest.mark.parametrize( + "dlss_enabled,offscreen_enabled", [(False, True), (True, False), (True, True)] +) +def test_convert_sim_config_applies_dlss_for_all_renderers_and_camera_modes( + renderer: str, + headless: bool, + dlss_enabled: bool, + offscreen_enabled: bool, + monkeypatch: pytest.MonkeyPatch, + tmp_path: Path, +) -> None: + """Headless and auto-selected renderers retain explicit DLSS configuration.""" + monkeypatch.setattr( + "embodichain.lab.sim.utility.render_utils.select_default_renderer", + lambda _gpu_id: "hybrid", + ) + config = SimulationManagerCfg( + headless=headless, + sim_device="cpu", + width=640, + height=480, + render_cfg=RenderCfg( + renderer=renderer, + dlss=DLSSCfg( + dlss_enabled=dlss_enabled, + offscreen_dlss_enabled=offscreen_enabled, + dlss_quality=3, + target_width=1920, + target_height=1080, + ), + ), + ) + manager = SimpleNamespace(_material_cache_dir=tmp_path) + + world = SimulationManager._convert_sim_config(manager, config) + + assert world.dlss_config.dlss_enabled is dlss_enabled + assert world.dlss_config.offscreen_dlss_enabled is offscreen_enabled + assert world.dlss_config.dlss_quality == 3 + assert world.dlss_config.render_width == world.dlss_config.render_height == 0 + assert (world.win_config.width, world.win_config.height) == (640, 480) + assert world.open_windows is not headless + + class FakeCamera: """Simple camera stub for recorder unit tests."""