Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 30 additions & 14 deletions agent_context/topics/motion-planning/collision-worlds.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,28 @@ Read this when the request needs these details. [Topic overview](motion-planning
`CuroboWorldCfg.rigid_objects` accepts either a mapping or a sequence. Use
`Mapping[registry_id, RigidObject]` for a registry-backed integration. The
mapping key is the authoritative logical/source obstacle ID used by the
content-cache key, `collision_world_entity_ids`, and registry validation. For
`cuboid` and `mesh`, it is also the physical YAML obstacle name and dynamic
update key. For `sphere`, one static source expands to physical YAML names such
as `registry_id_0`; dynamic sphere configuration is rejected, while cache and
full-world identity remain keyed by `registry_id`. A registry mapping whose
source lacks mesh geometry required by the selected representation fails fast
instead of silently dropping the source. The sequence form is an advanced
direct-core path that derives names from each object's `uid` or an
content-cache key, `collision_world_entity_ids`, generated obstacle-name prefix,
dynamic update key, and registry validation. Scene generation reads physical
collision descriptors through `RigidObject.get_collision_shapes()` and emits a
mixed tensor-backed cuRobo scene. A compound source expands to physical names
such as `registry_id__shape_0`; cache and full-world identity remain keyed by
the unexpanded `registry_id`, and dynamic updates fan out through the physical
shapes' local poses. A registry mapping whose source lacks physical collision
shapes fails fast instead of silently dropping the source. The sequence form is
an advanced direct-core path that derives names from each object's `uid` or an
`obstacle_<index>` fallback.

Analytic box, plane, sphere, and capsule shapes retain their native cuRobo
representations. Mesh-backed `MESH`, `CONVEX`, and `SDF` shapes never become
direct cuRobo `Mesh` entries: scene generation computes one Open3D convex hull,
then samples its signed distance into a dense ESDF `VoxelGrid`. The global or
per-object `"voxel"` policy can apply the same conversion to an analytic shape;
the direct `"mesh"` policy is unsupported. `max_voxel_count` guards every ESDF
allocation and fails fast with an actionable error. `mesh_triangle_threshold`
remains accepted only for configuration compatibility and no longer changes
representation selection. The world cache format is versioned so caches that
may contain direct mesh entries are not reused.

`CuroboWorldCfg.multi_env` controls collision-world batching, not whether robot
states or goals are batched:

Expand All @@ -28,10 +40,13 @@ states or goals are batched:
different poses relative to their local robot bases, such as per-env pose
randomization.

The multi-env scene is cloned from the YAML generated using env 0; enabling the
flag does not load distinct initial simulator poses for other rows. Per-env
differences require `"cuboid"` or `"mesh"` representation, registration in
`dynamic_obstacle_names`, and current `(B, 4, 4)` world poses in
The multi-env scene is cloned from the cached tensor-backed scene dictionary
generated using env 0; enabling the flag does not load distinct initial
simulator poses for other rows. Each clone remains a dictionary until cuRobo
0.8 constructs its own `SceneCfg`, because its multi-env list parser expects a
dictionary per environment rather than pre-built `SceneCfg` instances. Per-env
differences require registration in `dynamic_obstacle_names` and current
`(B, 4, 4)` world poses in
`CuroboPlanOptions.dynamic_obstacle_poses`. Independent worlds replicate scene
data and collision caches, so retain the shared default for identical rebased
layouts.
Expand All @@ -53,8 +68,9 @@ contract. It requires unique canonical IDs and requires the dynamic subset to
belong to the complete world. `MotionGenerator.collision_world_info` forwards
that contract and retains derived ID/mode properties for callers. For cuRobo,
the complete set is every mapping key (or inferred sequence name), while the
dynamic set is exactly `CuroboWorldCfg.dynamic_obstacle_names`. Sphere-expanded
physical YAML names are not part of either logical ID declaration.
dynamic set is exactly `CuroboWorldCfg.dynamic_obstacle_names`.
Compound-expanded physical shape names are not part of either logical ID
declaration.
`CuroboWorldCfg` rejects duplicate obstacle names and requires every
`dynamic_obstacle_name` to match an object registered in `rigid_objects`, so a
planner-local mismatch fails before backend construction.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ bodies.
Light
LightCfg
RigidObject
CollisionShapeDesc
RigidBodyData
RigidObjectCfg
RigidObjectGroup
Expand Down Expand Up @@ -69,6 +70,9 @@ Rigid Object
:inherited-members:
:show-inheritance:

.. autoclass:: CollisionShapeDesc
:members:

.. autoclass:: RigidBodyData
:members:
:inherited-members:
Expand Down
6 changes: 5 additions & 1 deletion docs/source/api_reference/public_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -882,6 +882,7 @@ embodichain.lab.sim.objects.rigid_object

.. autosummary::

CollisionShapeDesc
RigidBodyData
RigidObject
RigidObjectCfg
Expand Down Expand Up @@ -952,7 +953,10 @@ embodichain.lab.sim.motion.planners.curobo.curobo_yaml
.. autosummary::

generate_curobo_robot_yaml
generate_curobo_world_yaml
generate_curobo_world_scene
visualize_curobo_collision_models
visualize_curobo_robot_collision_model
visualize_curobo_world_collision_model

embodichain.lab.sim.motion.planners.neural_planner
--------------------------------------------------
Expand Down
153 changes: 99 additions & 54 deletions docs/source/overview/sim/motion/planners/curobo_planner.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ planner_cfg = CuroboPlannerCfg(
planner_type="curobo",
world=CuroboWorldCfg(
rigid_objects=registry.collision_geometry_by_id(),
obstacle_representation="cuboid",
dynamic_obstacle_names=list(registry.dynamic_collision_entity_ids),
multi_env=collision_mode is SceneCollisionWorldMode.PER_ENV,
),
Expand All @@ -111,9 +110,15 @@ different planning GPU. A CPU value is rejected because cuRobo itself has no
CPU backend.

The robot configuration must be a cuRobo V2 robot profile with collision
spheres and self-collision data; the adapter generates this from the robot's
URDF automatically. A plain URDF alone is not sufficient for collision planning
without that sphere-fitting step.
spheres; the adapter generates this from the robot's URDF automatically. A plain
URDF alone is not sufficient for robot-to-world collision planning without that
sphere-fitting step.

:::{warning}
cuRobo self-collision checking is temporarily disabled in this backend.
Robot-to-world collision checking remains enabled, but planned trajectories
are not currently rejected when two robot links collide with each other.
:::

The adapter automatically rebases simulator-world Cartesian goals and dynamic
obstacle poses through the live simulator control-part base, so parallel arena
Expand Down Expand Up @@ -156,42 +161,77 @@ that use only one move type retain one planner backend; using both incurs a
second one-time warmup and its graph-resident memory, but still no subprocess or
second CUDA context.

The collision world is always auto-generated from live `RigidObject` meshes via
`CuroboWorldCfg.rigid_objects`. The canonical, registry-backed form is a mapping
from authoritative registry ID to live object; the adapter reads each object's
mesh (`get_vertices` / `get_triangles`) and world pose (`get_local_pose`) and
writes a cached cuRobo scene YAML on the first plan, using
`CuroboWorldCfg.obstacle_representation` (`"sphere"` by default for fast
collision queries; use `"cuboid"` for a local-frame AABB placed as an OBB via
the object pose, or `"mesh"` for the exact triangle mesh).
The collision world is auto-generated from live `RigidObject` **physical
collision shapes** via `RigidObject.get_collision_shapes()`. It does not use
`get_vertices()` / `get_triangles()`, which expose combined visual meshes and may
differ from the geometry used by DexSim physics. The canonical, registry-backed
form of `CuroboWorldCfg.rigid_objects` is a mapping from authoritative registry
ID to live object. The mapping key, rather than `RigidObject.uid`, becomes the
logical collision ID, cache identity, generated obstacle-name prefix, and
runtime-update key. The sequence form remains available for advanced callers
and derives IDs from each `uid` or an `obstacle_<index>` fallback.

`CuroboWorldCfg.representation="auto"` is the default. The policy preserves
boxes as cuboids, spheres and capsules as analytic primitives, and convex
collision shapes as meshes. Triangle meshes remain meshes up to
`mesh_triangle_threshold`; above that threshold they become voxel ESDF when the
estimated dense allocation fits `max_voxel_count`. Pose-dynamic meshes use twice
the threshold before voxelization, while static cached meshes favor ESDF sooner
for repeated collision queries. SDF descriptors fall back to
their canonical collision mesh because the current DexSim Python binding does
not expose reusable SDF grid data. Unsupported descriptors raise an explicit
error instead of silently falling back to visual geometry.

Forced voxel mode and per-object overrides remain available:

```python
world_cfg = CuroboWorldCfg(
rigid_objects={
"room_scan": room_scan,
"precision_fixture": precision_fixture,
},
representation="auto",
overrides={
"room_scan": "voxel",
"precision_fixture": "mesh",
},
)
```

`voxel_size` and `voxel_padding` configure generated ESDF layers. `plane_dims`
bounds an infinite DexSim plane as a thin cuRobo cuboid. Compound and ACD bodies
produce stable names such as `fixture__shape_0`; callers still use the owning
canonical ID in `dynamic_obstacle_names` and
`CuroboPlanOptions.dynamic_obstacle_poses`, and the adapter fans each update out
through the sub-shapes' local poses.

> **DexSim binding requirement:** Correct compound/USD offsets require
> `RigidBody.get_shape_geometry()` to copy each physical shape's local pose into
> `ShapeGeometry.local_pose`. Reusing a DexSim SDF as a voxel grid additionally
> requires grid metadata/data that the current Python API does not expose. Until
> those upstream bindings are available, verify compound offsets explicitly;
> SDF descriptors use their canonical collision mesh when one is exposed and
> otherwise raise an actionable error.

Generated poses are authored in the cuRobo base/world frame, so this is exact
when the robot base sits at the simulator world origin. The mapping key, rather
than `RigidObject.uid`, is the canonical logical/source ID used by cache
identity and collision-world validation. For `"cuboid"` and `"mesh"`, that ID
is also used unchanged as the physical YAML obstacle name and runtime update
key. For obstacles that move or live in an offset base frame, also declare their
canonical IDs in
when the robot base sits at the simulator world origin. For obstacles that move
or live in an offset base frame, declare their canonical IDs in
`CuroboWorldCfg.dynamic_obstacle_names` and update poses at plan time through
`CuroboPlanOptions.dynamic_obstacle_poses` (provision
`CuroboWorldCfg.collision_cache` before planning). Dynamic updates require the
`"cuboid"` or `"mesh"` representation because sphere fitting expands one object
into physical YAML obstacles named `<canonical_id>_0`, `<canonical_id>_1`, and
so on; dynamic sphere configuration is rejected. These derived names are
backend details. The cache and registry/planner full-world contract continue to
use the unexpanded canonical source ID.

Registry-backed mappings fail fast if a selected source has no mesh geometry
required by the chosen representation. This prevents a canonical collision ID
from being silently skipped during YAML generation. The advanced sequence form
retains its lower-level behavior independently of this registry contract.

`CuroboPlanner.collision_world_entity_ids` reports every configured logical
`CuroboPlanOptions.dynamic_obstacle_poses`.

Registry-backed mappings fail fast if a selected source has no physical
collision shapes. This prevents a canonical collision ID from being silently
omitted from the generated tensor-backed scene. The advanced sequence form may
skip such an object with a warning, independently of the registry contract.

`CuroboPlanner.collision_world_info.entity_ids` reports every configured logical
source ID: each mapping key on the registry path, or each inferred name on the
advanced sequence path. It deliberately does not expose sphere-expanded
physical YAML names. `dynamic_collision_entity_ids` reports exactly the
configured dynamic subset. Static entries therefore participate in
construction-time identity validation even though they do not receive per-plan
pose updates.
advanced sequence path. It deliberately does not expose compound-expanded
physical shape names. `collision_world_info.dynamic_entity_ids` reports exactly
the configured dynamic subset. `MotionGenerator` forwards these through its
`collision_world_entity_ids` and `dynamic_collision_entity_ids` properties.
Static entries therefore participate in construction-time identity validation
even though they do not receive per-plan pose updates.

`CuroboWorldCfg` validates this planner-local registration at construction:
obstacle IDs must be unique, and every dynamic obstacle ID must match an entry
Expand Down Expand Up @@ -230,14 +270,13 @@ differ, the adapter rejects the update and instructs the caller to enable

With `multi_env=True`, cuRobo allocates one collision world per batch row and
EmbodiChain sends row `i` of each dynamic obstacle pose to world `i`. The
auto-generated YAML still reads the static scene from env 0 and clones that
scene for every row; setting `multi_env=True` does not by itself discover each
environment's distinct initial object poses. Any object whose robot-relative
pose differs by environment must also:

1. Use `obstacle_representation="cuboid"` or `"mesh"`.
2. Be listed in `CuroboWorldCfg.dynamic_obstacle_names`.
3. Have its current `(B, 4, 4)` simulator-world poses passed through
auto-generated collision cache still reads the static scene from env 0 and
clones that scene for every row; setting `multi_env=True` does not by itself
discover each environment's distinct initial object poses. Any object whose
robot-relative pose differs by environment must also:

1. Be listed in `CuroboWorldCfg.dynamic_obstacle_names`.
2. Have its current `(B, 4, 4)` simulator-world poses passed through
`CuroboPlanOptions.dynamic_obstacle_poses` when planning.

For a registry-backed world, derive both the geometry mapping and dynamic ID
Expand All @@ -246,7 +285,6 @@ list from the same catalog:
```python
world_cfg = CuroboWorldCfg(
rigid_objects=registry.collision_geometry_by_id(),
obstacle_representation="cuboid",
dynamic_obstacle_names=list(registry.dynamic_collision_entity_ids),
multi_env=True,
)
Expand Down Expand Up @@ -293,11 +331,18 @@ robot's URDF and solver, so nothing robot-specific needs to be hardcoded:
The generated YAML is cached on disk (default `$XDG_CACHE_HOME/embodichain_curobo`
or `~/.cache/embodichain_curobo`) keyed by the URDF path, URDF content, control
part, tool frame, and fit parameters, so editing the URDF or changing the fit
settings regenerates automatically and subsequent inits reuse the cache. Tune the
fit with `CuroboPlannerCfg.auto_gen` (`fit_type="voxel"` by default for fast
first-generation; `"morphit"` for best quality; `force=True` to bypass the cache).
The default `sphere_density=0.1` keeps the per-link sphere count low (~80 for a
Panda) so planning stays fast; raise it for tighter collision coverage.
settings regenerates automatically and subsequent inits reuse the cache. Sphere
fitting always uses DexSim's `SphereFitType.MORPHIT`, with at most 2 convex hulls
per robot link and 16 per voxelized obstacle shape. The default
`sphere_density=0.1` keeps the
per-link sphere count low (~80 for a Panda) so planning stays fast; raise it for
tighter collision coverage, or set `force=True` to bypass the cache.

For an Open3D overlay of the robot collision spheres and sampled world collision
representations read back from those caches, call
`planner.visualize_robot_collision_models(control_part)`. Robot sphere centers are
transformed by the simulator's live link poses. The interactive cuRobo example
calls this once after planner initialization; close the Open3D window to continue.

## Generate a motion

Expand Down Expand Up @@ -374,9 +419,9 @@ python examples/sim/motion/planners/curobo_planner.py --headless --sim-device cp
~~~

The demo exports the DexSim `demo_block` into the cuRobo collision world via
`CuroboWorldCfg.rigid_objects` (the robot and world YAMLs are both
auto-generated), prints the result status and trajectory shape, then replays the
returned full-DoF trajectory. CUDA graph capture is enabled by default with the
`CuroboWorldCfg.rigid_objects` (the robot YAML and mixed collision-world cache are
auto-generated), prints the result status and trajectory shape, then replays
the returned full-DoF trajectory. CUDA graph capture is enabled by default with the
renderer-compatible `"thread_local"` mode; pass `--no-cuda-graph` to disable it.
Headless runs
automatically record this fixed offscreen camera view to an MP4. Set an explicit
Expand Down
1 change: 0 additions & 1 deletion docs/source/overview/task_program/scene_registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,6 @@ registry = SceneRegistry.from_simulation(

world = CuroboWorldCfg(
rigid_objects=registry.collision_geometry_by_id(),
obstacle_representation="cuboid",
dynamic_obstacle_names=list(registry.dynamic_collision_entity_ids),
multi_env=True,
)
Expand Down
23 changes: 23 additions & 0 deletions embodichain/lab/sim/motion/planners/base_planner.py
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,29 @@ def supports_move_type(self, move_type: MoveType) -> bool:
"""
return move_type in self.supported_move_types

def visualize_robot_collision_models(
self,
control_part: str,
env_id: int = 0,
) -> None:
"""Visualize the robot collision models used by this planner.

Planners that support collision avoidance should override this method
with their backend-specific visualization.

Args:
control_part: Robot control part whose collision models are visualized.
env_id: Simulator environment instance to visualize.

Raises:
NotImplementedError: If the planner does not support collision avoidance.
"""
logger.log_error(
f"{type(self).__name__} does not support collision avoidance or robot "
"collision model visualization.",
NotImplementedError,
)

def default_plan_options(self) -> PlanOptions:
"""Return backend-default planning options."""
return PlanOptions()
Expand Down
Loading
Loading