Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -606,6 +606,7 @@ topics:
- SceneSnapshotSupplier
- SceneProvider
- RigidObjectSceneProvider
- create_simulation_atomic_action_engine
- SceneRegistry
- RegistrySceneProvider
- SceneEntityRef
Expand Down Expand Up @@ -885,7 +886,6 @@ topics:
- open drawer
- hand over
- pour water
- rearrangement
- 配置生成环境
- 运行时环境注册
paths:
Expand All @@ -902,7 +902,6 @@ topics:
- embodichain_tasks/configs/tasks/manipulation/open_drawer/
- embodichain_tasks/configs/tasks/manipulation/hand_over/
- embodichain_tasks/configs/tasks/manipulation/tableware/pour_water/
- embodichain_tasks/configs/tasks/manipulation/tableware/rearrangement/
related_topics:
- atomic-actions
- env-framework
Expand Down
17 changes: 17 additions & 0 deletions agent_context/topics/atomic-actions/atomic-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,23 @@ instance of every type in `BUILTIN_ACTION_TYPES`; use `load_builtins=False` only
for isolated tests or a fully custom action set. A bound action cannot be
reused by another engine.

An engine may additionally borrow a default `SceneProvider`. In that case,
`engine.initial_context()` captures a `SceneSnapshot` from the provider using
the robot observation timestamp and generated environment IDs. An explicitly
supplied `scene=` snapshot takes precedence, and engines without either source
retain the empty-scene behavior. The provider is only an initial-context
convenience for direct-core planning; execution observations and scene revision
advancement remain owned by the runtime's `ObservationProvider`.

Direct simulation callers that only need selected rigid-object poses should use
`create_simulation_atomic_action_engine(..., scene_entities=(...))`. The factory
derives canonical direct-core IDs from the supplied objects' `uid` values and
installs the default provider; it never scans `SimulationManager`. Actions then
select the entries they consume through their goal and semantic entity IDs.
Articulation/link observations, aliases, collision roles, dynamic execution,
and external perception remain explicit `SceneProvider` or `SceneRegistry`
integration paths.

## Engine entry points

Choose the public engine entry point by lifecycle, not by skill type:
Expand Down
11 changes: 3 additions & 8 deletions agent_context/topics/expert-programs/expert-programs.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,19 +238,14 @@ evidence fails closed. Resource disjointness alone is never sufficient.
`simulation.pour` calls after built-in Pick. The configured lowerers preserve
verified held-object state and keep task motion values in the selected robot
policy preset.
- `Rearrangement-v3` uses registered `simulation.push_object` calls over
predeclared utensil-to-slot routes. Each utensil is pushed, settled, and
corrected from the latest measured pose; the action short-circuits inside its
configured completion tolerance, while segment validators remain the final
physical acceptance boundary.

Do not infer physical qualification from the two trajectory-only examples or
from unit/fake-port tests. Physical acceptance belongs in dedicated validation
integrations with measured evidence, controlled multi-seed/randomization runs,
and task validators. `EmbodiedEnv` is the common execution environment for all
five references. Their Gym configs own the complete supported composition
four references. Their Gym configs own the complete supported composition
roots, including the allowlisted services needed by Open Drawer, Hand Over,
Pour Water, and Rearrangement; none has a task environment module or subclass.
and Pour Water; none has a task environment module or subclass.

## Recommended change sites

Expand All @@ -265,7 +260,7 @@ Pour Water, and Rearrangement; none has a task environment module or subclass.
| Lazy Gym action/segment lifecycle | `bridge.py` and `embodichain/lab/gym/envs/demo.py` |
| Environment adapter binding, episode program selection, success/reset | `embodichain/lab/gym/envs/embodied_env.py` and `embodichain/lab/gym/utils/registration.py` |
| Config-created simple runtime and dynamic ID binding | `embodichain/lab/gym/envs/expert_program/configured_runtime.py` and `embodichain/lab/gym/utils/gym_utils.py` |
| Reference scene/profile/runtime values | `embodichain_tasks/configs/tasks/manipulation/{repeated_pick_place,open_drawer,hand_over}/env.json` and `embodichain_tasks/configs/tasks/manipulation/tableware/{pour_water,rearrangement}/env.json` |
| Reference scene/profile/runtime values | `embodichain_tasks/configs/tasks/manipulation/{repeated_pick_place,open_drawer,hand_over}/env.json` and `embodichain_tasks/configs/tasks/manipulation/tableware/pour_water/env.json` |
| Configured live-service implementations | `embodichain/lab/gym/envs/expert_program/_configured_runtime_services.py` |

Prefer changing the narrow owner. Do not add task-local motion generators,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ embodichain.lab.sim.atomic_actions
CommandDispatch
CommandOperation
ExecutionClock
create_simulation_atomic_action_engine
SimulationExecutionAdapter
ExecutionTick
EffectVerificationRequest
Expand Down Expand Up @@ -303,6 +304,8 @@ Engine and execution
.. autoclass:: MonotonicExecutionClock
:members:

.. autofunction:: create_simulation_atomic_action_engine

.. autoclass:: SimulationExecutionAdapter
:members:

Expand Down
1 change: 1 addition & 0 deletions docs/source/api_reference/public_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -766,6 +766,7 @@ embodichain.lab.sim.atomic_actions.sim_adapter

.. autosummary::

create_simulation_atomic_action_engine
RigidObjectSceneProvider
RigidObjectSceneProviderCfg
SceneSnapshotSupplier
Expand Down
2 changes: 2 additions & 0 deletions embodichain/lab/sim/atomic_actions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,7 @@
)
from .scene import SceneProvider
from .sim_adapter import (
create_simulation_atomic_action_engine,
RigidObjectSceneProvider,
RigidObjectSceneProviderCfg,
SceneSnapshotSupplier,
Expand Down Expand Up @@ -270,6 +271,7 @@
"CommandSink",
"ControlCommand",
"ControlPartCommandProfile",
"create_simulation_atomic_action_engine",
"CoordinatedHeldObjectState",
"CoordinatedPickGoal",
"CoordinatedPickment",
Expand Down
34 changes: 30 additions & 4 deletions embodichain/lab/sim/atomic_actions/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,9 @@
from .plans import ActionPlan, CompiledTrajectory, TimedTrajectory
from .policies import MotionPolicy, RecoveryPolicy
from .runtime import ActionPlanningServices
from .scene import SceneProvider
from .state import PlanningContext, RobotObservation, SceneSnapshot, TaskState
from .tracking import TrackingRuntime
from .tracking import TrackingPolicy, TrackingRuntime

if TYPE_CHECKING:
from embodichain.lab.sim.objects import Robot
Expand All @@ -52,6 +53,7 @@ def __init__(
*,
load_builtins: bool = True,
tracking_runtime: TrackingRuntime | None = None,
scene_provider: SceneProvider | None = None,
) -> None:
"""Initialize one engine and bind its built-in action implementations.

Expand All @@ -66,13 +68,19 @@ def __init__(
tracking_runtime: Optional exact-version feedback, projector, and
metric registries. Built-in joint tracking is installed when
omitted.
scene_provider: Optional default scene-observation source used by
:meth:`initial_context` when the caller does not supply an
explicit scene snapshot. The provider is borrowed by reference.
"""
if scene_provider is not None and not isinstance(scene_provider, SceneProvider):
raise TypeError("scene_provider must implement SceneProvider.")
self._planning_services = ActionPlanningServices(
motion_generator,
control_profiles=control_profiles,
tracking_runtime=tracking_runtime,
grasp_pose_generators=grasp_pose_generators,
)
self._scene_provider = scene_provider
self._actions: dict[str, AtomicAction] = {}
self._skill_catalog_revision = 0
if load_builtins:
Expand Down Expand Up @@ -194,6 +202,7 @@ def make_invocation(
*,
control_parts: Mapping[str, Mapping[str, str]] | None = None,
motion_policy: MotionPolicy | None = None,
tracking_policy: TrackingPolicy | None = None,
recovery_policy: RecoveryPolicy | None = None,
skill_options: OptionsT | None = None,
control_overrides: ActionControlOverrides | None = None,
Expand All @@ -211,6 +220,7 @@ def make_invocation(
goal: Action-specific typed goal.
control_parts: Direct ``slot -> endpoint -> control_part`` mapping.
motion_policy: Optional invocation motion policy.
tracking_policy: Optional typed tracking and terminal-acceptance policy.
recovery_policy: Optional invocation recovery policy.
skill_options: Optional action-specific invocation options.
control_overrides: Optional endpoint-scoped command overrides.
Expand All @@ -235,6 +245,11 @@ def make_invocation(
goal=goal,
binding=binding,
motion_policy=MotionPolicy() if motion_policy is None else motion_policy,
tracking_policy=(
TrackingPolicy.joint_position()
if tracking_policy is None
else tracking_policy
),
recovery_policy=(
RecoveryPolicy() if recovery_policy is None else recovery_policy
),
Expand Down Expand Up @@ -394,7 +409,9 @@ def initial_context(

Args:
task: Optional symbolic task state; an empty state is used otherwise.
scene: Optional scene snapshot; an empty snapshot is used otherwise.
scene: Optional explicit scene snapshot. It overrides the engine's
configured scene provider; an empty snapshot is used when both
are absent.
timestamp: Timestamp assigned to the captured robot observation.
control_dt: Explicit command period for action-owned interpolation.

Expand All @@ -410,15 +427,24 @@ def initial_context(
qvel_value = candidate.to(self.device)
qvel = torch.zeros_like(qpos) if qvel_value is None else qvel_value
batch_size = int(qpos.shape[0])
env_ids = torch.arange(batch_size, dtype=torch.long, device=self.device)
if task is None:
task = TaskState.empty(batch_size=batch_size, device=self.device)
if scene is None:
scene = SceneSnapshot.empty()
if self._scene_provider is None:
scene = SceneSnapshot.empty()
else:
scene = self._scene_provider.snapshot(
timestamp=timestamp,
env_ids=env_ids.clone(),
)
if not isinstance(scene, SceneSnapshot):
raise TypeError("scene_provider must return a SceneSnapshot.")
return PlanningContext(
robot=RobotObservation(timestamp=timestamp, qpos=qpos, qvel=qvel),
task=task,
scene=scene,
env_ids=torch.arange(batch_size, dtype=torch.long, device=self.device),
env_ids=env_ids,
control_dt=control_dt,
)

Expand Down
63 changes: 63 additions & 0 deletions embodichain/lab/sim/atomic_actions/sim_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
from embodichain.utils import configclass

from .bindings import JointPositionTarget, RuntimeEndpointTarget
from .control import ControlPartCommandProfile
from .engine import AtomicActionEngine
from .runner import (
CommandAcknowledgement,
CommandAckStatus,
Expand All @@ -40,10 +42,13 @@
SceneSnapshot,
TaskState,
)
from .tracking import TrackingRuntime

if TYPE_CHECKING:
from embodichain.lab.sim.objects import RigidObject, Robot
from embodichain.lab.sim.planners import MotionGenerator
from embodichain.lab.sim.sim_manager import SimulationManager
from embodichain.toolkits.graspkit import GraspPoseGenerator


@configclass
Expand Down Expand Up @@ -239,6 +244,63 @@ def _pose_change_mask(
)


def create_simulation_atomic_action_engine(
motion_generator: MotionGenerator,
scene_entities: Sequence[RigidObject],
control_profiles: Mapping[str, ControlPartCommandProfile] | None = None,
grasp_pose_generators: Mapping[str, GraspPoseGenerator] | None = None,
*,
load_builtins: bool = True,
tracking_runtime: TrackingRuntime | None = None,
) -> AtomicActionEngine:
"""Create an engine whose initial context observes selected rigid objects.

This is the direct-simulation convenience path for offline planning. Entity
IDs are derived from each rigid object's stable ``uid``; only explicitly
supplied objects are observed. Advanced integrations that need aliases,
articulation/link state, collision roles, or an external perception source
should construct :class:`AtomicActionEngine` with their own
:class:`SceneProvider` instead.

Args:
motion_generator: Motion-generation backend owned by the engine.
scene_entities: Non-empty sequence of simulation rigid objects to expose
in automatically captured initial scene snapshots.
control_profiles: Semantic commands keyed by robot control-part name.
grasp_pose_generators: Grasp-pose services keyed by grasp endpoint target.
load_builtins: Whether to install all built-in atomic actions.
tracking_runtime: Optional typed tracking runtime shared by action plans.

Returns:
Engine configured with a rigid-object scene provider.

Raises:
TypeError: If ``scene_entities`` is not a sequence.
ValueError: If an entity lacks a stable UID or UIDs are duplicated.
"""
if isinstance(scene_entities, (str, bytes)) or not isinstance(
scene_entities, Sequence
):
raise TypeError("scene_entities must be a sequence of rigid objects.")
entities_by_id: dict[str, RigidObject] = {}
for entity in scene_entities:
entity_id = getattr(entity, "uid", None)
if not isinstance(entity_id, str) or not entity_id.strip():
raise ValueError("Every scene entity must have a non-empty string uid.")
if entity_id in entities_by_id:
raise ValueError(f"Duplicate scene entity uid {entity_id!r}.")
entities_by_id[entity_id] = entity

return AtomicActionEngine(
motion_generator,
control_profiles=control_profiles,
grasp_pose_generators=grasp_pose_generators,
load_builtins=load_builtins,
tracking_runtime=tracking_runtime,
scene_provider=RigidObjectSceneProvider(entities_by_id),
)


SceneSnapshotSupplier = Callable[[float], SceneSnapshot]
"""Callback that returns the latest scene snapshot for a simulation timestamp."""

Expand Down Expand Up @@ -617,6 +679,7 @@ def _validate_timeout(timeout: float) -> None:


__all__ = [
"create_simulation_atomic_action_engine",
"RigidObjectSceneProvider",
"RigidObjectSceneProviderCfg",
"SceneSnapshotSupplier",
Expand Down
20 changes: 4 additions & 16 deletions scripts/tutorials/atomic_action/assemble.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,13 @@
from embodichain.lab.sim.atomic_actions import (
AssembleAffordance,
AssembleGoal,
AtomicActionEngine,
ControlPartCommandProfile,
EntityState,
create_simulation_atomic_action_engine,
GraspGoal,
PickUpOptions,
PlaceOptions,
MotionPolicy,
SceneEntityPose,
SceneSnapshot,
)
from embodichain.lab.sim.cfg import RigidBodyAttributesCfg, RigidObjectCfg
from embodichain.data import get_data_path
Expand Down Expand Up @@ -295,8 +293,9 @@ def run_assemble_demo(
lift_height=PLACE_LIFT_HEIGHT,
hand_interp_steps=PLACE_HAND_INTERP_STEPS,
)
engine = AtomicActionEngine(
engine = create_simulation_atomic_action_engine(
motion_generator=motion_gen,
scene_entities=(can, cube),
control_profiles={
"left_hand": ControlPartCommandProfile.joint_positions(
open=left_open,
Expand All @@ -320,14 +319,6 @@ def run_assemble_demo(
assemble_affordance = AssembleAffordance(
assemble_to_base_pose=assemble_to_base,
)
scene = SceneSnapshot(
timestamp=0.0,
version=0,
entities={
can.uid: EntityState(can.get_local_pose(to_matrix=True)),
cube.uid: EntityState(cube.get_local_pose(to_matrix=True)),
},
)
endpoint_mapping = {"primary": {"motion": "left_arm", "grasp": "left_hand"}}
compiled = engine.compile(
(
Expand Down Expand Up @@ -355,10 +346,7 @@ def run_assemble_demo(
skill_options=place_options,
),
),
engine.initial_context(
scene=scene,
control_dt=sim.sim_config.physics_dt,
),
engine.initial_context(control_dt=sim.sim_config.physics_dt),
)
success = compiled.plan_success
traj = compiled.trajectory.positions
Expand Down
Loading
Loading