Add Newton physics backend support - #276
Draft
yuecideng wants to merge 142 commits into
Draft
Conversation
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: Jietao Chen <chenjietao@dexforce.com> Co-authored-by: Yueci Deng <dengyueci@qq.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: yuanhaonan <yuanhaonan@dexforce.top>
…entation (#247) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: chenjian <chenjian@dexforce.com> Co-authored-by: yuecideng <dengyueci@qq.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ration (#239) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Chen Jian <mtfl1996@outlook.com> Co-authored-by: chenjian <chenjian@dexforce.com> Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: chenjian <chenjian@dexforce.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Yueci Deng <dengyueci@qq.com>
Co-authored-by: WaferLi <63717327+WaferLi@users.noreply.github.com> Co-authored-by: liwenfeng <liwenfeng@dexforce.top> Co-authored-by: chenjian <chenjian@dexforce.com> Co-authored-by: daojun <lookangela@qq.com> Co-authored-by: Chen Jian <mtfl1996@outlook.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
9 tasks
…s-backend # Conflicts: # docs/source/api_reference/embodichain/embodichain.lab.sim.cfg.rst # docs/source/features/interaction/preview_asset.md # docs/source/guides/configuration.md # docs/source/overview/sim/sim_manager.md # embodichain/lab/gym/envs/base_env.py # embodichain/lab/gym/utils/gym_utils.py # embodichain/lab/scripts/preview_asset.py # embodichain/lab/sim/cfg.py # embodichain/lab/sim/objects/articulation.py # embodichain/lab/sim/objects/cloth_object.py # embodichain/lab/sim/objects/rigid_object.py # embodichain/lab/sim/sim_manager.py # embodichain/learning/rl/train.py # embodichain_tasks/embodichain_tasks/special/franka_reach_apg.py # examples/sim/demo/pick_up_cloth.py # examples/sim/gizmo/gizmo_camera.py # examples/sim/gizmo/gizmo_object.py # examples/sim/gizmo/gizmo_robot.py # examples/sim/gizmo/gizmo_scene.py # examples/sim/gizmo/gizmo_w1.py # examples/sim/planners/neural_planner.py # examples/sim/sensors/batch_camera.py # examples/sim/sensors/create_contact_sensor.py # examples/sim/solvers/differential_solver.py # examples/sim/solvers/opw_solver.py # examples/sim/solvers/pink_solver.py # examples/sim/solvers/pinocchio_solver.py # examples/sim/solvers/pytorch_solver.py # examples/sim/solvers/srs_solver.py # examples/sim/utility/workspace_analyzer/analyze_cartesian_workspace.py # examples/sim/utility/workspace_analyzer/analyze_joint_workspace.py # examples/sim/utility/workspace_analyzer/analyze_plane_workspace.py # scripts/tutorials/gym/modular_env.py # scripts/tutorials/gym/random_reach.py # scripts/tutorials/sim/create_cloth.py # scripts/tutorials/sim/create_rigid_object_group.py # scripts/tutorials/sim/create_scene.py # scripts/tutorials/sim/create_softbody.py # scripts/tutorials/sim/gizmo_robot.py # scripts/tutorials/sim/motion_generator.py # scripts/tutorials/sim/srs_solver.py # tests/gym/utils/test_gym_utils.py # tests/sim/objects/test_rigid_object.py # tests/sim/workspace/test_sim_utils.py
# Conflicts: # scripts/benchmark/__main__.py # scripts/benchmark/planners/neural_planner/run_benchmark.py # tests/sim/planners/test_toppra_batched.py
Co-authored-by: xiemenghong <xiemenghong@dexforce.top> Co-authored-by: yuecideng <dengyueci@qq.com>
…s-backend # Conflicts: # agent_context/MAP.yaml # embodichain/gen_sim/scene_engine/pipeline/utils/assets_gravity_settler.py # embodichain/lab/scripts/preview_asset.py # embodichain/lab/sim/objects/__init__.py # embodichain/lab/sim/objects/rigid_object.py # scripts/tutorials/atomic_action/hand_over.py # scripts/tutorials/atomic_action/tutorial_utils.py # tests/sim/objects/test_articulation.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR makes Newton a first-class physics backend in EmbodiChain while preserving the existing default DexSim path. It also standardizes scene construction around DexSim Spawn and adds an end-to-end differentiable execution path for analytic policy gradient workloads.
The motivation is to keep backend-specific lifecycle, state access, spawning, and differentiation behind explicit contracts instead of scattering Newton conditionals through simulation objects and environments.
Issue reference: N/A.
What changed
Physics backend and lifecycle
DefaultPhysicsCfgandNewtonPhysicsCfgconfiguration, including Newton solver, collision-pipeline, device, substep, CUDA graph, and gradient settings.PhysicsBackendabstraction with concrete default and Newton implementations selected fromphysics_cfg.SimulationManager.prepare(), backend capabilities, scene context, and differentiable-runtime delegators.Spawn and object model
embodichain.lab.sim.spawnto translate EmbodiChain configs and USD sources into DexSim Spawn descriptors.preserve/overlaybehavior and typed backend-specific rigid, collision, material, articulation-root, and joint-drive properties.Differentiable execution
embodichain.lab.sim.diff.DifferentiableEmbodiedEnvwith explicit dynamics and kinematics execution modes, manager-owned Newton trajectories, deferred terminal resets, and safe forward/backward cleanup.FrankaReachApg-v0reference task to exercise differentiable stepping and analytic policy gradient behavior end to end.Compatibility, examples, and documentation
Backend capability summary
Known limitations
NewtonPhysicsCfg(requires_grad=True)with thesemi_implicitsolver.SimulationManager.spawn_resultand Spawn batch APIs; the removed independently ownedNewtonManager/ compatibilityPhysicsSceneownership path is not exposed.Dependencies
No new dependency is added by this branch. The implementation requires the Spawn/Newton APIs provided by the existing
dexsim_engine==0.4.3dependency.Suggested review order
embodichain/lab/sim/cfg.py,physics/, andsim_manager.pyembodichain/lab/sim/spawn/andobjects/backends/embodichain/lab/sim/diff/,DifferentiableEmbodiedEnv, and the Franka APG taskType of change
Screenshots
Not applicable.
Validation
Focused post-
mainmerge validation:Checklist