feat(scene-engine): add generated-scene authoring and editing

[skywhite1024]

Description

Stack

• Stack: #543
• Layer: 2/8
• Base: ljd/gen-sim-atomic-prerequisite
• Depends on: feat(atomic-actions): improve upright grasp selection #542
• Followed by: feat(task-engine): add TaskSpec and semantic task-graph contracts #533

Add the generated-scene authoring foundation for the task-first GenSim workflow. This layer owns scene generation and editing, layout and settling, service boundaries, and portable authoring import/export.

Architecture boundary

• Exposes GeneratedSceneGraph as an authoring-only graph, not live scene state.
• Emits stable content-derived scene IDs and a versioned graph schema.
• Preserves canonical object identity and ancestry across export/import.
• Publishes scene_authoring_evidence.json as an explicit audit_only sidecar with affordance evidence, GLB metadata and checksums, and physics provenance.
• Does not construct SceneRegistry, SceneManifest, providers, pose readers, simulator handles, or execution types. Canonical SceneManifest conversion remains owned by the feat(task-engine): add orchestration and end-to-end integration #538 integration layer.

This layer is independently reviewable and does not depend on Task Engine execution.

Refs #531

Type of change

• New feature (non-breaking change which adds functionality)

Validation

• pytest -q tests/gen_sim/scene_engine - 89 passed, 9 warnings
• pytest -q tests/docs/test_check_api_docs.py - 8 passed
• Black 26.3.1 - 782 Python files unchanged
• Public API docs - 1774/1774 exports documented
• git diff --check - passed
• Full Sphinx build not run locally because sphinx-build is unavailable in the validation environment

Checklist

• Code passes Black 26.3.1.
• Tests cover the affected behavior.
• Public API documentation is aligned.
• No new third-party dependency is required.

[greptile-apps]

Greptile Summary

The PR introduces generated-scene authoring and editing APIs, versioned authoring graphs, portable import/export, service boundaries, and layout processing.

• Adds image analysis, blueprint materialization, edit analysis, and edit materialization entry points.
• Adds generated-scene graph models, edit plans, asset preparation, import/export, layout, and settling utilities.
• Extends public API documentation and focused scene-engine tests.

Confidence Score: 3/5

The PR does not yet appear safe to merge because non-add edits remain coupled to unavailable asset services and edited exports can corrupt object coordinate frames.

Move-only and delete-only edits still initialize and health-check services they never use, while the importer retains the export-frame rotation and the edit exporter applies that rotation again.

Files Needing Attention: embodichain/gen_sim/scene_engine/pipeline/api.py, embodichain/gen_sim/scene_engine/pipeline/utils/scene_importer.py, embodichain/gen_sim/scene_engine/pipeline/utils/scene_exporter.py

Important Files Changed

Filename	Overview
embodichain/gen_sim/scene_engine/pipeline/api.py	Adds the public generation/edit orchestration boundaries, but move/delete edits still require unrelated asset services.
embodichain/gen_sim/scene_engine/pipeline/utils/scene_importer.py	Adds portable scene import, but does not invert the exporter's additional 180-degree frame rotation before re-export.
embodichain/gen_sim/scene_engine/pipeline/utils/scene_exporter.py	Extends versioned scene export and authoring metadata while retaining the rotation that is reapplied during edit cycles.
embodichain/gen_sim/scene_engine/pipeline/editing/scene_edit_understanding.py	Adds edit parsing and now constrains generated add categories to filename-safe identifiers.
embodichain/gen_sim/scene_engine/core/scene_graph.py	Introduces the versioned authoring-only GeneratedSceneGraph model and stable identifier validation.

_{Reviews (3): Last reviewed commit: "Merge branch 'ljd/gen-sim-atomic-prerequ..." | Re-trigger Greptile}

[greptile-apps]

Unused services gate all edits

When a move-only or delete-only edit runs while any asset-generation service is unconfigured or unavailable, materialize_edit still constructs and health-checks all three clients before reaching the no-add early return, causing a valid edit to fail before layout.

Prompt To Fix With AI

This is a comment left during a code review.
Path: embodichain/gen_sim/scene_engine/pipeline/api.py
Line: 255-266
Comment:
**Unused services gate all edits**
When a move-only or delete-only edit runs while any asset-generation service is unconfigured or unavailable, `materialize_edit` still constructs and health-checks all three clients before reaching the no-add early return, causing a valid edit to fail before layout.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Edit exports rotate coordinates again

When an existing export is edited, the importer retains coordinates in the additional 180-degree export frame and this call constructs SceneExporter with the same rotation enabled again, causing unchanged positions and XY metadata to flip between frames across edit cycles.

Prompt To Fix With AI

This is a comment left during a code review.
Path: embodichain/gen_sim/scene_engine/pipeline/api.py
Line: 304-308
Comment:
**Edit exports rotate coordinates again**
When an existing export is edited, the importer retains coordinates in the additional 180-degree export frame and this call constructs `SceneExporter` with the same rotation enabled again, causing unchanged positions and XY metadata to flip between frames across edit cycles.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Public module lacks export declaration

The new public client module defines ImageGenerationClient without __all__, leaving its intended API implicit and exposing helper symbols inconsistently with the repository's public-module convention.

Suggested change

	fromembodichain.gen_sim.scene_engine.configs.environmentimport (
	read_scene_engine_env_values,
	)
	classImageGenerationClient:
	fromembodichain.gen_sim.scene_engine.configs.environmentimport (
	read_scene_engine_env_values,
	)
	__all__= ["ImageGenerationClient"]
	classImageGenerationClient:

Context Used: CLAUDE.md (source)

Prompt To Fix With AI

This is a comment left during a code review.
Path: embodichain/gen_sim/scene_engine/clients/image_generation.py
Line: 26-31
Comment:
**Public module lacks export declaration**
The new public client module defines `ImageGenerationClient` without `__all__`, leaving its intended API implicit and exposing helper symbols inconsistently with the repository's public-module convention.
```suggestionfrom embodichain.gen_sim.scene_engine.configs.environment import ( read_scene_engine_env_values,)__all__ = ["ImageGenerationClient"]class ImageGenerationClient:```**Context Used:** CLAUDE.md ([source](https://github.com/dexforce/embodichain/blob/main/CLAUDE.md))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[Copilot]

Pull request overview

This PR refactors the GenSim Scene Engine pipeline into auditable “blueprint” stage boundaries, adds first-class scene-graph + edit flows, and expands export/import + layout utilities to support deterministic generation and iterative editing.

Changes:

• Introduces pipeline.api with analyze_* (blueprint capture) and materialize_* (generation/materialization) entry points, and updates CLI/generate/edit entry points to use them.
• Adds/extends core data structures and utilities: SceneGraph, SceneEditPlan, export/import of scene+graph, table support-surface metadata, and new layout construction helpers.
• Adds new service clients (image generation) and updates existing clients (segmentation endpoint naming, geometry seed handling), with broad unit test coverage.

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/gen_sim/scene_engine/test_simready_processor_utils.py	Adds unit coverage for VLM-driven SimReady scaling behavior.
tests/gen_sim/scene_engine/test_scene_understanding.py	Updates/extends scene understanding tests (mask overlays, orientation states, parsing rules).
tests/gen_sim/scene_engine/test_scene_layout_optimizer.py	Adds tests for table-region bounds and stacked placement behavior.
tests/gen_sim/scene_engine/test_scene_graph.py	Adds validation/serialization/constraint derivation tests for scene graphs.
tests/gen_sim/scene_engine/test_scene_generation.py	Adds calibration tests for scene-graph-conditioned orientation handling.
tests/gen_sim/scene_engine/test_scene_engine_config.py	Extends CLI behavior tests (generate vs edit vs both).
tests/gen_sim/scene_engine/test_scene_edit.py	Adds import/edit validation tests for existing exports.
tests/gen_sim/scene_engine/test_scene_edit_plan.py	Adds comprehensive tests for edit-plan parsing, validation, asset prep, and graph updates.
tests/gen_sim/scene_engine/test_scene_core_and_export.py	Expands export/import tests (graph export, global z-rotation, support metadata).
tests/gen_sim/scene_engine/test_pipeline_api.py	Adds end-to-end tests for blueprint persistence and immutability during materialization.
tests/gen_sim/scene_engine/test_clients.py	Updates client tests (segmentation endpoint rename, new image generation client, seed propagation).
embodichain/gen_sim/scene_engine/pipeline/utils/table_support_surface.py	Adds optimization rectangle computation + debug visualization for support regions.
embodichain/gen_sim/scene_engine/pipeline/utils/simready_processor.py	Refactors SimReady processing to support SceneGraph-conditioned transforms and table support-surface detection.
embodichain/gen_sim/scene_engine/pipeline/utils/scene_layout_utils.py	Adds shared pose/mesh utilities for y-up↔z-up placement and AABB measurement.
embodichain/gen_sim/scene_engine/pipeline/utils/scene_layout_constructor.py	Introduces graph-driven layout construction over table + stacked parents.
embodichain/gen_sim/scene_engine/pipeline/utils/scene_importer.py	Adds importer for exported scene+graph back into editable in-memory state.
embodichain/gen_sim/scene_engine/pipeline/utils/scene_exporter.py	Extends export to include graph + rotated export frame + metadata transforms + stale asset cleanup.
embodichain/gen_sim/scene_engine/pipeline/utils/image_segmentation_utils.py	Adds mask inversion heuristic and asset-ID overlay rendering utilities.
embodichain/gen_sim/scene_engine/pipeline/utils/gravity_settler.py	Adds generalized gravity settling for dynamic/static participants using Lab simulation.
embodichain/gen_sim/scene_engine/pipeline/utils/assets_group_layout_optimizer.py	Adjusts initial constraint handling by projecting AABBs into a rectangular support region.
embodichain/gen_sim/scene_engine/pipeline/utils/assets_gravity_settler.py	Removes legacy asset gravity settling implementation (superseded by GravitySettler).
embodichain/gen_sim/scene_engine/pipeline/generation/scene_understanding.py	Refactors understanding to return (Scene, SceneGraph) and adds orientation-state querying.
embodichain/gen_sim/scene_engine/pipeline/generation/init.py	Adds package init for generation pipeline module.
embodichain/gen_sim/scene_engine/pipeline/generate.py	Switches generation entrypoint to blueprint API (analyze_image → materialize_blueprint).
embodichain/gen_sim/scene_engine/pipeline/editing/scene_edit_layout_generation.py	Adds edit-time layout dispatch driven by updated goal scene graph.
embodichain/gen_sim/scene_engine/pipeline/editing/scene_edit_asset_preparation.py	Adds image→mask→geometry pipeline for newly added assets during editing.
embodichain/gen_sim/scene_engine/pipeline/editing/init.py	Adds package init for editing pipeline module.
embodichain/gen_sim/scene_engine/pipeline/edit.py	Adds edit entrypoint using blueprint API (analyze_edit → materialize_edit).
embodichain/gen_sim/scene_engine/pipeline/api.py	Introduces auditable blueprint/materialization API + manifest hashing + artifact recording.
embodichain/gen_sim/scene_engine/pipeline/init.py	Exposes new blueprint/materialization API from the pipeline package.
embodichain/gen_sim/scene_engine/errors.py	Adds SceneServiceError for consistent transient/remote failure signaling.
embodichain/gen_sim/scene_engine/core/scene_object.py	Extends scene object schema with center/support metadata used by layout/export/import.
embodichain/gen_sim/scene_engine/core/scene_edit_plan.py	Adds edit operation + plan model with validation and serialization.
embodichain/gen_sim/scene_engine/clients/image_segmentation.py	Renames segmentation endpoint configuration key/field to “by prompt”.
embodichain/gen_sim/scene_engine/clients/image_generation.py	Adds image generation client with seed acknowledgment and PNG validation.
embodichain/gen_sim/scene_engine/clients/geometry_generation.py	Adds seed propagation + unified SceneServiceError and response seed verification.
embodichain/gen_sim/scene_engine/cli/start.py	Extends CLI to support edit-only, generate-only, or generate+edit flows.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated 1 comment.

Suppressed comments (9)

embodichain/gen_sim/scene_engine/pipeline/generation/scene_generation.py:399

• The table-only path returns before _update_scene_final_y_up_layout_and_z_up_centers(). Consequently the table's rot, pos, and scale remain None even though SimReadyProcessor produced a valid table layout, and the subsequent SceneExporter rejects the scene because _scene_vector() requires all three final vectors. Update the table scene object before returning from this early branch.
  embodichain/gen_sim/scene_engine/pipeline/generation/scene_generation.py:274
• For edit/import paths where the table has a non-zero XY rotation, these support points are still in the table-local z-up frame, but only table_center_xy is added here. The final clamp and overlap optimizer then interpret an unrotated contour/rectangle in world XY, producing incorrect bounds and potentially placing assets outside the actual tabletop. Apply the table's complete final XY transform to both metadata arrays (or explicitly normalize the table frame before this stage).
  embodichain/gen_sim/scene_engine/pipeline/utils/image_segmentation_utils.py:115
• For masks up to 5 pixels wide/high, the 1/6 crop still covers the entire image, so outside_pixel_count is zero and this division raises ZeroDivisionError. This helper is otherwise intended to validate arbitrary server masks; return the original candidate (or use a defined fallback) when there is no outside region.

 outside_pixel_count = width * height - center_mask.width * center_mask.height
outside_foreground_ratio = outside_foreground_pixels / outside_pixel_count

embodichain/gen_sim/scene_engine/pipeline/utils/scene_exporter.py:75

• The default 180° export rotation is applied to object poses and center_xy/support metadata, but the semantic graph is written unchanged and SceneExportImporter never records or reverses this transform. After exporting then editing, table regions and left_of/in_front_of relations no longer describe the imported coordinates, so new/moved objects can be placed in the opposite region/direction. Persist the export transform and undo it during import, or keep authoring coordinates unrotated in the editable export.

 Scene layouts are y-up. The simulator automatically converts each y-up
GLB to z-up, so this exporter copies each GLB unchanged and converts
only its world position and rotation for ``init_pos`` and ``init_rot``.
The default applies one additional 180-degree global z-up rotation about
the table center to every object and XY layout metadata. ``body_scale``

embodichain/gen_sim/scene_engine/pipeline/utils/simready_processor.py:235

• When use_vlm_scale=True but use_vlm_rotation=False (a supported configuration), the scale is computed with rotate_about_x=rotate_about_x, even though the later rotate_glb_about_x_axis call does not apply that rotation. The resulting XY footprint is therefore scaled for a pose that is never used. Compute the effective applied-rotation flag first and use it for both scale calculation and mesh rotation.
  embodichain/gen_sim/scene_engine/clients/image_segmentation.py:141
• This now makes ImageSegmentationClient.from_dotenv() require SCENE_ENGINE_IMAGE_SEGMENTATION_BY_PROMPT_PATH, but the existing Scene Engine setup instructions still define SCENE_ENGINE_IMAGE_SEGMENTATION_SINGLE_OBJECT_PATH (docs/source/features/generative_sim/scene_engine.md:46). A documented .env therefore fails during client construction with a missing-key error; update the configuration documentation/sample in the same change or support the old key as a compatibility alias.

 "SCENE_ENGINE_IMAGE_SEGMENTATION_BY_PROMPT_PATH",

embodichain/gen_sim/scene_engine/core/scene_graph.py:143

• relation is only a Literal annotation here and is never checked at runtime. An invalid value such as "adjacent_to" is accepted, then _inverse_planar_relation() silently maps it to in_front_of, so validate() can persist a malformed semantic graph instead of rejecting it. Validate membership against the four supported planar relations before normalization.

 def __post_init__(self) -> None:
"""Validate local relation fields before graph-level checks."""
_validate_stable_id(self.source_id, field_name="source_id")
_validate_stable_id(self.target_id, field_name="target_id")
if self.source_id == self.target_id:
raise ValueError("relation endpoints must be different.")

embodichain/gen_sim/scene_engine/core/scene_graph.py:247

• Although this method is documented as applying an atomic batch, it mutates self.nodes and self.relations before validating all additions, parents, and relations. If any later check raises (for example, an unknown parent or a planar conflict), the caller is left with a partially applied graph and cannot safely retry. Apply updates to a copy and commit only after refresh() succeeds, or roll back on failure.

 # Delete all requested nodes before resolving new parents and relations.
self.nodes = [
node for node in self.nodes if node.object_id not in deleted_object_ids
]
self.relations = [
relation
for relation in self.relations
if relation.source_id not in deleted_object_ids
and relation.target_id not in deleted_object_ids

embodichain/gen_sim/scene_engine/pipeline/api.py:258

• This API eagerly constructs the VLM, image-generation, geometry, and segmentation clients and health-checks all of them even when the edit plan contains only moves/deletes. prepare_scene_edit_assets explicitly skips service use for plans without adds, but this wrapper still requires every .env key and live service, so a non-generative edit fails unnecessarily. Gate client creation and health checks on whether an add operation is present.

 effective_vlm = vlm_client or OpenAICompatibleVLM.from_dotenv()
image_generation = image_generation_client or ImageGenerationClient.from_dotenv()
geometry = geometry_generation_client or GeometryGenerationClient.from_dotenv()
segmentation = image_segmentation_client or ImageSegmentationClient.from_dotenv()
owned_clients = (