Uh oh!
There was an error while loading. Please reload this page.
Beta: components are the MuJoCo model - #85
Open
jonathanembleyriches wants to merge 32 commits into
Open
Conversation
A per-mode strategy replaces the branching that had grown across live, direct and puppet stepping. The worker reads shadow atomics rather than UPROPERTYs, joins before teardown, publishes only when it actually advanced, and paces the live loop with a hybrid sleep instead of a full spin.
Capture became the bottleneck: a synchronous readback stalled the stream. It is now an async pipelined GPU readback with a one-deep queue and drain-to-latest, a frame history addressable by frame id, and a render-on-demand fast path for callers that need the frame the step produced. Camera identity is canonical and filename-safe, and fovy converts from MuJoCo's vertical to Unreal's horizontal.
Per-domain handler files instead of one dispatcher, long editor operations report through a job poll rather than blocking, and control is claimed and released per articulation so two clients cannot fight over one robot.
An oversize reply falls back to ZMQ instead of stalling, and the shared-memory RPC contract is advertised in the handshake rather than assumed.
Two serialization paths had drifted. There is now a single intermediate representation carrying raw MuJoCo SI values, one msgpack encoder over it, and user-authored payload channels that are transport-neutral by construction.
URLabRos links the rcl C API directly -- no rclcpp, no shim -- and registers providers that read the same IR the other transports do. JointState, IMU, twist, tf2, clock, camera image, odometry, point cloud, occupancy grid and octomap. The core never references ROS, so the plugin builds and runs without it.
URDF export with an auto-generated SRDF, the simulated world published as a planning scene, a trajectory execution bridge, gripper control and a pick pipeline.
Class-inherited actuator gains survived neither import nor export. Geom type and size did not resolve through defaults. MJCF <include> fragments were not resolved at all, and exported meshes carried no normals.
The schema this rebuild generates from is MuJoCo's own, so the engine and the grammar move together.
MuJoCo ships `src/xml/mjcf.schema` and drives its own reader and writer from it. This is the same idea carried into a library: the schema is the single source of truth, and the object model, the reader, the writer and their tables are emitted from it rather than written by hand. What is here is the library and its generator, standalone and with its own build and test suite. Nothing in the plugin uses it yet. The point is the cost of the next MuJoCo release. Hand-written binding code is what makes a version bump expensive, so anything derivable from the schema is derived, and a schema change that our tables do not account for fails generation by name rather than compiling into something subtly wrong.
A robot is authored as Unreal components in a Blueprint, and those components ARE the MuJoCo model: there is no mirrored document and no translation layer. The components write straight into MuJoCo's own in-memory model description, scenes are composed with `mjs_attach`, and each component holds a handle to its element so binding needs no names. What that replaces: building MJCF text from the components, assembling a scene by string surgery and a regex over asset paths, loading the text back through the parser, and finding each component's simulation id by name afterwards. The reader and writer stay, and only at the file boundary. mjSpec cannot express which shorthand an actuator was authored as, so a `<position>` loaded through it comes back a `<general>`; importing and exporting through our own reader and writer is what keeps an authored element the element the user wrote. The three macro elements are expanded by MuJoCo's parser through one contained text hop, because nothing else can expand them. Everything the schema can describe is generated. What is left by hand is the walk over the components, and the seams where MuJoCo's C API has no field to write to.
Mount assets under the names the spec references, so a model whose visual and collision meshes share basenames stops resolving both to the visual one through MuJoCo's basename fallback. Read an explicit top-level <default class="main"> as the root class it already is. Plugins: install and load the engine plugin libraries, create the element a <plugin> actuator or sensor stands for, and carry <config> through MjShim, since mjs_setPluginAttributes takes ownership of a std::map the caller allocates and Unreal replaces operator new per module.
clang-format over the hand-written sources, and LF in the repository and on checkout on every platform. CRLF was reaching the commits, so a file touched on Windows read as changed in its entirety anywhere else. The formatters now skip *.gen.h and *.gen.cpp: the emitter owns those bytes and the codegen drift gate compares them exactly, so formatting them here and regenerating each undid the other.
The details-panel buttons were unreachable: they asked the geom's own `type`, which reads `sphere` when unset, and a menagerie model puts `type="mesh"` on a default class. `EffectiveType()` answers what the geom compiles as, which is what decides whether it can be decomposed. Decomposition is an action, not an attribute, so it lives on the Blueprint editor's component tree now rather than beside the geom's real attributes, with threshold and extrude offered where it is invoked. Extrude reaches CoACD at last -- it was hardcoded off -- and both parameters are part of the mesh cache hash, or changing them would reuse stale hulls. A hull also now gets a `<mesh>` element and the geom's `mesh` reference, which it never had: the OBJ and the UStaticMesh existed, so the hulls drew in the editor and compiled to a mesh geom naming nothing. The assets land beside the articulation's own, under its Decomposed folder.
`RootOfBlueprint` reads a template graph through FMjScsScope, and both are editor-only, but it was gated on URLAB_MJ_GEN alone. `OverBlueprint` already compiles its own body away without the editor; its helper did not, so the game target failed to compile and the plugin could not be packaged.
The component model changed underneath anyone updating from the previous generation, and nothing migrates their Blueprints. Say so at the top of the README, and point at the alpha branch and its frozen tag for users who would rather not re-import yet.
The plugin still called itself 0.1-alpha five releases in, which is what the preserved alpha snapshot was nearly tagged as. It is 0.6.0-beta now, and the alpha tag reads v0.5.0-alpha to match the last released milestone.
On Linux with Clang the spec template code instantiated far more than it needed and used tens of GB per file, which caused out of memory during the build. Two changes fix it with no runtime or behavior change: - ParentMap now walks the tree over base node pointers and dispatches to the concrete type once per node, instead of instantiating the record step for the whole element containment closure. - DispatchChild now dispatches only among the child types a parent admits, instead of the full element set, which removes the O(types squared) fan out in the MJCF reader and writer. Also make the four basic shape mesh path constants constexpr so Clang accepts them in the constexpr shape table (MSVC allowed the plain const form).
Pass UE's target and sysroot into the third party compiler and linker flags so the static libs target UE's glibc, not the host's newer one. Without this, on a host with a newer glibc the deps reference symbols UE's glibc lacks and the plugin link fails. Also build protospec from build_all.sh so it inherits the same toolchain, and ignore the per dep build scratch dirs that sit beside src.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mjcf.schema, and the UE components are the MuJoCo model rather than a mirror of it.mj_loadXMLproduces, checked field by field against every robot in a Menagerie checkout.Motivation
The previous element tree was hand-followed: every MuJoCo version bump meant chasing the schema by hand, and the components mirrored the model rather than being it, so the two could disagree. Generating the tree from
mjcf.schemamakes a version bump a regeneration, and storing every attribute asTOptional— where unset means the document did not author it — is what lets the default class answer instead.Breaking changes
MjArticulation assets from any alpha will not open. Their components are the previous generation's classes and nothing migrates them. Re-import the MJCF, or stay on the
alphabranch /v0.5.0-alphatag, which are frozen for exactly this.RPC: the control-owner override field is
control_owner, notsource.set_control_sourcealready spentsourceon"zmq"|"ui", so that op could never pass its own ownership check. Requests that do not set it are unaffected; the owner still falls back tosession_id.Notable fixes
multiccdfollows MuJoCo again. The scene manager disabled it on every scene, which mirrored MuJoCo's default when the flag was opt-in; MuJoCo has since moved it to the disable family where it defaults on. Convex-convex pairs were generating one contact point where MuJoCo generates several.<option>reaches the scene. The scene's<compiler conflict>ismergerather than MuJoCo'swarningdefault, because the attach target is a scene nobody authored and underwarningevery field an import brought lost to a default.ctrltofloat, so afloat64setpoint arrived rounded and diverged from stock MuJoCo given identical input. The path isdoubleend to end, which is whatmjtNumis.UPROPERTY, so a Blueprint recompile reinstanced them and left the index pointing at the objects it replaced.rgbais drawn; previously a colour only applied when a material was also named.mjtNum, and now has tests.What the diff size actually is
+212,904 / -85,544across 992 files. Broken into disjoint buckets that reconcile to that total:mjcf.schema, plus ProtoSpec's reflection / visit / XML binding. Nobody writes or reviews these; a drift gate fails the build if they go stale.Source/)mjcf.schema, plus its JSON tables.Scripts/codegen/and its snapshots, replaced by the above.Two things make the gross number large without making the review large:
Scripts/codegen/and its JSON snapshots — a single 27k-line introspection snapshot accounts for most of it.+50.5k / -34.4k, a net ~16k, because the hand-maintained tree comes out as the generated one goes in. Several files also moved (MuJoCo/Components/…toMuJoCo/Elements/…) and were rewritten enough that git scores them as add-plus-delete rather than renames.The reviewable change is concentrated in
Source/URLab/Private/MuJoCo/Spec/(reader, writer, spec build),Source/URLab/Private/Bridge/(RPC), andSource/URLabRos/(the new optional ROS module).The whole-tree clang-format pass and LF normalisation is isolated in
3381d85(1,214 lines) and listed in.git-blame-ignore-revs, so it can be skipped when reading history.Test evidence
Full
URLab.*automation suite, clean build with the editor closed:Also verified: 65 Menagerie robots matching stock
mj_loadXMLfield by field, the game target builds (packaging works), and the Python bridge's live integration suite green against a running editor.Manual verification
<mesh>elements and assets beside the articulation's own.Checklist
URLab.*automation suite passes