Remote Contexts

A remote context is another process's object tree, mirrored into this one and shown beside the local context: a second root in gcompose's context panel, and its own grouping area in the node graph.

The whole design turns on one decision. uLib::ContextClient is an ObjectsContext, not a peer of one. Everything that already walks a context tree — the context panel, the node graph, CaptureContext, a Python script — reaches a remote scene through the same GetChildren()/GetObjects() it uses for the local one, with no branch for "is this one remote". Adding a ContextClient to the root context is therefore all it takes for a separate context to appear in the tree.

The pieces

Layer	Class	Lives in
Transport, server half	uprops::PropertyServer	Core/uprops/backend/
Transport, client half	uprops::PropertyClient	Core/uprops/backend/
Publishing a context	uLib::ContextServer	Core/ContextServer.h
Mirroring a context	uLib::ContextClient	Core/ContextClient.h
Headless server binary	ucontext-server	app/ucontext-server/
GUI entry point	RemoteContextDialog, ContextPanel	app/gcompose/src/

ContextServer and ContextClient are both Qt-free and VTK-free. (The ucontext-server binary around them does link VTK when built with an embedded interpreter — see "Recipes run on the server too" — but the library layer does not.) gcompose keeps its own RemoteBridge on the serving side, because its objects have Geant4/VTK thread affinity, its edits must be journaled into the ActionRecorder, and its clients reach display properties through a "<id>#display" lookup — none of which belongs in Core. Both speak the same wire protocol, so a client cannot tell them apart.

Serving a context

ucontext-server --port 50051 --add ContainerBox:world
ucontext-server --port 50051 --scene scene.gcx
ucontext-server --list-classes

In code it is two lines:

uLib::ObjectsContext context;
uLib::ContextServer server(context, "my-app");
server.Start(50051);

ContextServer installs the container provider (the whole tree flattened, so a nested object is addressable too), the lookup, the whole-object state provider/applier, and the create/remove/duplicate/reparent/rename handlers. It also gives every object a stable id: PropertyServer's own fallback is the object's position in the provider list, which a new sibling renumbers.

If the objects have thread affinity, install a dispatcher:

server.SetTaskDispatcher([&](const std::function<void()>& task) { runOnGuiThread(task); });

Mirroring a context

auto* remote = new uLib::ContextClient();
remote->Host = "192.168.1.20";
remote->Port = 50051;
remote->Protocol = "flatbuffers";   // the default
remote->Connect();
localRoot.AddObject(remote);        // now it is in the tree

What the mirror holds are real local uLib::Objects, rebuilt by ObjectFactory from the SceneDocument the server sends — not proxies. Their properties are the ordinary Property<T> objects every panel already edits. A class the local build does not know is skipped and reported through GetWarnings(); the rest of the tree still mirrors.

The remoteness is the traffic around them:

• inbound, the server's property_changed events are queued by the socket thread and applied by ProcessEvents() on the owner's thread (gcompose runs a 200 ms timer in ContextPanel);
• outbound, an edit made on a mirrored object is pushed back as set_property, by an IPropertyBackend installed on each mirrored object.

The echo is cut by an applying flag around the inbound write, not by the payload's PropertyChangeOrigin — origin says which setter path was taken, not who asked, and SetValueFromString defaults to Remote.

A structural event (object_added / object_removed) is answered by re-pulling the whole tree. RestoreContext adopts what is already there rather than rebuilding it, so a pull invalidates neither a selection nor a node's canvas position.

Work happens where the object is

Editing a mirrored object sends the property, not the object: the outbound backend emits one set_property, and the server then emits Updated() on the object so the consequences are computed there. The client never pulls an object over, changes it locally and pushes the result back.

Running is the same idea with a command of its own. invoke carries an action name — run, start, stop, pause, resume for a uLib::Algorithm, and update for any object at all — and ContextServer::PerformAction carries it out. gcompose's Play button checks whether the selected algorithm lives in a remote context and, if it does, sends start (falling back to the blocking run if the server refuses it) instead of running anything locally. An application adds verbs of its own through ContextServer::SetActionHook.

Announced, not shipped

When a served object emits Updated(), the server broadcasts object_updated — the fact that it moved, not its new contents. Those events are coalesced, at most one per object per 50 ms, because a running algorithm emits Updated() freely and a client only needs to know that it should look again.

The client marks the mirror stale and stops. What actually gets fetched is decided by ContextClient::SetRefreshFilter, because Core cannot know what is worth fetching. gcompose installs one that answers yes when either:

• some pane is drawing the object (a Prop3D in a VTK mirror, or a node in a node graph), or
• something outside the remote context is wired to it (ContextClient::HasExternalConsumers) — a local object connected to a remote one's output is waiting on that update even when nothing is drawing the remote object itself.

An object the filter declines stays stale, so it is fetched the moment it does become worth showing rather than being skipped forever. With no filter installed, every stale object is fetched — the right default for a script that has no notion of visibility.

A fetch is a whole-object PullObject, which ends in NotifyPropertiesUpdated() and Updated(): the first carries the new values into whatever is connected downstream, the second tells the panes and the graph that the object moved. That is how the update reaches a local object wired to a remote output.

Documents

A .gcx records the whole networked composition: the remote context's address and its contents. CaptureContext refreshes every connected remote context in a pass of its own before the walk begins — under lazy refresh the mirror may be deliberately behind, and pulling mid-walk would mean changing the tree while capturing it — then descends into it like any other context.

Reopening repopulates the mirror from those records, so a document can be opened and read with every server it names switched off.

What it must not do is connect and pull: a pull replaces the mirror with the server's own state, and the restored objects — which for a scene that was saved exist nowhere else — would be dropped on the spot. So the load goes the other way. ContextClient::PublishContext() sends the mirror's contents to the server and makes the server match them: an object it already has is adopted by class and name and updated in place, what is missing is created, and what the document does not have is removed. The mirror is re-pulled at the end, so it comes back carrying the server's ids and watched for edits like any other.

gcompose does that automatically on open for every restored remote context that has contents, and Publish Context To Remote in the context tree does it on demand. A server that is not listening leaves the context offline holding what it last had, exactly as before — the connect is bounded by PropertyClient's timeout, so opening a file cannot hang on one. Publishing is refused on an empty mirror: "publish nothing" and "empty the server" are the same request on the wire, and a context that came back empty is far more likely to mean the first.

Protocols

ContextClient::Protocol selects the encoding for whole-object payloads, and defaults to flatbuffers: the generic, self-describing tier, so a peer that was not compiled against a class can still read its objects. native-bin is the lossless same-build alternative. GetSupportedProtocols() offers only the type-agnostic codecs — a codec registered for one type (stl for a mesh, vti for an image) would silently fall back for everything else.

The receiving end does not need to be told which was used: Push encodes with EncodeObjectPayload, and DecodeObjectPayload recovers the encoding by asking CodecRegistry to probe the bytes. Bytes no decoder claims are treated as a native SceneDocument payload, which is what every older sender writes.

A remote context is its own world

Move a recipe, an algorithm, or any object into a remote context and it should behave as if it had always been local there — so what it sees is that server's objects, and only those. uLib::EffectiveContextFor(root, object) answers that question once: the enclosing ContextClient if there is one, and root otherwise (which is every object in an ordinary session).

The rule is not symmetric, and the asymmetry is the whole of it:

• A local object may reference a remote one. The mirror is an ordinary local object, the link is made here, and nothing has to be sent anywhere.
• An object inside a remote context may only reference objects in that same context, because that link has to be made on the server too (ContextClient::SetReference) and a local object has no id the server could resolve.

So a candidate walk scoped to a remote context stops at the boundary, while a walk of the local root goes through it. What that scoping reaches:

• Reference dropdowns. PropertyEditor resolves the effective context of the object it is showing and hands it to ReferencePropertyWidget; the walk that fills the combo box does not descend into a nested ContextClient.
• A recipe's ctx. PythonEngine::Private::ScopeGuard, installed at the one choke point every recipe run passes through (Recipe::SetExecutor), points ctx.find / ctx.objects / ctx.roots at the recipe's own context for the duration — so a recipe in a remote context sees the server's scene and not the local one.
• Deleting. gcompose::RemoveObject — the one helper every delete in the GUI already goes through: the context tree's Delete key and its menu, the recipe panel's delete, the Python console, a replayed recipe — sends the removal to the server when the object lives in a remote context. Dropping it from the mirror alone would leave it on the server, and the next pull would bring it straight back, so the delete would look undone. Deciding it in the helper rather than at each entry point means none of them has to know about remote contexts at all.
• ctx.create and ctx.remove. Creating from a remote-scoped recipe goes over the wire through ContextClient::CreateRemoteObject; the object is built on the server, and the re-pull that follows brings it back into the mirror, so it appears in the tree and the node graph. Creating it locally instead would put it in a mirror the next Pull() discards, which looks exactly like nothing happening. ctx.remove goes through the same gcompose::RemoveObject as every other delete, so it is decided by where the object actually is rather than by the running scope. An explicit parent= wins over the running scope, so ctx.create(..., parent=x) always lands where x is.

Recipes run on the server too. A Recipe is an Algorithm, and it takes the same invoke path — the point of moving work to another machine is that the other machine does it. It needs one extra step first: a recipe keeps its source in a plain member rather than a Property, so no edit of it ever reaches the server on its own, and invoking it without pushing would silently run whichever copy the server last received. ContextClient::PushAndRun sends the whole object, source included, and only then runs it.

Both of gcompose's run buttons go through it — MainPanel's Play and RecipePanel's own Run, which reaches the engine directly rather than through Play, so it has to know about the remote case separately.

run, not start: a recipe's body is Python, which the server has to execute on the thread its interpreter belongs to rather than on a worker of its own. So the call lasts as long as the script does, and its timeout is a bound on the job rather than on the round trip.

For any of that to mean anything the server needs an interpreter, so ucontext-server embeds one (UCONTEXT_SERVER_PYTHON, on by default). It reuses gcompose's PythonEngine rather than growing a second, subtly different ctx facade — a recipe should mean the same thing wherever it runs. The cost is worth naming: that engine reaches VTK for the obj.display properties, so the "headless" server links VTK, and Qt through it. Headless here means no GUI, not no toolkit; turn the option off for a build that wants neither, and recipes sent there are then refused with "no script executor registered" rather than quietly running somewhere else.

Because an embedded interpreter belongs to one thread, ucontext-server also installs a task dispatcher that marshals every client command onto its main loop — which was doing nothing but waiting anyway. That fixes concurrent commands from separate client sockets at the same time.

Local edits of a mirrored object

A mirrored object can be edited here — a recipe writing into a VoxImage's buffer is the usual case — and that edit has to reach the server, or the next refetch of that object lands the server's untouched copy back on top of it and the change appears and then vanishes.

A raw buffer write fires no property event, so the property channel cannot carry it. What it does have is Updated(): the object saying its contents moved. ContextClient::WatchLocalEdits treats a locally-originated Updated() on a mirrored object as exactly that, marks the object dirty, and ProcessEvents() pushes it whole — coalesced per call rather than sent per signal, because Updated() is cheap to emit and a whole-object push is not.

Dirty is the mirror image of stale, and the two must not fight:

• PushDirty() runs before RefreshStale(), because an object edited here is ahead of the server and fetching it first would be undoing the edit and calling it an update;
• RefreshStale() skips a dirty object for the same reason;
• a successful Push or PullObject clears the mark, because either one leaves mirror and server agreeing. Leaving it set would let a later push send that same state a second time, on top of whatever the server had done since — an old value overwriting a new one.

A dirty object wins. An object edited here and not yet sent is ahead of the server, so anything the server has to say about it was said before the edit. Applying that would undo it — the change appears and is replaced by the old value a moment later. So while an object is dirty, inbound property_changed for it is skipped, and a whole-tree Pull() pushes first. The edit wins for at most one ProcessEvents() call, which is the same call that sends it.

PushLocalEdits (on by default) turns the whole thing off for a client that wants to be a pure viewer.

A move is lossless. AdoptObject and TransferObject encode with native-bin rather than the selected Protocol, whatever it is set to. The generic tier says so itself — FlatBuffersEncoder::IsLossless() is false, "whatever a class holds outside its own serialize() does not survive". Losing part of an object while syncing it is a stale value the next sync corrects; losing part of it while moving it is the only copy, and the property arrives at the far side sitting at its default.

Cross-context connections

A node graph edge (or a Properties-panel combo box) between two mirrored objects is drawn the ordinary local way — both ends are live local objects — and has to reach the server on its own, over one of two mechanisms depending on what kind of link it is.

A value connection (uprops::IProperty<T>::ConnectFrom, an ordinary port-to-port wire) rides the same channel a plain edit does:

• when the driven end is remote, the connection's own PropertyUpdated reaches the outbound backend and leaves as set_property;
• when the driving end is remote, the server's property_changed event updates the mirror, which re-drives the local consumer.

A reference connection (ReferencePropertyBase::SetReferencedObject — what a "(object)"-port wire or a reference combo box actually sets) cannot ride either of those. Its own PropertyUpdated() override does not build a PropertyChangePayload at all (it only fires Object::Updated()), and even if it did, the value would be another object's identity, not a string SetValueFromString could resolve. It has a wire pair of its own instead: set_reference (client → server, objectId/propertyName/targetObjectId, empty target meaning disconnect) and reference_changed (the broadcast every subscriber gets once the server has actually made the connection).

Both directions are automatic, driven by watching the reference property's own Object::Updated() — the one signal it reliably fires regardless of which overload ran:

• outbound, ContextClient::WatchReferences (installed by WatchMirror on every input-side reference property of every mirrored object) sends set_reference when the property moves locally. Only expressible when both ends are mirrored by this ContextClient; a reference to a local object, or into a different remote context, cannot be sent and is left local-only.
• inbound, ContextServer/RemoteBridge's own WatchReferences broadcasts reference_changed for the same reason — from a client's set_reference, or from an edit made directly in that process — and ContextClient::ProcessEvents applies it, resolving the target by remote id and calling SetReferencedObject under the same applying guard every other inbound write uses.

An explicit ContextClient::SetReference(owner, propertyName, target) is also available directly, for a caller that already knows both ends are mirrored and wants to skip the local assignment + watcher round trip.

Ports that appear later. Walking an object's properties once is not enough. A Recipe's ports are dynamic slots: made when its source is parsed, and destroyed and remade whenever it is edited. A wire drawn onto a port that did not exist at the last walk would never be watched, so it would never be sent, and the recipe would run on the server against nothing. Both ends therefore re-scan an object's reference properties on every Updated() — which is what an object emits when its port list changes.

Re-scanning also has to notice a slot that was replaced: the new property can land on the freed address of the old one, so the "already watched" test is whether the stored connection is still connected(), not whether the address is in the map.

Bulk content (a mesh, an image) does not move on a property event, so it moves whole: ContextClient::Push(object) and PullObject(object), wired to Push To Remote / Pull From Remote in the context tree. Push uses the selected protocol. PullObject currently rides the SceneDocument envelope regardless of Protocol, because making that direction selectable needs the requested protocol to reach the server's state provider and the line protocol does not carry it today.

Working in a remote context

A remote context is a place you can put things, not just a view of somebody else's scene.

Creating. Select the remote context (or anything inside it) and use the Add menu as usual: MainPanel::onCreateObject sees the selection is remote and routes to ContextClient::CreateRemoteObject, which sends create and re-pulls. The object is built on the server; what appears in the tree is its mirror. Selecting a mirrored object that can hold children nests the new object under it; anything else puts it at the remote context's own root. The create is deliberately not journaled into the ActionRecorder — replaying it would build the object here rather than on the server it belongs to.

Moving. Dragging an object onto a remote context's row in the tree moves it there: uLib::TransferObject re-creates it on the far side from a serialized copy (create for the class, then set_state with the payload encoded in the selected protocol) and only then drops the original. Dragging one back out is the same two steps in the other order — built locally by ObjectFactory, filled from the mirror's serialized state, then removed on the server.

The original is removed after the far end has it, never before: a transfer that half-fails should leave a duplicate, not a hole. TransferObject reports a move whose two ends turn out to be the same process through its sameProcess flag rather than performing it, because an in-process reparent also has to preserve the world transform — which lives in Math/HEP, out of Core's reach.

Not in the 3D viewport

A remote context has no VTK representation at all. It is a place objects live, not one of them: it has no geometry, and Vtk::ObjectsContext::CreateProp3D would otherwise fall through to its generic "this is a context, wrap it in an assembly" branch and put a selectable, gizmo-draggable group around its contents — standing for nothing that can actually be moved.

Its children are unaffected and draw exactly like any other object: OnObjectAdded recurses into an object's children whether or not the object itself got a prop, so removing the wrapper removes only the wrapper.

A remote context therefore shows up as a grouping in the context panel, as a labelled area in the node graph, and as a name a script or a reference can resolve — and as nothing at all in a 3D pane.

In the node graph

Every remote context gets a NodeGraphQt backdrop — its grouping primitive, so drag-with-contents and a resize handle come for free — labelled with the context's name and endpoint. The area is declared the moment the context appears (NodeGraphPanel::syncGroups), not when its first node does: a context that has just connected is usually empty, and an empty labelled rectangle is what the user aims at to put the first object in it. A remote context has no node of its own — it is the area, the same way the local root context has no node.

auto_layout lays the graph out one group level at a time, innermost first: a context's own nodes (and the nested groups inside it) are arranged within its area, and the area then takes part in the level above as a single block the size of its contents, which is what keeps a remote context's nodes inside its own area instead of interleaved with the local ones. At every level each connected set of items is drawn as a left-to-right layered flow (sources left, wires lined up port to port, long wires kept clear of the nodes they pass), and the flows, unconnected areas and loose nodes are then packed into rows sized to the view rather than stacked in one column. Each backdrop is refitted around what ended up in it. An area goes away when the context it stands for does, not when its last member leaves.