Streaming into ROOT, and what can be streamed at all¶

Two things live here: the sink and source that put a stream into a TFile and take it back out, and the review of which objects in the library can be put on a stream port — because the second is what made the first useful.

1. The review: who streams, who did not¶

The streaming mechanism itself is src/Core/uprops/docs/streaming.md (in the source tree, not the site); the Python side is Streaming between recipes. This is the census of objects.

Producers¶

Object	How
`Abstract::Collection<T>`	for `T` deriving `Object` — so `GeantEventCollection` streams its events
`DataSource`	the bytes it holds
`DataBuffer`	a relay: `Input` in, a bounded queue out, `NotifyEvery` to announce
`MuonScatterSource`	the live one: a bounded, waitable queue a run pushes into
`Geant::Scene`	an activator — asking it for muons runs it
`Math::Field`, `FieldSampleSet`	samples, driven or collected
`Recipe`	one `GenericStreamProducer` per declared stream output
`RootScatterReader`, `RootStreamSource`	a file, read back
`MuonScatterCollection`	added — see below
`Math::PointSet`	added
`Math::EventTable`	added

Consumers¶

DataSink (ConsumeStream), MuonTomographyAlgorithm (Stream port, DriveStream), Math::Field, Recipe (a declared stream input), RootScatterWriter (muons only), and RootStreamSink (anything).

What was missing, and what was done about it¶

MuonScatterCollection was not a producer. This is the one that mattered. Every muon consumer in the library reads a stream port, and the collection is where the muons of a finished run are — so reading a completed run meant copying it into something streamable first. It now hands its events over as MuonScatterEvent carriers, one refilled carrier, which is the pattern MuonScatterSource established.

PointSet and EventTable were not producers. They hold exactly the shapes a measurement and a record of measurements have. Each now streams FieldSamples: for a point, the abscissa is the sample's point and y with its error bars is the value; for a table, the row is the value and the row index the point. The column names stay on the table, where GetColumns() says what the positions mean.

Abstract::Collection<T> streams nothing when T is not an Object. GetStreamItem returns null for a Collection<float>, so such a collection is a producer in name and empty in practice. Not fixed, because the fix is not general: a value needs a carrier object to travel in, and which carrier depends on the value. The three classes above show the shape of the fix for anything that needs it — a member carrier, refilled per item.

Left alone deliberately. Histogram (a binning, not a sequence), MultiVoxImage (a volume; streaming its slices would be inventing a meaning nobody asked for), PlaneProbe and TrackRecorder (their records are Geant-run state, and the run already streams through the scene's ports). Each would be a real design decision rather than a missing port.

2. `RootStreamSink`: a stream into a TFile¶

anything with a Stream port --(Stream)--> RootStreamSink --> TFile / TMemFile

RootScatterWriter writes muons and only muons. This writes whatever the stream carries, deciding per item:

Item	Tree
`MuonScatter` / `MuonScatterEvent`	the row `RootScatterWriter` writes, so either file reads the same
`Math::FieldSample`	`point[]` and `value[]`, variable length — numbers and vectors
anything with properties	a branch per serializable scalar property, derived from the property model

That last row is the general case and the one worth noticing: a ContainerBox streams its Position, Rotation, Scale, Size and Origin because those are what it declares. Nothing taught the sink about boxes, and a class written tomorrow needs no change here. The branch names come from the same declarations the FlatBuffers schema and the agent context come from, so a tree cannot describe a version of a class that no longer exists.

One tree per TreeName and per item type: changing TreeName between runs means another tree, and a stream carrying two kinds of item gets a tree each rather than one tree that fits neither. The type's name is written beside each tree as a TNamed, which is what the source reads back as a label.

It is an Algorithm¶

So it runs, chains and auto-runs like everything else:

RootStreamSink sink("file");
sink.FilePath.Set("run.root");
sink.SetStream(producer);
sink.Run();

and it is also an IMuonScatterSink, so a Geant::Scene can push muons straight into it with no queue in between.

Writing while the events arrive¶

The live arrangement, and the two settings that make it one:

source->NotifyEvery.Set(16);     // announce every 16 events
sink->DriveStream.Set(false);    // drain what arrived; do not ask for more
sink->CloseOnFinish.Set(false);  // one file for the whole run
sink->SetAutoRun(true);          // answer the announcements
sink->SetStream(source);

The sink follows its Stream port: an announcement from the producer runs it. That is not decoration — without it nothing runs the sink while the run is going on, the bounded queue upstream fills, and everything past its Capacity is dropped. src/Root/examples/03_live_stream_to_root.cpp found exactly that: 436 of 500 events dropped before the sink was taught to follow, none after. MuonScatterSource::Dropped is what makes such a loss visible rather than mysterious.

End a run with one explicit Run(): the last partial batch is below the threshold, so no announcement is coming for it.

In memory¶

InMemory makes the file a TMemFile — a whole ROOT file, with trees ROOT itself reads, that never touches a filesystem:

sink.InMemory.Set(true);
sink.CloseOnFinish.Set(false);           // keep it open; the bytes are wanted
sink.Run();
std::vector<char> bytes = sink.CopyBuffer();   // a complete .root
sink.SaveAs("later.root");                     // ... written out if you like

What a test wants (no temporary files, no cleanup), what a short-lived pipeline wants (write, read, discard), and what sending a file somewhere wants.

3. `RootStreamSource`: the file as a stream¶

RootStreamSource source("replay");
source.FilePath.Set("run.root");
source.CompleteOnly.Set(true);           // muons that crossed both chambers

StreamInput input;
input.Connect(&source);
while (Object *item = input.Read()) { ... }

Or from the bytes of an in-memory file: source.OpenBuffer(sink.CopyBuffer()).

What it hands back are the carriers consumers already read — MuonScatterEvent for a muon tree, FieldSample for samples and for property trees. Which kind a tree is comes from its branches, not from its name, so a file written by RootScatterWriter or by somebody else's ROOT program reads too. ListTrees() reports every tree with its kind, entry count, recorded type name and branch names.

It is a pull producer (CanPullStream() is true): the entries are already there, nothing waits, and a file can be re-read — ResetStream() rewinds, which is what makes it a collection rather than a queue.

4. Examples and tests¶


`src/Root/examples/02_streaming_into_root.cpp`	numbers, vectors, muons and an object's properties — on disk and in memory, each read back and each file opened with plain ROOT
`src/Root/examples/03_live_stream_to_root.cpp`	events written while they arrive, through a bounded queue
`src/Root/testing/RootStreamTest.cpp`	the file: every item kind, both directions, mixed trees, and every refusal
`src/Root/testing/RootStreamCoverageTest.cpp`	the ports: the three new producers, a document round trip, and the live chain with `AutoRun` on and off

Every example checks its own files with plain ROOT at the end. A .root that only this library can read would be a private format with a misleading extension.