The data manifest

Stable since 1.0

The manifest schema, diagram-manifest element id, and data-node-* attribute names are stable as of nf-metro 1.0 and covered by semantic versioning. Incompatible schema changes will bump the version field and the nf-metro major version together.

Every SVG nf-metro renders is a self-describing, addressable artifact. A downstream tool can position overlays on it, restyle its nodes and look up which processes a node represents, all from the committed file alone and without re-running whatever drew it.

That contract is not specific to metro maps. This page documents the format as a standalone standard, along with the tooling nf-metro ships to produce and consume it, so any diagram tool can emit a conforming SVG. nf-metro is only the first producer.

Just want to make one? Start with the tutorial

If you only want to produce an SVG and drive it from events, skip the spec and jump to the tutorial. It builds one from scratch and finishes with a single script you can copy and run. The sections in between are the format reference, aimed at implementers.

Headed for its own package

The tooling lives in nf_metro.manifest, a dependency-free module that uses the Python standard library only and imports nothing else from nf-metro. It is structured so it can later be lifted into its own distribution unchanged. Until then, import it from nf-metro.

Terminology

The format is tool-neutral, so its vocabulary is generic rather than metro-flavoured:

Term	Meaning
manifest	The JSON description of the diagram, embedded in the SVG.
node	An addressable point on the diagram - the thing a consumer locates, restyles, or lights up. Has an id, a centre (x/y), a radius (r), and an optional label.
group	An optional multi-membership category a node can belong to several of, each with a display color (e.g. a colour-coded series).
region	An optional single-membership container a node sits inside (e.g. a labelled box).
pattern	A regex on a node that identifies it against a runtime string. A node carries zero or more.
target	What the patterns are matched against, named in the manifest’s match block (for a Nextflow run, the fully-qualified process name).

A producer with no grouping concept uses nodes alone and leaves groups and regions empty.

How nf-metro maps onto it

nf-metro draws metro maps, so its own code, .mmd files and live server speak metro: stations, lines, sections, processes. The renderer’s adapter translates those into the neutral wire vocabulary, so the SVG you get uses the generic terms above:

nf-metro (metro)	Manifest (neutral)
station	node
line	group
section	region
%%metro process: patterns	node patterns

If you author a metro map and then read the rendered SVG, you will find nodes rather than stations. That is expected, and it is what makes the file portable.

What’s in the file

The data is carried two redundant ways, both sanitization-safe. Neither uses <script>, so both survive the inline-SVG sanitizers a host web app typically runs:

1. A JSON manifest in a <metadata id="diagram-manifest"> element.
2. data-node-* attributes on each node’s wrapping <g>.

A node’s id is the join key. It equals data-node-id="<id>" on the element, so a consumer can go from manifest to element and back without guessing.

Manifest schema

{
  "version": "1.0",
  "match": { "target": "fqProcessName", "type": "regex", "flags": "i" },
  "title": "nf-core/rnaseq",
  "width": 1829,
  "height": 724,
  "groups": [
    { "id": "star_salmon", "label": "STAR + Salmon", "color": "#e64949" }
  ],
  "regions": [{ "id": "preprocessing", "label": "Pre-processing" }],
  "nodes": [
    {
      "id": "fastqc",
      "label": "FastQC",
      "x": 120.0,
      "y": 80.0,
      "r": 5.0,
      "groups": ["star_salmon", "star_rsem"],
      "region": "preprocessing",
      "patterns": ["FASTQC", "MULTIQC"]
    }
  ]
}

• nodes are the addressable points, covering every node in the diagram. Unmapped ones carry an empty patterns list, so the manifest is a complete inventory rather than only the subset that lights up.
• id is the join key and equals data-node-id="<id>" on the element.
• Coordinate space. x/y/r are absolute SVG user units inside viewBox="0 0 width height", and the producer must emit no outer transform, so an overlay sharing that viewBox lines up exactly. r is a single nominal marker radius. Coordinates are rounded to one decimal place.
• groups and regions are optional metadata. A node references them by id through node.groups and node.region.
• Forward compatibility. Consumers MUST ignore unknown fields, and additive fields keep the same major version.

A machine-readable JSON Schema (draft 2020-12) ships with the package as nf_metro/manifest/schema.json, and manifest_schema() returns it as a dict. Its required fields are exactly the minimum-conforming set.

To validate an SVG, read its manifest out and check it against the schema. In Python, where jsonschema is not an nf-metro runtime dependency but pip install "nf-metro[validate]" adds it:

import jsonschema
from nf_metro.manifest import read_manifest, manifest_schema

manifest = read_manifest(open("pipeline.svg").read())
if manifest is None:
    raise SystemExit("no diagram manifest embedded in this SVG")
jsonschema.validate(manifest, manifest_schema())   # raises ValidationError if it doesn't conform

Or from the command line, without writing any code:

nf-metro validate-svg pipeline.svg
# Valid: 42 nodes, schema version 1.0   (exits non-zero if it doesn't conform)

validate-svg needs the same package, so pip install "nf-metro[validate]" covers it too.

Add --geometry to check the drawn picture as well as the schema. It flags a route drawn through a station’s label or marker, with rail interchanges excepted. The offset-collapse check, which catches distinct lines merging into one stroke, needs the engine’s assigned offsets, so it runs only via render --validate.

nf-metro validate-svg pipeline.svg --geometry

In another language, extract the <metadata id="diagram-manifest"> JSON the same way and feed it, with the shipped schema.json, to any standard JSON Schema validator.

Per-node attributes

<g
  data-node-id="fastqc"
  data-node-cx="120.0"
  data-node-cy="80.0"
  data-node-r="5.0"
  data-node-groups="star_salmon,star_rsem"
  data-node-region="preprocessing"
>
  ...the node's drawn glyph...
</g>

The geometry attributes mirror the manifest’s x/y/r, so a consumer can position against either half interchangeably. data-node-region is omitted when the node belongs to no region. A producer may add its own attributes or classes alongside these, and nf-metro tags the group nf-metro-station-group, but only the data-node-* set is part of the contract.

Matching semantics

patterns are regular expressions matched case-insensitively against a runtime target string. The match block names the target so that a consumer using neither Python nor Nextflow can reproduce the rule. For a Nextflow run the target is the fully-qualified process name, such as NFCORE_RNASEQ:RNASEQ:FASTQC. Another producer sets target to whatever identifier its own runtime emits.

Keep patterns within the regex subset common to Python re and JavaScript RegExp, so that two implementations cannot diverge. That subset covers character classes, anchors, ./*/+/?, bounded {m,n}, alternation and groups. Avoid Python-only constructs such as named groups (?P<>), inline flags (?i), possessive quantifiers and \Z.

A target may legitimately match more than one node. How to resolve that is a consumer-side policy decision rather than a schema error.

The minimum conforming file

The shortest path to a file a consumer can drive:

Required. An overlay positions itself from these alone:

• An SVG root with viewBox="0 0 width height" and no outer transform.
• Exactly one <metadata id="diagram-manifest"> holding the JSON, with at least version, width, height, and nodes - each node carrying an id and x/y/r.

Required only for matching, such as lighting up nodes from a running job:

• The match block (target/type/flags) and a patterns list on each node that represents something.

Recommended. This lets a consumer find and restyle the drawn node in place rather than only overlaying on top of it:

• Wrap each node’s glyph in a <g> with data-node-id="<id>" (matching the manifest id) and data-node-cx/-cy/-r.

Everything else is optional: label, groups, regions and the live state model below.

The toolkit functions

The whole toolkit is a handful of small functions, all importable from nf_metro.manifest and re-exported from nf_metro.render. Grouped by job:

Function	What it does
build_manifest_data(*, title, width, height, nodes, groups=(), regions=(), match_target="fqProcessName")	Assemble the manifest dict from plain node data. Rounds coordinates; fills the match block.
node_data_attrs(*, id, x, y, r, groups=(), region=None)	Return the data-node-* attributes for one node’s element, as a dict to spread onto your <g>.
manifest_metadata_svg(manifest)	Return just the <metadata> element (as a string) - use it when you assemble the SVG yourself.
inject_manifest(svg, manifest)	Insert that <metadata> into an existing SVG string, right after the opening <svg> tag. Returns the new SVG.
read_manifest(svg)	Parse the embedded manifest back out of an SVG string; returns the dict, or None if there’s no manifest.
match_node_ids(manifest, target)	Node ids whose patterns match target (case-insensitive) - “which node does this runtime name light up?”.
matching_node_ids(target, patterns_by_id)	The same matcher over a plain {id: [pattern]} map, when your data isn’t a full manifest.
overlay_svg(manifest, body="", *, extra_attrs="")	A transparent overlay <svg> sized to the manifest’s viewBox, to stack over the base so coordinates line up.
manifest_json(manifest)	Deterministic JSON serialization of a manifest (sorted keys); rarely needed directly.
manifest_schema()	Return the JSON Schema (draft 2020-12) for a manifest, to validate a producer’s output in any language.

Producing a file uses the first four. Consuming one uses read_manifest and match_node_ids, and a live overlay adds overlay_svg. The two constants MANIFEST_SCHEMA_VERSION and MANIFEST_ELEMENT_ID ("diagram-manifest") are exported too. The rest of this page shows them in context.

Produce a conforming SVG

In Python (any diagram, not just metro maps)

nf_metro.manifest builds a manifest from plain node data and embeds it into an SVG you drew by any means. It never needs a MetroGraph:

from nf_metro.manifest import (
    build_manifest_data, node_data_attrs, inject_manifest,
)

manifest = build_manifest_data(
    title="My Tool",
    width=100, height=100,
    nodes=[
        {"id": "trim", "x": 50, "y": 50, "r": 4, "patterns": ["TRIM.*"]},
    ],
)

# Decorate the node's element with the addressable mirror...
attrs = node_data_attrs(id="trim", x=50, y=50, r=4)
attr_str = " ".join(f'{k}="{v}"' for k, v in attrs.items())
svg = f'<svg viewBox="0 0 100 100"><g {attr_str}><circle cx="50" cy="50" r="4"/></g></svg>'

# ...and splice the manifest in after the opening <svg> tag.
svg = inject_manifest(svg, manifest)

Each nodes entry requires id, x, y and r, and optionally takes label (which defaults to id), groups, region and patterns. Coordinates are rounded for you. groups and regions are optional grouping metadata.

A node is addressed as a centre point plus a nominal radius, which is overlay-shaped rather than the full glyph outline. If your nodes are boxes, pass the box centre as x/y and a representative radius for r. An overlay only needs somewhere to anchor, not your exact geometry.

If your runtime doesn’t emit Nextflow process names, set match_target to the identifier it does emit, so the file honestly describes what its patterns match:

build_manifest_data(..., match_target="stepName")
# -> "match": { "target": "stepName", "type": "regex", "flags": "i" }

In any language

You don’t need this library to produce a conforming file. Emit the bytes directly:

1. Draw your SVG with viewBox="0 0 width height" and no outer transform.
2. Insert a <metadata id="diagram-manifest"> element holding the JSON above as CDATA. CDATA cannot contain ]]>, so if a regex does, split it as ]]]]><![CDATA[>.
3. (Recommended) For each node, wrap its glyph in a <g> carrying a stable data-node-id and its centre and radius as data-node-cx/-cy/-r, to one decimal place. Keep this geometry in agreement with the manifest, since id is the join key between them.

Read and match

nf_metro.render re-exports the canonical reader and matcher, which are also available from nf_metro.manifest:

from nf_metro.render import read_manifest, match_node_ids

manifest = read_manifest(open("pipeline.svg").read())
match_node_ids(manifest, "NFCORE_RNASEQ:RNASEQ:FASTQC")   # -> ["fastqc"]

read_manifest is a plain regex extract and needs no XML library. A consumer in another language reproduces the matcher by walking nodes[].patterns, testing each regex case-insensitively against the target, and collecting the ids that hit.

match_node_ids takes a whole manifest, keyed on the schema’s nodes. matching_node_ids is the same matcher over a plain id -> [pattern] mapping, for a producer whose data isn’t manifest-shaped.

Drive a live overlay: the state snapshot

Everything above defines the static compatibility contract of geometry and addressing. This section defines the second, optional half: the runtime state vocabulary that a progress overlay is driven from. A consumer that only needs static addressing can stop at the manifest and skip this section.

Worked example

The tutorial at the end of this page builds a small pipeline diagram and lights it up from a progress snapshot in about 40 lines.

The manifest gives an overlay everything it needs without a re-render: the viewBox to share, and each node’s id, centre and radius. The standard fixes the geometry, the addressing and the state vocabulary below, but not the visual style. How you draw “running” against “done”, whether as a halo, a badge or a colour change on the node itself, is up to you.

How the pieces fit together

Three layers, each narrower than the last:

1. The standard. This manifest schema plus the state snapshot schema below. It is tool-neutral, and any producer can emit it.
2. A binding. This turns one runtime’s native events into the state snapshot. nf-metro ships one binding for Nextflow: the -with-weblog HTTP receiver (nf-metro serve) and, optionally, the nf-metro Nextflow plugin that wires it up from pipeline config. A binding for another workflow engine or CI system would translate that system’s own events into the same snapshot shape instead.
3. nf-metro itself. One producer of conforming SVGs, and the author of the binding above. A consumer that already owns authoritative task state, such as a platform that tracks a run’s tasks directly, needs only layer 1, meaning the manifest and this state vocabulary, and neither the weblog server nor the plugin.

The snapshot shape

A snapshot is the full progress picture at one instant: every node’s display state, plus the run’s own lifecycle state. nf-metro’s live server serves one at GET /state, and pushes a fresh one as the data: payload of every Server-Sent Event on GET /stream:

{
  "run": { "name": "grave_babbage", "state": "running" },
  "stations": {
    "trim": { "state": "done", "done": 2, "total": 2 },
    "qc": { "state": "running", "done": 0, "total": 1 }
  }
}

• stations is keyed by node id, the same id a manifest node and data-node-id carry, so a consumer joins the two without any translation. A node the runtime hasn’t reported on yet is absent from a hand-built snapshot. In nf-metro’s own server it is present as {state: "pending", done: 0, total: 0}, because the server pre-populates every mapped node so a fresh subscriber never sees a missing key.
• run is one lifecycle value for the whole snapshot, not one per node.

A machine-readable JSON Schema (draft 2020-12) ships with the package as nf_metro/live/state_schema.json, and state_schema() returns it as a dict, mirroring manifest_schema():

import jsonschema
from nf_metro.live import state_schema

jsonschema.validate(snapshot, state_schema())   # raises ValidationError if it doesn't conform

Unlike the manifest, no version field travels inside the snapshot itself, because it is a live response rather than a durable artifact committed to a repo. The schema file is versioned on disk as STATE_SCHEMA_VERSION, currently "1.0", and evolves under the same forward-compatibility rule as the manifest, given below.

The state enum and its transitions

Each station’s state is one of:

State	Meaning
pending	No task has been submitted for this node yet. The initial value.
queued	At least one task has been submitted; none of them is running yet. done may already be greater than 0 if an earlier task for the same node finished before this one was submitted.
running	At least one task for this node is currently running.
done	Every task submitted for this node reached a terminal status, and none of them failed.
failed	Sticky - once any task for this node has failed, it reports failed for the rest of the run, even if other tasks for the same node are later running or complete.

When several conditions hold at once, a producer must apply the precedence failed > running > queued > done > pending, checked in that order. A node with one failed task and another still running therefore reads failed, never running.

done / total semantics

• total is the count of tasks submitted so far for this node in the current run, not a fixed denominator. A workflow engine’s task count for a node is dynamic, for instance when scattering over samples, so total only ever reflects what has been seen so far and can still grow after the node starts reporting running.
• done is the count of tasks that have reached a successful terminal status for this node so far. A failed task is not counted in done.
• Both are cumulative across the run and reset only by a fresh started event, described below. They never decrease mid-run.

Run lifecycle

The run object’s state is one lifecycle shared by the whole snapshot:

State	Meaning
idle	No run has reported in yet. The initial value; name is null.
running	A run announced itself has started, and no terminal event has followed yet.
complete	The run finished successfully.
error	The run (or the binding reporting it) signalled a failure.

Only a fresh started-equivalent event moves a run off complete or error, and doing so resets every station back to pending/0/0. Re-running the same pipeline therefore re-animates a clean map rather than layering a new run’s progress on top of the last one’s leftovers.

Forward compatibility

The rule is the same as for the manifest: consumers MUST ignore unknown fields, so a producer can add fields to run, to a station entry, or a wholly new top-level key without breaking an existing reader. A change that is not purely additive, such as removing a field, changing an enum’s existing members or changing done/total semantics, is a breaking change. It bumps the schema file’s own version and nf-metro’s major version together, exactly as for the manifest.

Bring your own binding

nf-metro’s serve is one reference implementation. It turns Nextflow’s -with-weblog task events into the snapshot above, mapping process_submitted to queued, process_started to running, and process_completed to done or failed depending on status. It then draws a glowing halo per node and recolours it by state (see Live progress). A host application is free to write its own binding from its own runtime’s events to this same snapshot shape, and to map the state vocabulary onto its own visual language, whether that is filled badges, a progress bar or a colour change on the node itself. Take the geometry from the manifest and the state vocabulary from this section, then bring your own events and your own paint.

Tutorial: light up a diagram as a job runs

A complete, self-contained example for a tool that is not nf-metro. By the end you will have a small pipeline diagram that shows progress as work happens. Every snippet runs as written, with no pipeline, no server and no Nextflow, because the progress is faked. It comes to about 50 lines and uses only nf_metro.manifest.

The idea. Draw the diagram once and embed a manifest in it. Then, whenever progress changes, draw a thin overlay of status markers on top. The diagram itself never re-flows, and only the lightweight overlay updates. The base SVG is the map, drawn once and durable, and the overlay is a cheap, disposable status layer you redraw as things change. The example models a three-step pipeline: Fetch → Align → Report.

What’s doing the work. The only library is nf_metro.manifest, the standard-library-only module described above. There is no MetroGraph, no nf-metro renderer and no drawing or templating library. We assemble the SVG as plain Python strings and use nf_metro.manifest for four manifest-specific jobs: building it (build_manifest_data), embedding it (node_data_attrs, inject_manifest), reading it back (read_manifest), and matching runtime names to nodes (match_node_ids).

Step 1 - draw the diagram and embed a manifest

We hand-draw three circles, one per step, wrap each in a <g> carrying its data-node-* attributes, and splice in the manifest. There is no need to absorb every field yet. The only new ideas are that each node needs coordinates, and that the manifest gets embedded into an otherwise ordinary SVG.

• For now, the fields that matter are id and x/y/r: the node’s name, where it sits and how big it is. An overlay anchors to these.
• Later, patterns (the names this node answers to) and match_target come into play for Step 2’s matching, so ignore them until then. This example matches against step names, so match_target="stepName".

from nf_metro.manifest import (
    build_manifest_data, node_data_attrs, inject_manifest,
    read_manifest, match_node_ids,
)

NODES = [
    {"id": "fetch",  "label": "Fetch",  "x": 70,  "y": 42, "r": 13, "patterns": ["FETCH"]},
    {"id": "align",  "label": "Align",  "x": 180, "y": 42, "r": 13, "patterns": ["BWA.*", "STAR.*"]},
    {"id": "report", "label": "Report", "x": 290, "y": 42, "r": 13, "patterns": ["MULTIQC"]},
]
W, H = 360, 92

def node_svg(n):
    attrs = " ".join(
        f'{k}="{v}"'
        for k, v in node_data_attrs(id=n["id"], x=n["x"], y=n["y"], r=n["r"]).items()
    )
    return (
        f'<g {attrs}>'
        f'<circle cx="{n["x"]}" cy="{n["y"]}" r="{n["r"]}" fill="#dfe3ee" stroke="#333"/>'
        f'<text x="{n["x"]}" y="{n["y"] + 30}" text-anchor="middle" font-size="13">{n["label"]}</text>'
        f'</g>'
    )

edges = (
    '<line x1="83" y1="42" x2="167" y2="42" stroke="#aaa"/>'
    '<line x1="193" y1="42" x2="277" y2="42" stroke="#aaa"/>'
)
base = (
    f'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 {W} {H}" width="{W}" height="{H}">'
    f'{edges}{"".join(node_svg(n) for n in NODES)}</svg>'
)
svg = inject_manifest(
    base,
    build_manifest_data(
        title="Toy pipeline", width=W, height=H, nodes=NODES, match_target="stepName"
    ),
)

Three functions did the work. node_data_attrs produced each node’s data-node-* attributes, build_manifest_data assembled the manifest from the node list, and inject_manifest placed that manifest inside the SVG. svg is now a self-describing file: three labelled nodes, a <metadata id="diagram-manifest"> block and data-node-* attributes. Save it to a .svg if you like, since everything below works from that file alone.

Step 2 - connect the diagram to the work

Something has to actually run your pipeline’s steps, whether that is a workflow engine, a CI job or a plain script. Call it the runtime. As it works, it announces each step by a name: it might log that a step called BWA_MEM has started, then that it finished, and so on.

Those names cause two problems. You usually don’t choose them, since a tool may call your “Align” step BWA_MEM or STAR_ALIGN, and they rarely equal your node ids. Each node’s patterns exist for exactly this: regexes that match the names your runtime uses. match_node_ids answers the question “which node does this name belong to?”:

manifest = read_manifest(svg)

match_node_ids(manifest, "BWA_MEM")   # -> ['align']
match_node_ids(manifest, "multiqc")   # -> ['report']  (matching is case-insensitive)

Nothing is running yet, and this only queries the file. Matching is the bridge from “a name the runtime mentioned” to “a node on the diagram”.

Step 3 - show progress

Give each node a state, one of pending, queued, running, done or failed, and draw a coloured ring per node at its manifest position. The colours are your choice, since the standard only tells you where each node is:

COLORS = {
    "pending": "#b8c0d0", "queued": "#ffb020",
    "running": "#ffc23a", "done": "#2bee92", "failed": "#ff4d4d",
}

def progress_halos(manifest, states):
    """One status ring per node, positioned from the manifest geometry."""
    return "".join(
        f'<circle cx="{n["x"]}" cy="{n["y"]}" r="{n["r"] + 5}" fill="none" '
        f'stroke="{COLORS[states.get(n["id"], "pending")]}" stroke-width="3.5"/>'
        for n in manifest["nodes"]
    )

A tutorial has no real runtime, so simulate one. The list below holds (step_name, new_state) announcements of the kind a real engine sends as a run progresses. Fold each one into a {node_id: state} map using Step 2’s matcher, then redraw the overlay. That sequence of redraws is the animation:

# A real runtime would send these live; we hard-code them so the tutorial runs
# on its own.
events = [
    ("FETCH",   "running"), ("FETCH",   "done"),
    ("BWA_MEM", "running"), ("BWA_MEM", "done"),   # the Align step, by its tool name
    ("MULTIQC", "running"), ("MULTIQC", "done"),   # the Report step
]

states = {}
for name, new_state in events:
    for node_id in match_node_ids(manifest, name):
        states[node_id] = new_state
    frame = svg.replace("</svg>", progress_halos(manifest, states) + "</svg>")
    # draw `frame`: write it to a file, or update the page in a browser

A single frame taken just after Fetch finished and Align started, when states is {"fetch": "done", "align": "running"}, looks like this. Green is done, amber is running and grey is still pending:

Replaying the whole events list redraws the overlay step by step, which animates the run from start to finish:

The rings are deliberately plain. Swap in pulses, fills, per-node counts or your own palette without touching the contract.

Step 4 - plug in a real runtime

Up to now everything ran in one Python script. In a real system the same logic splits between an event source, whatever runs your pipeline, and a UI, usually a browser. Only one thing in this tutorial was fake: the hard-coded events list. Replace it with announcements from a real run and nothing else changes. You still match_node_ids each name to a node (Step 2) and fold it into the states map that progress_halos draws (Step 3).

What Nextflow does, in the tutorial’s terms. Run a pipeline with nextflow run ... -with-weblog http://localhost:8080/events and Nextflow becomes the source of that events list. Every time a task is submitted, starts or finishes, it POSTs a small JSON message to that URL carrying the process name and its status. It sends you ("BWA_MEM", "running"), then ("BWA_MEM", "done"), live, instead of you writing them out.

What nf-metro serve is. This same tutorial running as a small web server, so you write none of the Python yourself:

1. it renders the diagram’s SVG once and builds an overlay of one ring per node, positioned from each node’s coordinates, on the same principle as progress_halos;
2. it listens on http://localhost:8080/, where /events is the URL Nextflow POSTs to;
3. on each message it runs Step 2 (match_node_ids on the process name) and Step 3 (fold the result into a per-node states map);
4. it pushes the updated states to the open browser page over Server-Sent Events, and the page recolours the matching overlay ring.

nf-metro serve is therefore this tutorial wired to a live event source and a browser. See Live progress to run it, which also covers the multi-run dashboard and the optional Nextflow plugin. The glowing-LED styling there is its own choice, and yours can differ.

Doing it yourself in a browser takes the same three steps client-side: read_manifest on the committed SVG, match_node_ids per incoming event, then restyle the matched node. Keep the overlay as a separate layer over the base so you never redraw the diagram. overlay_svg builds one sized to match, so coordinates line up:

from nf_metro.manifest import overlay_svg

# a transparent layer the same size/viewBox as the base, holding the rings:
layer = overlay_svg(manifest, progress_halos(manifest, states),
                    extra_attrs='style="pointer-events:none"')
# stack `layer` directly over the base SVG; on each event, update its rings.

The complete script

Everything above as one file. It needs only nf-metro installed (pip install nf-metro), and it writes the diagram plus one frame per event, with no pipeline or server. Save it as demo.py, run python demo.py, then open toy_pipeline.svg and the progress_*.svg frames in order. You should get one static diagram plus six frames that turn Fetch, then Align, then Report from grey through amber to green, with the terminal printing each event as it maps to a node.

"""Make a conforming SVG and drive it from a stream of (step, state) events.

Uses only nf_metro.manifest (Python standard library only) - no pipeline, no
server. Run `python demo.py`, then open toy_pipeline.svg and progress_*.svg.
"""

from nf_metro.manifest import (
    build_manifest_data, node_data_attrs, inject_manifest,
    read_manifest, match_node_ids,
)

# --- the diagram: one node per pipeline step --------------------------------
NODES = [
    {"id": "fetch",  "label": "Fetch",  "x": 70,  "y": 42, "r": 13, "patterns": ["FETCH"]},
    {"id": "align",  "label": "Align",  "x": 180, "y": 42, "r": 13, "patterns": ["BWA.*", "STAR.*"]},
    {"id": "report", "label": "Report", "x": 290, "y": 42, "r": 13, "patterns": ["MULTIQC"]},
]
W, H = 360, 92

def node_svg(n):
    attrs = " ".join(
        f'{k}="{v}"'
        for k, v in node_data_attrs(id=n["id"], x=n["x"], y=n["y"], r=n["r"]).items()
    )
    return (
        f'<g {attrs}>'
        f'<circle cx="{n["x"]}" cy="{n["y"]}" r="{n["r"]}" fill="#dfe3ee" stroke="#333"/>'
        f'<text x="{n["x"]}" y="{n["y"] + 30}" text-anchor="middle" font-size="13">{n["label"]}</text>'
        f'</g>'
    )

edges = (
    '<line x1="83" y1="42" x2="167" y2="42" stroke="#aaa"/>'
    '<line x1="193" y1="42" x2="277" y2="42" stroke="#aaa"/>'
)
base = (
    f'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 {W} {H}" width="{W}" height="{H}">'
    f'{edges}{"".join(node_svg(n) for n in NODES)}</svg>'
)
svg = inject_manifest(
    base,
    build_manifest_data(
        title="Toy pipeline", width=W, height=H, nodes=NODES, match_target="stepName"
    ),
)
with open("toy_pipeline.svg", "w") as f:
    f.write(svg)
print("wrote toy_pipeline.svg  (the conforming diagram)")

# --- drive it from a stream of events ---------------------------------------
COLORS = {
    "pending": "#b8c0d0", "queued": "#ffb020",
    "running": "#ffc23a", "done": "#2bee92", "failed": "#ff4d4d",
}

def progress_halos(manifest, states):
    return "".join(
        f'<circle cx="{n["x"]}" cy="{n["y"]}" r="{n["r"] + 5}" fill="none" '
        f'stroke="{COLORS[states.get(n["id"], "pending")]}" stroke-width="3.5"/>'
        for n in manifest["nodes"]
    )

# In a real run these arrive live (e.g. from Nextflow's -with-weblog); here we
# hard-code them so the demo runs on its own.
events = [
    ("FETCH",   "running"), ("FETCH",   "done"),
    ("BWA_MEM", "running"), ("BWA_MEM", "done"),   # the Align step, by its tool name
    ("MULTIQC", "running"), ("MULTIQC", "done"),   # the Report step
]

manifest = read_manifest(svg)
states = {}
for i, (name, state) in enumerate(events):
    hits = match_node_ids(manifest, name)        # which node(s) does this name light up?
    for node_id in hits:
        states[node_id] = state
    frame = svg.replace("</svg>", progress_halos(manifest, states) + "</svg>")
    with open(f"progress_{i}.svg", "w") as f:
        f.write(frame)
    print(f"  event {name:<8} {state:<8} -> {hits}")

print(f"wrote progress_0.svg .. progress_{len(events) - 1}.svg  (open them in order)")

Swap the hard-coded events for messages from your real runtime, or let nf-metro serve do it as in Step 4, and the same loop drives a live diagram.