Reference architecture · Replicated state

PCIe Reflective Memory

Replicate application-defined memory regions across independent nodes using SISCI broadcast segments and supported PCIe multicast. Regions commonly use system memory and can also use compatible PCIe device memory.

State location System or supported device memory
Distribution Hardware PCIe multicast
Programming model SISCI broadcast segments and events
Consistency Defined by the application
Reference topology Multicast shared state
Source node Published state region One update enters the multicast group
Node B Reflected region Receives the replicated update
Node C Reflected region Receives the replicated update
Node D Reflected region Receives the replicated update
Control plane PCIe data plane
Operating model

How the architecture works

The sequence describes system behaviour rather than a product feature list.

Publish

A CPU, Dolphin DMA engine or supported PCIe master writes or transfers data to a SISCI broadcast segment configured for reflective-memory operation.

Replicate

A supported multicast-capable PCIe fabric forwards the transaction to the configured destination regions.

Consume

Receiving applications may use SISCI events or data interrupts together with sequence information and application-defined consistency rules to process the new state.

System definition

Reference topology and architectural boundaries

Ownership, data movement, software responsibility and the limits of the pattern are defined separately.

System-level view Multicast shared state
Source node Published state region One update enters the multicast group
Node B Reflected region Receives the replicated update
Node C Reflected region Receives the replicated update
Node D Reflected region Receives the replicated update
Qualification required Application-aware design
Memory model

Application-defined memory regions

Reflective regions commonly reside in system memory; supported configurations can also use PCIe device memory such as FPGA or GPU buffers.

Distribution model

One-to-many multicast

A source update can be delivered to several participating nodes.

Synchronization model

Producer and consistency model

Multiple nodes can write, but producer ownership, ordering, atomicity, conflict handling and stale-data detection remain application responsibilities.

Topology limit

Hardware and release dependent

Multicast group count, memory size and supported topologies must be confirmed for the selected platform.

Engineering criteria

Design decisions that determine whether the architecture will work

These items must be resolved for the actual hosts, endpoints, operating systems, topology and workload.

State model
Define region size, record layout, producer/consumer ownership, update granularity and versioning.
Consistency rules
Specify ordering, sequence counters, atomic update boundaries, stale-data detection and recovery after a missed event.
Distribution pattern
Confirm one or several producers, multicast groups, target nodes and whether device-originated transfers are needed.
Timing behaviour
Measure update latency and jitter under simultaneous traffic; PCIe multicast alone does not create application determinism.
Recovery
Define node join, restart, initial synchronization, link-loss handling and re-establishment of a consistent state image.
Implementation layers

Building blocks used to realize the architecture

The final system combines compatible hardware, software, application logic and validation—not one standalone product.

Building block

NTB-capable nodes

Independent hosts connected through compatible PCIe networking hardware.

Building block

Multicast-capable fabric

Adapters and switch topology that support the required multicast mode.

Building block

SISCI software

Broadcast-segment setup, PIO or DMA transfer, event and multicast interfaces.

Building block

State protocol

Data layout, sequence control, producer rules and recovery behaviour.

Building block

Health monitoring

Node state, missed updates, sequence gaps and fabric diagnostics.

Building block

Qualification workload

Representative region sizes, update rates, competing traffic and restart scenarios.

Selection boundary

Where this architecture fits

Use the pattern when its ownership and data-movement model match the engineering requirement.

Deployment fit

Use this architecture

Several nodes need the same low-latency operational state with one-to-many distribution.

Architecture boundary

Use another architecture

The requirement is arbitrary message exchange, transactional storage replication or simultaneous writes without an application consistency model.

Deployment fit

Typical deployments

Simulation state, test coordination, sensor status, control variables and distributed event information.

Project definition

Inputs required before system configuration

Architecture selection starts with the actual platform, traffic, software and recovery requirements.

State definition Data structures, region size, producer count and update granularity.
Distribution scope Node count, multicast groups and traffic direction.
Consistency requirements Ordering, atomicity, freshness, timeout and stale-data policy.
Recovery behaviour Initial synchronization, restart, reconnection and data revalidation.

Define the topology before selecting the components.

Primionics can review the root-complex model, endpoint inventory, lane and bandwidth budget, software path, operating-system support and qualification requirements for the complete PCIe system.

Discuss the system