Reference architecture · Single-root I/O

Transparent PCIe Expansion

Extend native PCIe devices beyond the host enclosure while keeping one root complex, one PCIe hierarchy and the endpoint’s supported driver model.

Root domain One host owns the endpoints
Data path Native MMIO and DMA
Interconnect Transparent host, target, cable and optional switch
Best fit Remote slots and external accelerators
Reference topology Single-root expansion
Host system Root complex Enumerates and owns every endpoint
Expansion fabric Target adapter / switch Extends the hierarchy outside the host
Remote endpoints GPU · FPGA · NVMe · DAQ Use their supported operating-system drivers
Control plane PCIe data plane
Operating model

How the architecture works

The sequence describes system behaviour rather than a product feature list.

Enumerate

The host firmware and operating system enumerate compatible remote endpoints within one PCIe hierarchy; available bus numbers and MMIO/BAR resources must be sufficient for the complete endpoint set.

Operate

Applications use the endpoint’s supported native driver while configuration access, MMIO, interrupts and DMA traverse the transparent PCIe path.

Validate

The complete path is checked for resource allocation, reset and power sequencing, boot order, cooling, error handling and sustained data movement.

System definition

Reference topology and architectural boundaries

Ownership, data movement, software responsibility and the limits of the pattern are defined separately.

System-level view Single-root expansion
Host system Root complex Enumerates and owns every endpoint
Expansion fabric Target adapter / switch Extends the hierarchy outside the host
Remote endpoints GPU · FPGA · NVMe · DAQ Use their supported operating-system drivers
Qualification required Application-aware design
Ownership model

Single host

All downstream devices belong to one root complex.

Software model

Native endpoint drivers

No application-level network transport is introduced by the expansion link; device support still depends on the endpoint, driver, operating system and complete topology.

Data plane

PCIe transactions

Configuration transactions, MMIO, interrupts and DMA remain native PCIe operations.

Boundary

One PCIe hierarchy

Independent hosts require an NTB-based architecture instead.

Engineering criteria

Design decisions that determine whether the architecture will work

These items must be resolved for the actual hosts, endpoints, operating systems, topology and workload.

Resource allocation
Confirm BAR and MMIO requirements, 64-bit prefetchable space, firmware allocation and operating-system limits for the complete endpoint set.
Link engineering
Define PCIe generation, lane width, cable type, reach, switch hops and the acceptable bandwidth margin.
Start-up and reset
Validate host/target power sequence, fundamental reset, secondary-bus reset, endpoint recovery and any model- and operating-system-specific hot-add or hot-remove behaviour.
Platform controls
Review ACS, IOMMU, interrupt routing and peer-to-peer requirements across the root complex, switches and endpoints; these controls can change visibility and transaction routing.
Mechanical and power
Size chassis slots, auxiliary power, cooling, cable routing and service access around the actual cards.
Implementation layers

Building blocks used to realize the architecture

The final system combines compatible hardware, software, application logic and validation—not one standalone product.

Building block

Host interface

Transparent host adapter, or a dual-mode adapter explicitly configured for transparent operation.

Building block

External path

Compatible copper or optical PCIe link with the required lane width and generation.

Building block

Expansion endpoint

Compatible target adapter, system-slot module, external switch, backplane or expansion chassis.

Building block

Device layer

Qualified GPU, FPGA, NVMe, acquisition or other PCIe endpoints.

Building block

Host software

Firmware, operating system and native endpoint drivers.

Building block

Validation tools

Topology, link, error, reset and sustained-transfer diagnostics.

Selection boundary

Where this architecture fits

Use the pattern when its ownership and data-movement model match the engineering requirement.

Deployment fit

Use this architecture

One host needs more slots, remote I/O or physically separated accelerators without changing the endpoint software model.

Architecture boundary

Use another architecture

Several independent computers must communicate, share state or take controlled ownership of a device.

Deployment fit

Typical deployments

External GPU/FPGA chassis, high-rate acquisition, storage expansion and remote instrumentation.

Project definition

Inputs required before system configuration

Architecture selection starts with the actual platform, traffic, software and recovery requirements.

Endpoint inventory Card type, quantity, BAR size, lane width, generation, power and cooling.
Host platform Root-port capability, firmware resource allocation, operating system and IOMMU policy.
Physical deployment Distance, cable route, chassis format, environment and serviceability.
Operational behaviour Boot order, reset, hot-add, recovery and availability expectations.

Define the topology before selecting the components.

Primionics can review the root-complex model, endpoint inventory, lane and bandwidth budget, software path, operating-system support and qualification requirements for the complete PCIe system.

Discuss the system