backends — the allocator seam (L0)

In one paragraph

tr::mem::mem_backend_t is a small, user-implementable interface: subclass it to bind libtracer to any memory — a heap, a fixed arena, live registers, a DMA ring. libtracer never allocates payload bytes on its own; it asks a backend. (Its own bookkeeping — a rope’s spilled link chain, the CAN splitter’s window vector, the route-handle label tables — allocates from the global heap or from an injected std::pmr::memory_resource; those seams are failable allocation and backpressure.) Three backends are provided: mem_heap (owns malloc’d bytes), mem_borrowed (wraps your bytes, frees nothing), and mem_pool (a bounded fixed-slab, alloc-or-null).

What it does

The protocol treats application data as opaque, and mem_backend_t extends that to the memory plane: libtracer is a transparent byte router (ADR-0012 — memory binding is a modular spectrum). A backend declares its own per-architecture contract (alignment, cache hooks, ISR-safety) and owns reclamation; the layers above see only segment_ts. The interface deliberately makes allocation optional (alloc may return nullptr) because many substrates — MMIO, hardware FIFOs — cannot allocate at all.

Backend	Owns	destroy does	Use
mem_heap	malloc’d bytes	frees bytes + control block	hosted targets
mem_borrowed	nothing (your bytes)	frees only the control block	live/raw, MMIO header, ROM
mem_pool	a caller slab	returns the slot to a free list	bounded / MCU / deterministic

A fourth, mem_cuda, is not part of core at all: it is a tier module under backends/cuda/, its own CMake project that consumes core (backends/cuda/CMakeLists.txt:add_library(libtracer_cuda STATIC src/mem_cuda.cpp)). It needs the CUDA toolkit and a GPU, so it is never built in CI. Core carries no vendor name and no #ifdef for it — a DEVICE-space backend plugs in by registering a transfer hook (below).

mem_pool is the bounded “custom allocator”: it carves a caller-owned slab into fixed slots with the free list threaded through the slab (no auxiliary heap), and returns nullptr when full — the BACKPRESSURE signal. pool_t is not synchronized; synchronized_pool_t<Sync> (core/include/libtracer/mem_pool.hpp:synchronized_pool_t) composes over it and guards the free list with a compile-time guard, which is what any shared seam needs — a segment self-routes its reclaim on whatever thread drops the last reference, concurrent with a writer’s alloc. Since RFC-0028 slice 10 the guard is the SAME tr::guard trait the last-known-value slot uses, and synchronized_pool_t<> binds the build’s one tr::graph::guard_t: the host tr::mutex_guard_t (a bounded spin, then a nap) or, on ESP-IDF, the interrupt-masked tr::esp::critical_guard_t (integrations/esp-idf/libtracer/include/libtracer_esp/critical_guard.hpp; the pool spelling is tr::esp::critical_pool_t — it needs FreeRTOS headers, so it ships with the ESP-IDF component rather than in core/). The pool is opt-in; no seam defaults to it.

Each concrete backend also carries four compile-time traits the module set reads without a virtual call — needs_cache_ops, is_isr_safe, is_nonblocking, owns_bytes (ADR-0047 — build-time closed module sets §2). owns_bytes is the one a caller must respect: mem_borrowed sets it false, so a segment it produced must not be stored durably.

The seam lives at L0 (tr::mem); the segments it produces are owned at L1 (tr::view). A backend constructs and reclaims tr::view::segment_t — the one sanctioned L0↔L1 boundary type, and the only tr::view symbol the L0 interface is permitted to name (ADR-0016 — substrate, zero-copy, layer namespaces §2). alloc returns a raw segment_t* with a refcount of 1; the caller adopts it with tr::view::segment_ptr_t::adopt (core/include/libtracer/segment.hpp:segment_ptr_t::adopt). The handle-producing conveniences heap_alloc / borrow / borrow_const therefore live in tr::view, not here.

The backend module set is a per-target compile-time type list with tag dispatch: a single-member set folds to direct calls, and a multi-member set keeps mixed backends coexisting in one rope (for example heap and GPU).

Interface

class mem_backend_t : public tr::mem::block_source_t

A memory backend: a block_source_t that also vends refcounted segments — the L0 seam libtracer binds any substrate behind (RFC-0028 §4.9, D9).

A backend is a block source (RFC-0028 slice 10): the raw failable block (block_source_t::try_alloc / block_source_t::release) is the one allocation seam, and a backend is that seam plus a refcounted segment and the space / cache hooks device memory needs (ADR-0024). The default alloc draws ONE block through try_alloc and places the view::segment_t header at its head, the payload after it; the default destroy hands that one block back through a sized release. So a backend that implements the two block virtuals gets segments for free, and a deployer that injects one slab has one slab — every injection point that takes a block_source_t& accepts a backend.

Subclass this to bind libtracer to any allocator — a heap, a fixed caller-owned arena, live registers, lwIP pbufs, DMA descriptors. Allocation stays optional: many substrates cannot allocate (MMIO, hardware FIFOs), and the default try_alloc refuses, so the default alloc returns nullptr for them. A substrate whose segment does not live in one block (a borrowed span, device memory with a host-side header) overrides alloc and destroy directly.

Note

Each backend declares its own concurrency/ISR-safety contract; the protocol mandates none (docs/adr/0012).

Subclassed by tr::graph::inline_value_backend_t, tr::mem::detail::borrowed_backend_t, tr::mem::detail::borrowed_device_backend_t, tr::mem::heap_backend_t, tr::mem::pool_t, tr::mem::source_backend_t, tr::mem::synchronized_pool_t< Sync >

Public Functions

inline explicit mem_backend_t(const char *name) noexcept

Construct a backend with a stable, human-readable name (e.g. “mem_heap”).

inline virtual void *try_alloc(std::size_t bytes, std::size_t align = alignof(std::max_align_t)) noexcept override

Obtain one raw block — the block_source_t half of a backend.

The default refuses, which is right for an allocation-incapable substrate (a borrowed span, MMIO, a hardware FIFO): its alloc then refuses too.

Return values:

nullptr – Exhaustion, or a substrate that cannot allocate.

inline virtual void release(void *p, std::size_t bytes, std::size_t align = alignof(std::max_align_t)) noexcept override

Return a block try_alloc handed out. The default has nothing to return.

inline virtual view::segment_t *alloc(std::size_t size, alloc_hint_t hint = alloc_hint_t::NONE)

Allocate a fresh segment of at least size bytes (refcount = 1).

The default segment: one block through mem_backend_t::try_alloc.

The returned segment is the caller’s to adopt via tr::view::segment_ptr_t::adopt. A raw segment_t* is returned, not a segment_ptr_t, to keep L0 from naming L1’s owning handle (docs/adr/0016 §2).

The default is ONE block: try_alloc for the header padded to alignment plus size, the header placed at the block’s head (placement.hpp). One allocation where the pre-slice-10 heap backend made two.

Parameters:

hint – Backend-private allocation hint; NONE for “don’t care”.

Return values:

nullptr – Backpressure (pool exhausted / OOM) or allocation unsupported.

inline virtual void destroy(view::segment_t *seg) noexcept

Reclaim a segment whose refcount has reached zero (the only reclaim path).

The default reclaim: one sized release of the block mem_backend_t::alloc drew.

Frees whatever the backend owns (the bytes and/or the segment_t control block) and nothing it does not — a borrowed backend never frees the user’s bytes. Invoked by segment_ptr_t at zero, never by user code. The default is the mirror of the default alloc(): one sized release of the whole block.

Warning

Never called on a live segment.

inline virtual void before_io(view::segment_t*, io_dir_t) noexcept

Cache prep before handing the segment to a DMA transfer.

Clean or invalidate per dir so the device sees coherent memory. No-op by default and on cacheless cores (Cortex-M0/M3/M4); only DMA-class backends override it (docs/reference/09 §cache coherency).

inline virtual void after_io(view::segment_t*, io_dir_t) noexcept

Cache reconcile after a DMA transfer completes.

Invalidate per dir so the next CPU reader sees HW’s writes. No-op by default and on cacheless cores.

inline virtual std::size_t alignment() const noexcept

The alignment (bytes) this backend guarantees for allocated bytes.

inline virtual std::size_t max_segment_size() const noexcept

The largest single segment this backend can produce.

inline virtual mem_space_t space() const noexcept

The address space this backend’s segments live in (default HOST).

A DEVICE backend (one from the backends/ tier) must override this; segments inherit it (segment.hpp), and the codec uses it to skip CPU access to device links.

inline virtual backend_tag tag() const noexcept

The build-time-closed module-set tag (default UNKNOWN, ADR-0047 §2).

A backend that participates in the fast destroy dispatch overrides this to return its backend_tag; segments read it once at construction (like space). A backend that leaves the default is dispatched through its virtual destroy.

The DMA/allocation enums the seam uses:

enum class tr::mem::io_dir_t : std::uint8_t

Direction of a DMA / cache-coherency transfer, for the cache hooks.

The hook method carries the timing (before/after the transfer); this enum carries the direction; the backend maps the pair to clean/invalidate.

Values:

enumerator DEVICE_TO_CPU

After DMA-in: invalidate so the CPU reads HW’s writes.

enumerator CPU_TO_DEVICE

Before DMA-out: clean so HW reads the CPU’s writes.

enum class tr::mem::alloc_hint_t : std::uint32_t

Opaque, backend-private allocation hint.

A hint’s meaning is private to the backend that defines it: there is no cross-backend hint registry, no two backends share a value’s meaning, and a hint-ignoring backend accepts any value (docs/adr/0016 §”Considered options”). This strong typedef also stops a hint being swapped for a size argument.

Values:

enumerator NONE

“Don’t care” — the default for every alloc call.

The failable-block seam — block_source_t

The second L0 seam, and a distinct one (ADR-0065 — failable allocation gets its own seam, reference/09). mem_backend_t above vends refcounted segments for payload bytes; this one vends raw single-owner blocks and reports exhaustion by value, because std::pmr::memory_resource structurally cannot — its allocate signals failure only by throwing, and on a -fno-exceptions target that lowers to the toolchain’s abort() stub, which a peer can provoke.

The policy the seam exists to serve — which allocations a peer can reach, which status each exhaustion answers with, and how to size a bounded source — is described in failable allocation and backpressure. This page documents only the API.

class block_source_t

The nothrow block seam every FAILABLE allocation draws from — the ones a PEER can provoke (#551, ADR-0065; ADR-0039 erratum 5/6).

RFC-0014 made vertex registration a runtime, wire-driven operation: a peer’s CREATE frame reaches register_vertex_key. Every allocation on that path is an unguarded throwing one, and ESP-IDF link-wraps __cxa_throw / __cxa_allocate_exception to abort() stubs — so on the shipping profile a peer can reboot the node by exhausting the heap. This seam is the failure-by-value answer: exhaustion returns nullptr and the operation answers BACKPRESSURE.

Nor does a budget-tracking variant fix it — one that counts its own bytes and answers nullptr at the ceiling before delegating. Tracking a budget does not make the adapter honest: a FRAGMENTED pmr resource can throw well BELOW the budget, so the adapter is correct except exactly when the underlying resource is in the state the bound was supposed to protect against. Such an adapter is deliberately not offered and must not be added.

**The supported answer for “reuse my existing arena” is pool_source_t’s span constructor**, which carves from a caller-provided slab with caller-provided size classes and is not pmr at all — point it at the same storage the pmr resource was partitioning, rather than at the resource. The one direction that IS offered is the opposite one: tr::mem::source_resource_t (mem_source_pmr.hpp) serves a std::pmr container FROM a block_source_t.

Note

“Failable”, not “control-plane”: CONTEXT.md already binds control plane to the : field-write addressing plane, and this seam is orthogonal to that axis — a DATA-plane branch write is one of its first consumers.

Note

Deliberately NOT a std::pmr::memory_resource, and not derived from one. That type’s allocate is annotated __attribute__((__returns_nonnull__)) (libstdc++ bits/memory_resource.h), so a caller’s if (p == nullptr) is undefined-behaviour-deletable. Measured on riscv32-esp-elf-g++ 15.2.0 with the deployment flags: the soft-fail branch survives at -O0/-O1/-O2/-O3 and is GONE at -Os/-Oz — the level the reference node ships at (CONFIG_COMPILER_OPTIMIZATION_SIZE), and the level at which no job exercises an allocation-failure path (see ADR-0065 §1). Inheriting would keep that allocate() publicly callable on this object, one token away from every correct try_alloc call site, with no diagnostic at any warning level. A separate type makes the slip a compile error.

Warning

DO NOT WRAP A std::pmr::memory_resource BEHIND THIS SEAM. The note above says why this type is not a pmr resource; this one is about the REVERSE adaptation, which is the mistake a host migrating an existing pmr arena actually makes (#1493). The obvious adapter compiles, looks correct and passes review:

void* try_alloc(std::size_t n, std::size_t a) noexcept override {
    return mr_->allocate(n, a);   // <-- CANNOT report exhaustion
}

std::pmr::memory_resource::allocate signals exhaustion only by THROWING and has no nothrow form, so this try_alloc either succeeds or never returns — it never answers nullptr. Under -fno-exceptions the throw reaches ESP-IDF’s link-wrapped __cxa_throw → abort() stub, which is exactly the reboot-a-node-by-exhausting-the-heap failure this seam exists to remove, reintroduced by a class whose declaration promises the opposite. A comment on the caveat does not fix it; it labels the landmine.

Note

Also distinct from mem_backend_t, which vends a refcounted view::segment_t. Control-plane blocks have a single owner and no header; a refcount on them is pure overhead (a segment_t measures 20 B on rv32 / 40 B on x86-64 against a vertex_t of 72 B on rv32 / 96 B on x86-64 — the sizes the config_t ratchets pin, re-measured by the #1487 census).

Note

Blocks are host-owned storage: the source MUST outlive the graph_t and every object built in its blocks. Teardown is driven by whoever holds the source, never by the object itself — a vertex_t has no room for the pointer (core/tests/vertex_size_test.cpp).

Note

Each source declares its own concurrency contract, exactly as mem_backend_t does (ADR-0012). The RFC-0014 wire-driven registration path runs on a transport thread, so an injected source must be thread-safe on that target. heap_source_t is.

Subclassed by tr::mem::slab_pool_t< graph::guard_t, std::size(graph::config_t::kSizeClasses)>, tr::graph::rx_loan_source_t, tr::mem::bump_source_t, tr::mem::heap_source_t, tr::mem::host_root_t, tr::mem::host_values_t, tr::mem::mem_backend_t, tr::mem::null_source_t, tr::mem::pool_source_t< Sync >, tr::mem::slab_pool_t< Sync, N, kCounters >

Public Functions

inline explicit constexpr block_source_t(const char *name) noexcept

Construct a source with a stable, human-readable name (e.g. "heap").

virtual ~block_source_t() = default

Sources are held by pointer and outlive their users; virtual teardown.

block_source_t(const block_source_t&) = delete

Non-copyable — a source is an identity, not a value.

block_source_t &operator=(const block_source_t&) = delete

Non-assignable.

virtual void *try_alloc(std::size_t bytes, std::size_t align = alignof(std::max_align_t)) noexcept = 0

Obtain bytes of storage aligned to at least align — NOTHROW.

Parameters:

• bytes – Size of the block; a zero-sized request is implementation-defined and callers do not make one.

• align – Minimum alignment, a power of two.

Return values:

nullptr – Exhaustion. The caller answers BACKPRESSURE; it never falls back to the global heap and never aborts.

virtual void release(void *p, std::size_t bytes, std::size_t align = alignof(std::max_align_t)) noexcept = 0

Return a block previously handed out by try_alloc.

Warning

bytes and align MUST match the originating try_alloc call (sized reclaim), so a bump or pool source needs no per-block header.

inline const char *name() const noexcept

The source’s stable name, for census and diagnostics.

inline virtual source_stats_t stats() const noexcept

This source’s census — the interface-level introspection seam (#1492, #1503).

The whole vocabulary of this seam used to be name, so a host holding a block_source_t& could introspect NOTHING: not the ceiling it injected, not how much of it was gone, and above all not whether anything had been refused — try_alloc → nullptr was uncounted at every implementation in the tree.

Optional, in the tr::net::transport_t::drop_stats mould (#932): the DEFAULT is all-zero, which is the honest answer for a source that counts nothing, never a fabricated number. Concrete sources override it — bump_source_t and pool_source_t do; heap_source_t does not (the platform heap’s ceiling is not this seam’s to report), and neither does null_source_t, whose refusals are its whole contract and are the CALLER’s to count.

Counted, never enforced, and never on the hot arm: the refusal counters are bumped only where try_alloc is already returning nullptr, so a successful allocation pays nothing at all (core/STYLE.md §Introspection, counting doctrine 1 — ADR-0039’s bench_forward_heap == 0 hop and ADR-0067’s rv32 text figure are the standing referees).

The seam’s census block — the one introspection vocabulary every bounded resource in the tree answers with (core/STYLE.md §Introspection).

struct source_stats_t

One block source’s census, in the unified introspection vocabulary (core/STYLE.md §Introspection; #1503).

The five nouns every bounded resource answers with, spelled the same way here as at every other seam: an effective ceiling, used-polarity occupancy, a high-water mark, and the two numbers a sizing operator actually needs — how often a request was refused, and how big the biggest refused one was (#1492: the TAIL is what refuses, so a median request size tells the operator nothing).

All-zero is the honest default for a source that counts nothing, exactly as tr::net::transport_drop_stats_t is for a link that counts nothing (#932) — never a fabricated number. A field a particular source cannot answer stays 0; capacity == 0 means “unbounded, or not reported”, never “a zero-byte ceiling”.

Snapshot coherence is the core/STYLE.md §Introspection clause: monotonic since construction, sampled unsynchronized, and the intended reading is the DIFFERENCE between two snapshots rather than an instant.

Public Members

std::size_t capacity = 0

The effective byte ceiling this source serves from — the caller’s injected slab, not a compile-time constant. 0 = unbounded / not reported.

std::size_t in_use = 0

Bytes handed out and not returned to this source, USED-polarity (free is capacity - in_use, and is deliberately not the primary).

std::size_t peak = 0

High-water mark of in_use since construction.

std::size_t refused = 0

block_source_t::try_alloc calls this source answered nullptr — requests refused BY VALUE, so the caller was told (it answered BACKPRESSURE). Distinct from a dropped, where nobody was told.

std::size_t largest_refused = 0

Bytes of the LARGEST request in refused — the number a deployment grows its slab to. 0 iff refused is 0.

class heap_source_t : public tr::mem::block_source_t

The default source: the platform heap, nothrow.

Behaviour is byte-identical to today for a host that injects nothing, EXCEPT that exhaustion returns nullptr instead of reaching the ESP-IDF __cxa_throw abort stub. Thread-safe: the global nothrow operator new is.

Public Functions

inline constexpr heap_source_t() noexcept

Constant-initializable, so the process-wide default costs no dynamic init.

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

Nothrow heap allocation; nullptr on exhaustion.

The virtual entry — and only it, never acquire — consults the test-only tr::detail::probe_hook_ok seam first, so an injected refusal reaches whatever draws through the process-default source (a value_t block, a control-plane container) without the hot direct-call arm heap_backend_t takes paying for it.

Why the plain- arm exists (#873 phase 1)

A request at or below __STDCPP_DEFAULT_NEW_ALIGNMENT__ takes the PLAIN nothrow operator new, not the over-aligned one. The two are not the same code: libstdc++ routes the align_val_t overload through aligned_alloc/posix_memalign even when the alignment is one the plain allocator already guarantees, which is a different glibc path with a different size-class layout. That distinction became load-bearing when #873 phase 1 moved channels that had ALWAYS used the plain operator new — the child-registry chunks and the graph’s pmr-served control blocks — behind this seam: with this arm, “the process default preserves today’s behaviour byte-for-byte” is literally true rather than approximately true. Over-aligned requests (a DMA source’s clients, a cache-line-padded stripe) keep the aligned pair, so nothing loses a guarantee it had.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Sized reclaim matching try_alloc, on whichever arm served the block.

Public Static Functions

static inline void *acquire(std::size_t bytes, std::size_t align) noexcept

The platform-heap acquisition arm as a FREE, non-virtual entry point (#873 phase 3).

The same two arms try_alloc dispatches, callable without an object and therefore without a virtual call. It exists so that the one in-tree mem_backend_t that still acquires its bytes from the platform heap — heap_backend_t — can express that acquisition on the substrate rather than on a second, independently-spelled platform-heap pair. Phase 3’s re-layering is a layering claim, not a new indirection: heap_backend_t is the process default and sits on the hottest allocation path in the library, so routing it through a block_source_t& would have bought no bounding (a deployer who wants bounding injects a source and gets source_backend_t) at the cost of the virtual draw #873 phase 2 measured at +22.7 % on the hazard domain. A static entry point gives the layering with a direct call.

Parameters:

• bytes – Size of the block.

• align – Minimum alignment, a power of two.

Return values:

nullptr – Exhaustion.

static inline void reclaim(void *p, std::size_t bytes, std::size_t align) noexcept

The sized-reclaim twin of acquire, on whichever arm served the block.

block_source_t &tr::mem::heap_source() noexcept

The process-wide default block_source_t (the platform heap).

A namespace-scope constinit object behind a function, NOT a function-local static: the latter costs a __cxa_guard word in .bss and an acquire fence on every call.

class null_source_t : public tr::mem::block_source_t

The source that serves nothing — every request is exhaustion.

The upstream to give a bump_source_t when its buffer must be the HARD bound, so a frame that outgrows it is rejected rather than reaching the global heap. This is the bounded-node composition, and the honest replacement for std::pmr::null_memory_resource(), which signals the same thing by throwing.

Public Functions

inline constexpr null_source_t() noexcept

Constant-initializable, like heap_source_t.

inline virtual void *try_alloc(std::size_t, std::size_t) noexcept override

Always nullptr.

inline virtual void release(void*, std::size_t, std::size_t) noexcept override

Unreachable — this source hands out nothing to return.

block_source_t &tr::mem::null_source() noexcept

The process-wide null_source_t (serves nothing; see the class docs).

class bump_source_t : public tr::mem::block_source_t

A caller-owned buffer handed out by bump, falling back to upstream once full.

The nothrow twin of std::pmr::monotonic_buffer_resource over a fixed span. Blocks carved from the span are never individually reclaimed (release is a no-op for them, exactly as a monotonic resource behaves); blocks that came from the upstream are returned to it, so a decode that outgrows the buffer still frees what it borrowed.

Note

The upstream is what keeps this a capability-preserving substitution: a monotonic_buffer_resource also spills past its buffer, but it spills to a THROWING default resource, which on -fno-exceptions is the abort() this whole seam exists to remove. Pass a bounded source (or a null-serving one) to make the buffer the hard limit instead.

Note

Single-threaded by contract — a bump cursor is not synchronized. Its intended use is a function-scoped buffer on the calling thread’s stack.

Warning

SCOPE-LIFETIME USE ONLY. A bump block is never reclaimed, so a source that outlives one burst of work monotonically fills and then refuses everything. Construct it per operation (as the branch-write decode does), or reset it between operations. It is NOT a long-lived seam: an 8 KiB bump source wired as a router’s rx decoded 6 frames and rejected the next 194 — measured. A long-lived bounded seam wants pool_source_t, which recycles.

Public Functions

inline explicit bump_source_t(std::span<std::byte> buffer, block_source_t &upstream = heap_source()) noexcept

Carve from buffer; once it cannot serve a request, draw from upstream.

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

Bump-allocate, aligned; falls back to the upstream when the buffer cannot fit.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

No-op for a bump block; a sized return to the upstream otherwise.

inline void reset() noexcept

Hand the whole buffer back for reuse — the next try_alloc starts at 0.

For a source reused across scopes (a terminus decoding frame after frame into the same slab). Deliberately NOT called release: on a std::pmr::monotonic_buffer_resource that name means exactly this, while on block_source_t it means “return one block”, and the two must not be confusable at a call site.

Warning

Every block previously carved from the buffer dangles afterwards. Blocks that came from the upstream are NOT reclaimed by this — return those first.

inline std::size_t used() const noexcept

Bytes carved from the buffer so far (diagnostics; excludes upstream blocks).

inline virtual source_stats_t stats() const noexcept override

This bump source’s census (source_stats_t; core/STYLE.md §Introspection).

capacity/in_use/peak describe the CALLER’S BUFFER — the span size this source was handed, how much of it the cursor has carved, and the deepest any reset cycle got. Upstream spill is deliberately outside all three: those bytes are the upstream’s census to report, and folding them in here would make in_use exceed capacity on the very source whose ceiling the number exists to describe.

refused counts what a caller experienced: a try_alloc that answered nullptr, which for this source means the buffer could not fit the request AND the upstream refused it too. Against a bounded upstream (null_source() — the composition that makes the buffer a hard limit) that is exactly “the buffer overflowed”; against heap_source() it stays 0 until the platform heap is gone, which is the honest reading in both cases.

Plain counters, no atomics: this source is single-threaded BY CONTRACT (see the class note), so the ownership discipline that already protects used_ protects these (core/STYLE.md §Introspection, counting doctrine 5).

template<::tr::lockable Sync = ::tr::no_guard_t>
class pool_source_t : public tr::mem::block_source_t

A BOUNDED, RECYCLING source: segregated exact-size free lists over a caller slab.

The long-lived counterpart to bump_source_t, and the source a node with a RAM ceiling injects. Exhaustion is nullptr — never the platform heap, never an abort.

Public Functions

inline pool_source_t(std::span<std::byte> slab, std::span<size_class_t> classes) noexcept

Serve allocations from slab, recycling through classes.

This is also the supported “reuse my existing arena” path (#1493). A host that already partitions a static slab with std::pmr — the monotonic_buffer_resource under a synchronized_pool_resource shape ADR-0039 describes — points this constructor at the SAME STORAGE rather than at the resource. There is no pmr in the result and nothing to adapt, so exhaustion stays a nullptr all the way down; see block_source_t’s warning for why wrapping the resource instead cannot work.

Parameters:

• slab – Caller-owned storage; must outlive every block carved from it.

• classes – Caller-owned free-list slots. Running out is safe but lossy — see overflowed.

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

Pop a recycled block of this exact shape, else carve a fresh one; nullptr when full.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a block to its class’s free list.

bytes and align MUST match the originating try_alloc, per the seam’s sized contract — that is what lets a block carry no header. A pointer from outside the slab is ignored rather than trusted, mirroring bump_source_t — two compares are cheaper than the corruption a foreign pointer would cause.

inline std::size_t used() const noexcept

Bytes carved from the slab so far, recycled blocks included (diagnostics).

inline std::size_t classes_used() const noexcept

Class slots in use — the number to size the injected span against.

inline std::size_t overflowed() const noexcept

Blocks lost because the class table was full; non-zero means the span is too small.

Note

A RECYCLING DEGRADE, not an allocation refusal, and the two are deliberately separate counters (core/STYLE.md §Introspection): the block stays carved — bounded and safe — and the caller that freed it was never refused anything. The refusal number is refused.

inline std::size_t refused() const noexcept

try_alloc calls this pool answered nullptr — primary slab exhaustion (#1492). Also reachable via stats.

inline std::size_t largest_refused() const noexcept

Bytes of the largest request in refused — the number to grow the slab to (#1492: the tail is what refuses).

inline virtual source_stats_t stats() const noexcept override

This pool’s census (source_stats_t; core/STYLE.md §Introspection).

capacity is the injected slab, and in_use is used — bytes CARVED, recycled blocks sitting on a free list included, because a carved block is never returned to the slab and so is not available to a different size class. That makes carving monotonic, and peak therefore equals in_use by construction: the high-water mark costs this source not one instruction.

Plain counters under the existing Sync section, not atomics: the refusal bump sits inside the same guard_t try_alloc already holds, so a shared pool’s counters are as synchronized as its free lists are and nothing new is locked. On rv32 an atomic wide enough to matter is not lock-free anyway — it takes a hidden libatomic lock per access (core/STYLE.md §Introspection, counting doctrine 5).

Note

overflowed is NOT in this block. It counts a recycling degrade rather than a refusal, so it is neither refused nor dropped in the shared vocabulary, and it stays this type’s own named accessor.

struct size_class_t

One recycling free-list, keyed by the exact (bytes, align) pair it serves.

Caller-supplied storage: a pool_source_t is handed a span of these, so the number of classes is a deployment property rather than a constant in this header (ADR-0065’s injected-bounds rule). Sizing it is measurable — see pool_source_t::classes_used.

Public Members

std::size_t bytes = 0

Normalized payload size; 0 marks an unused slot.

std::size_t align = 0

Normalized alignment this class’s blocks satisfy.

void *head = nullptr

Intrusive free-list head; the link lives in the block.

The std::pmr adapter — one direction only, and outside the library. It lets an embedder point a std::pmr container at an injected block_source_t; it does not make that container failable. See the memory substrate reference for when to reach for it and when to migrate the store instead.

class source_resource_t : public std::pmr::memory_resource

Serve a std::pmr container from an injected block_source_t.

The adapter ADR-0079 ends on: *”`std::pmr` survives only as a thin adapter for

std-container interop on non-failable paths.”* It exists for the one case a store migration cannot solve by retyping — a

std::pmr container whose element type is neither trivially copyable nor trivially destructible, so block_array_t’s two static assertions reject it (tr::net::can_reassembly_t’s slice map holds a refcounted tr::view::view_t; #873 family 5). Pointing such a container at a bounded pool_source_t is strictly better than leaving it on the process heap.

Note

Stateless beyond the one pointer. There is deliberately NO refusal counter here: a non-atomic one races on a shared adapter and an atomic one puts a shared RMW on an allocation path, which #873’s cadence rules a reject. Counting belongs to the injected source, which already has the vocabulary (pool_source_t::used, pool_source_t::classes_used, pool_source_t::overflowed).

Note

Concurrency is entirely the injected source’s contract — the adapter adds no shared state of its own. Own one source per receiver: ADR-0060 erratum 1 measured a shared free-list pool collapsing to ~1/15 of its single-thread rate on a 12-core host. Wrapping it in a memory_resource does not change that.

Note

Blocks are host-owned: both the source and this adapter must outlive every container built over them.

Warning

THIS DELIVERS PLACEMENT AND BOUNDING, NOT FAILABILITY. std::pmr’s only exhaustion signal is a throw, so this adapter’s boundary is a std::bad_alloc on a hosted build and a std::abort() under -fno-exceptions — byte-for-byte the behaviour libstdc++ itself produces for the same throw on that profile. A peer-provoked path must therefore NOT be moved onto a std::pmr container just because this exists: a store that has to SURVIVE exhaustion migrates via the route-handle pattern (docs/reference/09-memory-substrate.md) onto block_array_t and fails by value. What this buys is that the bytes come from the deployer’s slab instead of the global heap, and that the slab’s size is the bound.

Public Functions

inline explicit source_resource_t(block_source_t &src) noexcept

Serve every request from src; src must outlive this adapter.

source_resource_t(const source_resource_t&) = delete

Non-copyable — a resource is an identity, exactly as a source is.

source_resource_t &operator=(const source_resource_t&) = delete

Non-assignable.

inline block_source_t &source() const noexcept

The source the bytes come from — for census and for sizing its slab.

Protected Functions

inline void *do_allocate(std::size_t bytes, std::size_t alignment) override

Draw bytes aligned to alignment from the source.

Throws:

std::bad_alloc – The source refused. This is the adapter’s boundary and the reason it is not a failable seam; under -fno-exceptions the refusal is std::abort() instead, which is what libstdc++ does with the same throw.

inline void do_deallocate(void *p, std::size_t bytes, std::size_t alignment) override

Return a block to the source.

std::pmr’s deallocate carries the original size and alignment, which maps 1:1 onto block_source_t::release’s sized-reclaim contract — that is what lets a pool_source_t recycle these blocks with no per-block header.

inline bool do_is_equal(const std::pmr::memory_resource &other) const noexcept override

Identity comparison — two adapters are equal only when they are the SAME object.

Address identity rather than a dynamic_cast on the source pointer: the reference node ships -fno-rtti, so a cross-type dynamic_cast is not available to this header at all. libstdc++’s own monotonic_buffer_resource answers the same way.

Note

The consequence is worth stating: two source_resource_ts over the SAME block_source_t compare unequal, so containers built over them will copy rather than steal storage on a container move-assign. Construct one adapter per source and pass it around, rather than one per container.

The mem_backend_t wrapper — the same one-direction relationship on the other seam. Since #873 phase 1, mem_backend_t is a wrapper TYPE over the substrate (a source plus the refcount / DMA-hook table) rather than an injection seam of its own, and this class is that wrapper. A refused block becomes a null segment_t*, which is the BACKPRESSURE signal alloc already documented — the substrate’s raw nullptr translated at this adapter’s own boundary and nowhere else.

class source_backend_t : public tr::mem::mem_backend_t

Serve refcounted view::segment_t allocations from an injected block_source_t.

What this type IS, after #873 phase 1

ADR-0079 kept mem_backend_t separate from block_source_t because a segment carries things raw bytes do not: an intrusive refcount and the DMA cache-op hook pair. The 2026-08-26 ruling settled the relationship between them — the substrate is block_source_t, and mem_backend_t survives as a wrapper TYPE (a source plus that refcount/DMA-hook table) rather than as an injection seam of its own. This class is that wrapper. graph_t no longer takes a backend at construction; it builds one of these over the single source it IS given.

The failure convention

The substrate speaks raw nullptr-on-exhaustion and does no wrapping (the ruling’s round-2 detail). This adapter translates at its own boundary and nowhere else: a refused try_alloc becomes a null , which is precisely the BACKPRESSURE signal mem_backend_t::alloc already documents. Nothing throws, nothing aborts, and no result_t appears in the substrate — contrast source_resource_t, whose std::pmr contract forces it to translate the same nullptr into a std::bad_alloc.

ONE block per segment (#873 phase 3)

The control block and the payload live in a SINGLE block_source_t::try_alloc call — the segment header first, padded up to alignment, then the payload. Phase 1 took two separate draws, mirroring heap_backend_t’s operator new pair one for one, and said so — the packing was the obvious improvement, deliberately deferred because phase 1’s contract was that the process-default composition behaves as it always did. It is taken here because this type is never on the process-default path (see the tag note below), and because on the deployments that DO construct it the two-draw shape is actively wrong: a pool_source_t serves the control block from one size class and the payload from another, so a bounded node paid two class allocations, two refusal opportunities and the per-class rounding twice for one segment. One draw is the shape pool_t (the other bounded backend) has always had — header and payload carved into one slot.

Note

The tag stays backend_tag::UNKNOWN, and phase 3 is where that stops being a deferral and becomes a decision. Phase 1 left the module-set SOURCE enumerator to “the phase that decides whether this type is the only backend left”; it is not. The fast set keeps heap_backend_t (the process default, whose bytes now come from the substrate’s own heap_source_t arm), pool_t (a caller-owned slab with no acquisition to re-layer — its slab IS the bound) and the two borrowed backends (which acquire nothing at all). An enumerator would devirtualize reclaim only for the injected-source composition, at the cost of a fifth switch arm and this type’s destroy body in backend_set.cpp for every target that links the multi-member set — i.e. every host target, none of which is the one that benefits. Recorded here as DECIDED, not deferred again. The virtual destroy fallback is the same path every out-of-core backend already takes, and it costs the DEFAULT composition nothing: graph_t folds a process-default source back onto heap_backend (tagged HEAP) and never constructs this type at all.

Note

alloc and destroy are defined OUT OF LINE (core/src/mem_source_backend.cpp), unlike heap_backend_t’s, and that is deliberate rather than stylistic. This type is tagged UNKNOWN, so its reclaim is a virtual call in every case and inlining the bodies buys nothing — while making them visible in a TU that holds one (graph.cpp does, as graph_t’s internal wrapper) MEASURABLY re-partitions GCC’s inline budget there: it flipped tr::view::segment_ptr_t::reset from an out-of-line call into graph_t::dispatch_edge_remote, growing that pinned symbol by 32 B (318 → 350) for no reason connected to what the wrapper does. Bisected against the symbol ratchet; the same hazard class graph_t’s payload_right_store_ comment records. Keep them out of line.

Note

Concurrency, alignment and lifetime are ENTIRELY the injected source’s contract — this adapter adds no state beyond one pointer. ADR-0060 §2’s requirement stands: a value segment self-routes its reclaim on whatever thread drops the last ref, so a source injected into a graph must be thread-safe on a target where that happens.

Public Functions

inline explicit source_backend_t(block_source_t &src) noexcept

Serve every segment from src; src must outlive this backend.

inline block_source_t &source() const noexcept

The source the bytes come from — for census and for sizing its slab.

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

A raw block, straight from the wrapped source (the block_source_t half).

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a raw block to the wrapped source, sized.

inline virtual source_stats_t stats() const noexcept override

The wrapped source’s census: this backend keeps no bytes of its own.

virtual view::segment_t *alloc(std::size_t size, alloc_hint_t hint = alloc_hint_t::NONE) override

Allocate a size-byte segment (refcount 1) from the source, in ONE block.

Parameters:

size – Payload bytes; a zero-size request yields an empty-but-valid segment, exactly as heap_backend_t’s does (an empty bytes span).

Return values:

nullptr – The source refused — BACKPRESSURE. One draw, so there is nothing to unwind on a refusal.

virtual void destroy(view::segment_t *seg) noexcept override

Return the single block to the source, in the size alloc took it in.

inline virtual std::size_t alignment() const noexcept override

The alignment block_source_t::try_alloc’s default requests.

Public Static Functions

static inline constexpr std::size_t block_bytes(std::size_t size) noexcept

The single block alloc draws for a size-byte segment (segment_block_bytes). Always one block: the source is sized for one draw per segment, so a segment here is never split across draws.

Public Static Attributes

static constexpr std::size_t kBlockAlign = segment_block_align(alignof(std::max_align_t))

The block alignment one segment is drawn at — the stricter of the payload’s fundamental alignment and the control block’s own (segment_block_align).

static constexpr std::size_t kHeaderBytes = segment_header_bytes(kBlockAlign)

Bytes the control block occupies at the head of the block, padded so the payload that follows it starts at kBlockAlign (segment_header_bytes).

static constexpr bool needs_cache_ops = false

The source declares its own DMA needs; this wrapper adds none.

static constexpr bool is_isr_safe = false

Whatever the source is; assume the strictest.

static constexpr bool is_nonblocking = false

Whatever the source is; assume the strictest.

static constexpr bool owns_bytes = true

The blocks are the source’s, held for the segment’s whole life.

The standard-Allocator face, for the containers neither of the two above can serve. A block_array_t needed trivially copyable and trivially destructible elements until #1776; a store holding std::shared_ptrs or std::vector keys has neither, and before #873 phase 1 those sites were stranded on the global heap. This changes only where the vector’s block comes from and leaves the element type alone. It still THROWS on exhaustion — the Allocator requirements admit no other signal — so it delivers placement and bounding, and the by-value refusal comes from tr::detail::try_reserve one level up.

template<class T>
class source_allocator_t

A standard Allocator serving T from an injected block_source_t.

Why this exists when block_array_t already does

block_array_t is the right container for a failable path and the wrong one for most of the graph’s growth: it requires trivially copyable AND trivially destructible elements, and the graph’s collection tables hold std::vector<std::byte> keys, std::shared_ptr LKVs and refcounted view_ts. Those sites were therefore stranded on std::vector over the GLOBAL heap — #873’s channel 4 — each with a #981 residual note in the source saying exactly that. An allocator changes only WHERE the vector’s block comes from and leaves the element type and its destructors alone, so those sites can be bounded without being rewritten.

The probe this makes CORRECT, which is the real prize

mem_heap.hpp carries a standing warning against generalizing tr::detail::try_reserve to std::pmr::vector, and the reason is the probe: on the -fno-exceptions profile the helper tests the GLOBAL heap with a throwaway operator new, while a pmr container allocates from its INJECTED resource — so it answers a question about memory the container will never touch, and the real allocation still aborts. That objection does not apply here, because this allocator’s storage and the probe can be the SAME source: try_reserve’s allocator-aware overload probes with src.try_alloc / src.release on the very store the growth will draw from. The probe is still probe-then-commit and still carries the #850 race window; what changes is that it stops asking the wrong allocator.

Note

Stateful (one pointer), so a container carrying it is one word wider, and two allocators over different sources compare UNEQUAL — which is correct: a container move-assign across them must copy rather than steal a block the other source owns.

Note

The source must outlive every container built over this allocator.

Warning

allocate still THROWS on exhaustion, because the standard Allocator requirements leave it no other signal. That is not a regression — a std::vector growing on the global heap throws too — and the growth helpers in mem_heap.hpp are what turn it into a value. A path that must SURVIVE exhaustion with no throw anywhere in the picture still migrates to block_array_t; this is for the paths that cannot.

Public Types

using value_type = T

The element type the standard Allocator requirements name.

Public Functions

inline explicit source_allocator_t(block_source_t &src) noexcept

Serve from src; src must outlive every container using this.

template<class U>
inline explicit(false) source_allocator_t(const source_allocator_t<U> &o) noexcept

Rebinding conversion — a container allocates its own node types through this.

inline block_source_t &source() const noexcept

The store the blocks come from — for a census, and for the growth helpers’ probe.

inline T *allocate(std::size_t n)

Take storage for n elements.

Throws:

std::bad_alloc – The source refused. See the class warning: the Allocator requirements admit no by-value refusal, so the conversion to a value happens one level up, in tr::detail::try_reserve / try_push_back.

inline void deallocate(T *p, std::size_t n) noexcept

Return storage for n elements — the seam’s SIZED reclaim, matched exactly.

template<class U>
inline bool operator==(const source_allocator_t<U> &o) const noexcept

Two allocators are interchangeable iff they serve from the SAME source.

using tr::mem::sync_none_t = ::tr::no_guard_t

The no-op synchronization policy — the default, and the one the hot seam wants.

A pool_source_t owned by exactly one thread needs no synchronization at all, and that is the intended shape for a per-receiver source: ownership removes the race instead of guarding it. See pool_source_t’s threading note for why this matters more than the choice of free-list algorithm.

Since #1703 this is the layer-neutral tr::no_guard_t (guard.hpp), so the memory layer keeps one lock vocabulary, not two. A policy is any tr::lockable (lock()/unlock(), both noexcept); a target supplies its own where it needs one (an interrupt-disable critical section on single-core FreeRTOS, tr::mem::sync_mutex_t from mem_source_sync.hpp on a host). This header stays freestanding-clean, so it pulls in no threading facility of its own.

Deprecated: Kept as an alias for one release (#1703); name tr::no_guard_t.

template<class T>
class block_array_t

The core’s failable vector: a nothrow growable array of T drawn from a block_source_t (ADR-0083 Decision 2, #1776).

The container a failable path uses where a std::vector or std::pmr::vector would otherwise sit (#551 Q2, #588). Three properties carry the whole point:

1. Growth reports refusal by value and never throws. std::pmr::vector::push_back on an exhausted resource throws, which on ESP-IDF reaches the link-wrapped __cxa_throw abort() stub — a peer-reachable reboot when the container sits on the RX decode path. Here every growing call answers false or nullptr, and a refused call leaves the array exactly as it was: same elements, same block, and an argument passed by rvalue is not consumed.

2. Every byte comes from the injected source. Nothing reaches the global heap, so the array is usable on a static-arena node with no heap at all.

3. Relocation is a for a trivially copyable . That case needs no move loop and no destruction, and it is the shape the hot users have (wire::arena_tlv_t, the walk’s open-node record). Any other T is relocated by move construction, which must be noexcept, and destroyed in place; that branch is compiled only for such a T, so a trivially copyable array generates the same code it always did.

Same footprint as std::pmr::vector (four words), one virtual call per growth instead of the allocator’s two. Non-copyable: a copy is a failable allocation, so it is not hidden in a constructor.

Public Types

using value_type = T

The element type.

using iterator = T*

A mutable element iterator (a plain pointer: the storage is contiguous).

using const_iterator = const T*

A read-only element iterator.

Public Functions

inline explicit block_array_t(block_source_t &src) noexcept

An empty array that will draw its storage from src.

inline ~block_array_t()

Destroys the elements and returns the block, if one was taken.

block_array_t(const block_array_t&) = delete

Non-copyable — one array, one block.

block_array_t &operator=(const block_array_t&) = delete

Non-assignable.

inline block_array_t(block_array_t &&o) noexcept

Move-constructible so a decode can return its arena by value.

inline block_array_t &operator=(block_array_t &&o) noexcept

Move-assignable (destroys and releases this array’s contents first).

inline bool reserve(std::size_t n) noexcept

Ensure room for n elements without growing again.

Return values:

false – The source is exhausted — the array is unchanged.

inline bool resize_for_overwrite(std::size_t n) noexcept

Set the size to n, growing to exactly n when needed; elements past the old size are left UNINITIALIZED for the caller to overwrite (a key rendered back to front). Offered only for a trivially copyable T.

Return values:

false – The source is exhausted — the array is unchanged.

inline bool push_back(const T &v) noexcept

Append a copy of v.

Warning

For a trivially copyable T, v must not refer to an element of this array: growth releases the old block before the copy is read. Use emplace_back, which builds in the fresh block first, when it might.

Return values:

false – The source is exhausted — the array is unchanged (BACKPRESSURE).

inline bool push_back(T &&v) noexcept

Append v by move.

Offered only for a T that is not trivially copyable: for one that is, a move is a copy, and an rvalue keeps binding to the copying overload and its pre-#1776 code.

Return values:

false – The source is exhausted — the array is unchanged and v is not moved from.

template<class ...Args>
inline T *emplace_back(Args&&... args) noexcept

Construct one element at the end from args and return it.

An argument may refer to an element of this array: on growth the new element is built in the fresh block before the old one is released.

Return values:

nullptr – The source is exhausted — the array is unchanged and no argument is moved from (BACKPRESSURE).

template<class ...Args>
inline T *emplace_at(std::size_t i, Args&&... args) noexcept

Construct one element at index i from args, shifting the tail up by one.

The insertion the sorted map is built on. Precondition: i <= size().

Return values:

nullptr – The source is exhausted — the array is unchanged and no argument is moved from (BACKPRESSURE).

inline T *push_slot() noexcept

Claim one uninitialized slot at the end and return it — fill it IN PLACE.

The form the hot paths use. push_back(T{...}) has to materialize the aggregate on the stack and copy it in, and for a 48-byte T written field-by-field then read back as wide loads that is a store-forwarding stall on every element: measured on the terminus decode, ~45 % slower with FEWER instructions executed. Writing through this slot removes the temporary entirely. Offered only for a trivially copyable T, whose lifetime the caller’s stores begin; any other T uses emplace_back.

Return values:

nullptr – The source is exhausted — the array is unchanged (BACKPRESSURE).

inline bool append(const T *p, std::size_t n) noexcept

Append n elements copied from p, growing geometrically (or to fit them, if more) when needed — so repeated appends stay amortized O(1) per element.

Offered only for a trivially copyable T (a byte string, a pointer table). p must not point into this array. Call reserve first to size the block exactly.

Return values:

false – The source is exhausted — the array is unchanged (BACKPRESSURE).

inline void pop_back() noexcept

Drop the last element. Precondition: not empty.

inline void erase_at(std::size_t i, std::size_t n = 1) noexcept

Remove the n elements from i, shifting the tail down — one move of the tail however many go. Precondition: i + n <= size().

inline void erase_front(std::size_t n) noexcept

Drop the first n elements, keeping the rest at the front in order — the one compaction a stream buffer or a FIFO table takes (#1780). Never allocates. Precondition: n <= size().

inline void clear() noexcept

Destroy every element; the block is kept for reuse.

inline T &back() noexcept

The last element. Precondition: not empty.

inline const T &back() const noexcept

The last element (const). Precondition: not empty.

inline T &front() noexcept

The first element. Precondition: not empty.

inline const T &front() const noexcept

The first element (const). Precondition: not empty.

inline T &operator[](std::size_t i) noexcept

Element i, unchecked.

inline const T &operator[](std::size_t i) const noexcept

Element i, unchecked (const).

inline std::size_t size() const noexcept

Element count.

inline std::size_t capacity() const noexcept

Elements the current block holds before the next growth.

inline bool empty() const noexcept

True when no elements are held.

inline T *data() noexcept

First element, or nullptr when empty — the contiguous block.

For handing the array to an API that takes a pointer/length pair, e.g. building a std::span over an egress iov table. The pointer is invalidated by any growth.

inline const T *data() const noexcept

First element (const), or nullptr when empty.

inline iterator begin() noexcept

Iterator to the first element; invalidated by any growth.

inline iterator end() noexcept

Iterator past the last element.

inline const_iterator begin() const noexcept

Read-only iterator to the first element.

inline const_iterator end() const noexcept

Read-only iterator past the last element.

inline block_source_t &source() const noexcept

The source this array draws from.

The core container set (ADR-0083 Decision 2, #1776): block_array_t above is the vector, and the name/string store and the sorted map below complete it. All three draw every byte from a block_source_t, report a refused growth by value, and leave the container (and any argument passed by rvalue) unchanged when they do. Nothing has migrated onto them yet.

class string_t

An owning, NUL-terminated string whose every byte comes from a block_source_t, and whose growth reports refusal by value (ADR-0083 Decision 2).

The block is sized EXACTLY by assign (a name is written once and read many times, so slack is wasted RAM on a small node) and doubled by append. A refused call leaves the string as it was. Non-copyable, because a copy is a failable allocation: copy with dst.assign(src), which says so. Movable, which steals the block.

Reads go through view, or the implicit conversion to std::string_view; c_str is for the C APIs that want a terminator. Comparison is byte-wise against any std::string_view, so a sorted_map_t keyed by string_t can be searched with a view without building a key.

Four words: the source, the block, the length and the capacity.

Public Functions

inline explicit string_t(block_source_t &src) noexcept

An empty string that will draw its storage from src. Allocates nothing.

inline ~string_t()

Returns the block, if one was taken.

string_t(const string_t&) = delete

Non-copyable — a copy is a failable allocation; use assign.

string_t &operator=(const string_t&) = delete

Non-assignable by copy; use assign.

inline string_t(string_t &&o) noexcept

Steal o's block; o is left empty, on the same source.

inline string_t &operator=(string_t &&o) noexcept

Release this string’s block, then steal o's.

inline bool assign(std::string_view s) noexcept

Replace the contents with s. s may view this string’s own bytes.

Reuses the block when it is big enough; otherwise takes a block of exactly s.size() + 1 bytes.

Return values:

false – The source refused — the string is unchanged.

inline bool append(std::string_view s) noexcept

Append s. s may view this string’s own bytes.

Return values:

false – The source refused — the string is unchanged.

inline bool reserve(std::size_t n) noexcept

Ensure room for n characters (plus the terminator) without growing again.

Return values:

false – The source refused — the string is unchanged.

inline void clear() noexcept

Make the string empty; the block is kept for reuse.

inline std::string_view view() const noexcept

The contents, valid until the next mutating call.

inline operator std::string_view() const noexcept

The contents as a view — the spelling every std::string_view parameter takes.

inline const char *c_str() const noexcept

The NUL-terminated contents; "" while no block has been taken.

inline std::size_t size() const noexcept

Length in bytes, excluding the terminator.

inline std::size_t capacity() const noexcept

Characters the current block holds, excluding the terminator.

inline bool empty() const noexcept

True when the length is zero.

inline block_source_t &source() const noexcept

The source this string draws from.

Friends

inline friend bool operator==(const string_t &a, std::string_view b) noexcept

Byte-wise equality with any view.

inline friend bool operator==(const string_t &a, const string_t &b) noexcept

Byte-wise equality of two strings.

inline friend std::strong_ordering operator<=>(const string_t &a, std::string_view b) noexcept

Byte-wise ordering against any view (the std::string_view ordering).

inline friend std::strong_ordering operator<=>(const string_t &a, const string_t &b) noexcept

Byte-wise ordering of two strings.

template<class K, class V, class Less = std::less<>>
class sorted_map_t

A key/value map over one sorted block_array_t, with failable growth (ADR-0083 Decision 2).

Lookup is a binary search, O(log n) with no pointer chasing; insert and erase shift the tail, O(n). That is the right trade for the core’s maps, which are built at registration time and read on every frame. A refused insert leaves the map and the caller’s arguments untouched.

Lookup is heterogeneous: with the default std::less<>, a map keyed by string_t is searched with a std::string_view and builds no key. A key is constructed only when an insert actually adds an entry.

Any growth may move the entries, so a pointer or iterator into the map is invalidated by every insert and erase.

Template Parameters:

• K – The key type; nothrow-movable. Its order must not change while it is in the map.

• V – The mapped type; nothrow-movable.

• Less – A strict weak order over keys, transparent when lookups use another type.

Public Types

using iterator = entry_t*

A mutable entry iterator, in key order (a plain pointer).

using const_iterator = const entry_t*

A read-only entry iterator, in key order.

Public Functions

inline explicit sorted_map_t(block_source_t &src, Less less = Less{}) noexcept

An empty map that will draw its storage from src. Allocates nothing.

template<class Q>
inline V *find(const Q &key) noexcept

The value under key, or null when absent.

template<class Q>
inline const V *find(const Q &key) const noexcept

The value under key (const), or null when absent.

template<class Q>
inline bool contains(const Q &key) const noexcept

True when key is present.

template<class KK, class ...Args>
inline emplace_result_t try_emplace(KK &&key, Args&&... args) noexcept

Insert {key, V(args...)} unless key is present.

The key and value are constructed only when the entry is added. On a refused insert neither key nor args are moved from, so the caller still owns them.

Returns:

{value, true} when added; {existing, false} when key was present; {nullptr, false} when the source refused (BACKPRESSURE).

template<class Q>
inline bool erase(const Q &key) noexcept

Remove the entry under key.

Return values:

false – It was absent.

inline bool reserve(std::size_t n) noexcept

Ensure room for n entries without growing again.

Return values:

false – The source refused — the map is unchanged.

inline void clear() noexcept

Remove every entry; the block is kept for reuse.

inline std::size_t size() const noexcept

Entry count.

inline bool empty() const noexcept

True when no entries are held.

inline iterator begin() noexcept

The first entry in key order.

inline iterator end() noexcept

Past the last entry.

inline const_iterator begin() const noexcept

The first entry in key order (read-only).

inline const_iterator end() const noexcept

Past the last entry (read-only).

inline entry_t &at(std::size_t i) noexcept

Entry i in key order, unchecked.

inline const entry_t &at(std::size_t i) const noexcept

Entry i in key order, unchecked (read-only).

inline void erase_at(std::size_t i, std::size_t n = 1) noexcept

Remove the n entries from i, shifting the tail down once. Precondition: i + n <= size().

template<class Q>
inline std::size_t lower_bound(const Q &key) const noexcept

Index of the first entry whose key is not less than key — where a range scan over a key prefix starts (size() when there is none).

struct emplace_result_t

What try_emplace answers.

Public Members

V *value

The entry’s value, or null when the source refused the insert.

bool inserted

True when this call added the entry; false when the key was present or the insert was refused.

struct entry_t

One entry: the key and its value, stored side by side.

Public Functions

template<class KK, class ...Args>
inline entry_t(std::in_place_t, KK &&k, Args&&... args) noexcept

Build the key from k and the value from args, in place.

Public Members

K key

The key; changing its order while it is in the map is undefined.

V value

The mapped value.

The bounded reference backend:

class pool_t : public tr::mem::mem_backend_t

A fixed-slot allocator over a caller-owned slab; alloc-or-nullptr.

Carves the slab into equal slots with the free list threaded through the slab (no auxiliary heap), so memory use is exactly the caller’s slab and exhaustion is a return value, not an OOM. The deterministic MCU choice.

Public Functions

pool_t(std::span<std::byte> slab, std::size_t slot_payload, std::size_t align = alignof(std::max_align_t)) noexcept

Carve slab (caller-owned; must outlive the pool) into slots.

Each slot holds a segment_t control block plus slot_payload usable bytes, payload aligned to align (a power of two). The slot count is whatever fits after aligning the slab base.

virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

One slot as a raw block — the block_source_t half (RFC-0028 §4.9): a request that fits a slot (header included) at no stricter alignment than the slab’s.

Return values:

nullptr – The pool is empty, or the request does not fit one slot.

virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a slot try_alloc handed out (the size is implied by the stride).

virtual view::segment_t *alloc(std::size_t size, alloc_hint_t hint = alloc_hint_t::NONE) override

Hand out the next free slot as a segment_t of size bytes — the one-block layout, the header at the slot’s head.

Return values:

nullptr – size exceeds the slot payload, or the pool is exhausted.

virtual void destroy(view::segment_t *seg) noexcept override

Return seg's slot to the free list (placement-destroying it).

inline virtual std::size_t alignment() const noexcept override

The alignment (bytes) this backend guarantees for allocated bytes.

inline virtual std::size_t max_segment_size() const noexcept override

The largest single segment this backend can produce.

inline virtual backend_tag tag() const noexcept override

The build-time-closed module-set tag (default UNKNOWN, ADR-0047 §2).

A backend that participates in the fast destroy dispatch overrides this to return its backend_tag; segments read it once at construction (like space). A backend that leaves the default is dispatched through its virtual destroy.

inline std::size_t capacity() const noexcept

Total slots — the effective ceiling, i.e. whatever fitted in the caller’s slab, never a constant.

inline std::size_t available() const noexcept

FREE slots.

Note

The one shipped free-polarity accessor, kept for compatibility. New code reads in_use — the unified vocabulary is used-polarity throughout (core/STYLE.md §Introspection), and available is the derived legacy name (capacity() - in_use()).

inline std::size_t in_use() const noexcept

Slots handed out and not yet returned — occupancy in the used-polarity vocabulary (core/STYLE.md §Introspection).

Public Static Attributes

static constexpr bool needs_cache_ops = false

No DMA cache maintenance (plain RAM slab).

static constexpr bool is_isr_safe = false

Unsynchronized free-list RMW — NOT safe concurrent with an ISR.

static constexpr bool is_nonblocking = true

alloc/destroy are O(1) free-list ops — no heap, no syscall.

static constexpr bool owns_bytes = true

Bytes are backend-managed (freed only on destroy) — durably storable.

The host slab pool

On a host build the process default root, tr::mem::default_root(), is a size-classed slab pool (ADR-0083 Decision 6, #1777). It asks the platform heap only for whole slabs, carves them into the classes of default_config_t::kSizeClasses, keeps up to kSlabClassCap fully free slabs per class and returns the rest, and derives three sub-pools: values (with a per-thread cache), tables and net. A graph built with no source draws from them; a graph given a source draws from that source alone. A build that binds kSlabPool = false keeps the platform heap.

template<::tr::lockable Sync, std::size_t N, bool kCounters = graph::kInstrumentCounters>
class slab_pool_t : public tr::mem::block_source_t

A size-classed slab pool: the host default adapter behind the allocation seam (ADR-0083 Decisions 4, 6 and 10, #1777).

A request takes the smallest class of the table that holds it and is served from a SLAB of that class: a block of the root’s memory, a power of two in size and aligned to it, carved into equal blocks. The root is asked for whole slabs only, never for one block, so the platform allocator’s own size classes stop deciding what a payload costs (the 1 KiB cliff of #1768 was one of them).

• Lazy carving. A fresh slab is carved one block at a time, so its untouched pages are address space, not resident memory.

• Release. Each class keeps at most cap fully FREE slabs, however many slabs are live. A slab whose last block comes back while its class already keeps cap free ones is released to the root at once; otherwise it is kept for the next burst, so an alloc/free pair at a slab boundary does not draw and release a slab on every turn. trim releases every fully free slab, on the caller’s schedule — the library keeps no timer to do it.

• Oversize. A request above the last class, or aligned past kHeaderBytes, is its own block from the root, at its own size, and goes back to the root when freed.

• Bounded. Over a bounded root, a pool_source_t on a caller slab, the pool is bounded by that slab and never touches a heap. That is the shape for sizes a PEER chooses (receive segments, WRITE payloads, label routes; #1646): an exact-size pool gives every distinct length a class of its own, where this one rounds it up to a row of the table. The root then sees only slab sizes, a few powers of two, so its exact classes are degenerate again, and a slab one class frees can serve another.

• Locking. One lock per class, of type Sync. Under tr::no_guard_t it is empty and costs nothing. The slab of a block is found by masking the block’s address, and the class from the sized release, so a block carries no header.

• Counting. The :stats census (stats) counts the SLAB bytes this pool takes from its root — not oversize blocks, which pass through uncounted — and is updated only on the slab path, never on the block path (core/STYLE.md §Introspection, counting doctrine 1). Per-class detail is class_stats, rounding waste included, compiled only with kCounters.

Template Parameters:

• Sync – The per-class lock, a tr::lockable.

• N – Rows in the size-class table.

• kCounters – Whether the per-class block counters of class_stats are kept.

Public Functions

inline constexpr slab_pool_t(const char *name, std::span<const std::size_t, N> classes, block_source_t &root, std::size_t slab_bytes = kSlabBytes, std::size_t cap = kSlabClassCap) noexcept

A pool over classes, drawing slabs from root.

Parameters:

• name – The census name ("values", "tables", "net").

• classes – The size-class table; slab_classes_valid must hold of it.

• root – Where slabs come from; it must outlive the pool.

• slab_bytes – The base slab size, a power of two of at least 4 KiB.

• cap – The fully free slabs a class keeps (at least 1).

inline ~slab_pool_t() override

Returns every slab to the root, live blocks or not: the pool’s blocks die with it.

inline std::size_t class_of(std::size_t bytes, std::size_t align) const noexcept

The class that serves bytes at align: the smallest row that holds it and keeps the alignment, or kNoClass.

A function of the two arguments alone, so release finds the class try_alloc chose.

inline std::size_t class_bytes(std::size_t i) const noexcept

The block size of class i.

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

A block of the smallest class that holds bytes; nullptr when the root refuses a slab.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a block try_alloc handed out, sized as asked.

inline std::size_t take(std::size_t i, void **out, std::size_t n, std::size_t request) noexcept

Take up to n blocks of class i under one hold of its lock — the refill of a thread cache, or one try_alloc.

Parameters:

request – The caller’s request, for the refusal census.

Returns:

The blocks written to out: n, unless the root refused a slab, then fewer (0 when not even one block could be served).

inline void give(std::size_t i, void *const *blocks, std::size_t n) noexcept

Return n blocks of class i under one hold of its lock.

inline void trim() noexcept

Release every fully free slab to the root, whatever the cap (ADR-0083 Decision 6).

The application calls it on its own schedule: after a burst, on a low-memory signal. Blocks a thread cache holds keep their slab.

inline virtual source_stats_t stats() const noexcept override

This pool’s census, in the core/STYLE.md §Introspection vocabulary.

in_use is the SLAB bytes held from the root: what the slabs cost the node, never less than their live blocks. An oversize block (past the last class) is NOT in it — it passes straight through to the root uncounted, so a node holding large values holds more than in_use says; its refusals are still counted. peak is its high-water mark, refused the requests answered nullptr because the root refused a slab, and largest_refused the biggest of those requests. capacity is 0: the pool caps retention, not demand. Sampled with relaxed loads (the snapshot-coherence clause).

inline slab_class_stats_t class_stats(std::size_t i) noexcept

Class i's detail (slab_class_stats_t), read under its lock.

inline std::size_t slab_bytes(std::size_t i) const noexcept

The slab size class i draws from the root.

inline void *oversize_alloc(std::size_t bytes, std::size_t align) noexcept

An oversize request: its own block from the root, at its own size. Public for a front end that has already consulted the test probe (the host value cache), so one request consumes it once.

Not rounded up to slabs or pages: rounding a value just past the last class up to whole 64 KiB slabs asked glibc for 128 KiB, its mmap threshold, and a 64 KiB heap_backend draw went from 26 ns to 2.5 us on bench-local; whole pages still moved the heap top past its trim threshold on every free. At its own size the block is what the platform allocator saw before the pool existed.

Nor is it in in_use: it passes straight through to the root, and counting it (one atomic add on the draw, one subtract on the release) cost a 64 KiB draw +8 ns on bench-local (20.9 -> 28.7 ns), more than the platform allocator’s own 26 ns before the pool. A refusal is still counted.

inline void oversize_release(void *p, std::size_t bytes, std::size_t align) noexcept

Return an oversize block, sized as oversize_alloc drew it.

Public Static Functions

static inline constexpr std::size_t classes() noexcept

Rows in the size-class table.

Public Static Attributes

static constexpr std::size_t kMinAlign = alignof(std::max_align_t)

The alignment every class block keeps: the platform allocator’s guarantee.

static constexpr std::size_t kHeaderBytes = 64

Bytes the slab header takes at the head of a slab; also the highest alignment a class block can be asked for.

static constexpr std::size_t kNoClass = N

class_of’s answer for a request no class serves (an oversize block).

static constexpr std::size_t kLookupBytes = 4096

Requests up to this size find their class by one table load.

class host_root_t : public tr::mem::block_source_t

The host default root (ADR-0083 Decisions 3 and 4, #1777): one root with the value, table and net sub-pools derived from it.

Each sub-pool is a slab_pool_t that draws whole slabs from the platform heap, so the host allocator sees slab-sized requests only. A graph_t constructed without a source uses this root (tr::mem::default_root()): its values come from values, its registration and container blocks from tables, and the router and transport defaults from net when the application injects nothing for them (Q21).

As a source of its own the root serves from tables, and its census is the three sub-pools’ together: what the node holds from the platform heap.

Process-wide and never destroyed, as the platform heap it replaces is: a value released after every static destructor has run still has somewhere to go.

Public Functions

inline host_values_t &values() noexcept

The value sub-pool (:stats.mem.values).

inline host_pool_t &tables() noexcept

The table sub-pool (:stats.mem.tables).

inline host_pool_t &net() noexcept

The net sub-pool (:stats.mem.net).

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

A block from the table sub-pool.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a block try_alloc handed out.

virtual source_stats_t stats() const noexcept override

The three sub-pools’ census, summed: slab bytes held from the platform heap.

peak is the SUM of the sub-pools’ own peaks, an upper bound on the root’s true high-water mark: the three need not have peaked at once. largest_refused is the largest of theirs.

void trim() noexcept

Release every fully free slab of all three sub-pools (this thread’s value cache first).

class host_values_t : public tr::mem::block_source_t

The VALUE sub-pool of the host root: the shared size classes behind a per-thread cache (ADR-0083 Decision 10, Q19).

A thread keeps a few free blocks of each class it uses and takes from them without a lock, refilling and spilling half a cache at a time under the class lock. The contention evidence is the one ADR-0060 Erratum 1 and ADR-0079 Amendment 2026-08-20 §3 measured: one shared locked pool collapses under many writers, where per-thread lists scale. A thread’s cache is returned to the shared classes when the thread exits.

There is one of these, inside host_root(): the per-thread cache is the process’s, so the object that owns it is never destroyed.

Public Functions

virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

A cached block of the class that holds bytes.

virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a block to this thread’s cache.

inline virtual source_stats_t stats() const noexcept override

The shared classes’ census (slab_pool_t::stats).

void trim() noexcept

Return this thread’s cache to the shared classes, then slab_pool_t::trim.

inline host_pool_t &shared() noexcept

The shared classes under the cache.

struct slab_class_stats_t

One size class’s detail, for tuning the size-class table (kInstrumentCounters).

Not part of :stats (RFC-0010 Amendment 3 keeps per-class detail out of the wire surface); a C++ accessor on a build that opts into the counters.

Public Members

std::size_t bytes = 0

The class’s block size.

std::size_t slabs = 0

Slabs the class holds from the root now.

std::size_t live = 0

Blocks out of the class: handed out, or parked in a thread cache.

std::size_t released = 0

Slabs given back to the root, by the cap or a trim.

std::size_t rounding = 0

Bytes the live blocks lose to rounding up: the class size less what each request asked, summed over the blocks slab_pool_t::try_alloc handed out (#1646). A block a thread cache holds is live but not in it.

constexpr bool tr::mem::slab_classes_valid(std::span<const std::size_t> classes) noexcept

Whether classes is a usable slab-pool table: non-empty, strictly ascending, at most 255 rows, and every row a multiple of alignof(std::max_align_t).

host_root_t asserts it of tr::graph::default_config_t::kSizeClasses, so a fragment that binds a table the pool cannot serve fails the build.

host_root_t &tr::mem::host_root() noexcept

The process-wide host default root (host_root_t).

Constant-initialized storage that is never destroyed. On a build whose kSlabPool is false nothing in the library uses it.

block_source_t &tr::mem::default_root() noexcept

The default root a graph_t takes when it is handed no source (ADR-0083 Decision 4, #1777): the host root (mem_slab_pool.hpp) where kSlabPool is true, the platform heap (heap_source) otherwise.

block_source_t &tr::mem::value_source() noexcept

The default VALUE sub-pool (:stats.mem.values): the host root’s, from a per-thread cache, where kSlabPool is true; the platform heap otherwise.

The source a value made outside any graph draws from (tr::graph::value_ref_t::make), and the one heap_backend draws its segments from.

block_source_t &tr::mem::table_source() noexcept

The default TABLE sub-pool (:stats.mem.tables), on the same terms as value_source.

block_source_t &tr::mem::net_source() noexcept

The default NET sub-pool (:stats.mem.net), on the same terms as value_source, that is: the block source a router or link draws from when the application injects none of its own (ADR-0083 Q21).

mem_backend_t &tr::mem::net_backend() noexcept

The segment backend over net_source, the default receive, flatten and egress backend of the router and the links when the application injects none (Q21).

heap_backend itself where kSlabPool is false.

The placement module

One module owns how a segment’s header and payload sit in the blocks a backend draws (ADR-0083 Decision 5, #1775): the header size, the padding rule, a pool slot’s stride, the inline value’s layout and the receive-loan reserve. Every backend asks it; none keeps its own recipe. A segment is one block on every backend: the padded header, then the payload. The #1768 split of a large heap segment into two blocks is deleted (#1777), because the heap backend’s blocks now come from the host slab pool’s size classes, where a 1 KiB value is one class block and the platform allocator sees only whole slabs.

constexpr std::size_t tr::mem::segment_block_align(std::size_t align) noexcept

The alignment a segment block is drawn at for a backend that guarantees align: the stricter of align and the header’s own (RFC-0028 §4.9).

constexpr std::size_t tr::mem::segment_header_bytes(std::size_t align) noexcept

Bytes the view::segment_t header occupies at the head of a one-block segment, padded so the payload after it starts at segment_block_align(align).

constexpr std::size_t tr::mem::segment_block_bytes(std::size_t size, std::size_t align) noexcept

The whole block a one-block segment of size payload bytes draws at align.

constexpr std::size_t tr::mem::inline_block_bytes(std::size_t prefix, std::size_t len) noexcept

The block an INLINE value of len bytes occupies: the prefix, the embedded segment, and the bytes.

inline view::segment_t *tr::mem::place_segment(mem_backend_t *owner, void *block, std::size_t size, std::size_t align) noexcept

Place a view::segment_t over the one block block, reclaimed by owner: the header at the head, size payload bytes after it (a null, empty span for 0).

block was drawn as segment_block_bytes(size, align) at segment_block_align(align).

The heap backend and the space tags

class heap_backend_t : public tr::mem::mem_backend_t

The process-default per-value backend: owns its blocks, returns them and the segment_t control block on destroy.

Exposed here (rather than TU-local) so the module-set destroy dispatch (backend_set.cpp, ADR-0047 §2) can devirtualize its release; a final class, so the qualified call in that switch is a direct call.

Where the bytes come from (#1777)

Every block is drawn by detail::value_block_alloc: on a host build (kSlabPool), the value sub-pool of the host root (mem_slab_pool.hpp), from this thread’s cache, so the platform allocator is asked for whole slabs and never for a segment; elsewhere the platform heap, through heap_source_t::acquire. Both are direct calls, not the virtual draw #873 phase 2 measured at +22.7 % on the hazard domain: this backend is the process default on the hottest allocation path in the library.

How many draws a segment costs (RFC-0028 §4.9, #1777)

ONE: the padded header and the payload share a block (RFC-0028 slice 10), at every size. The two-block split above glibc’s 1,032 B tcache ceiling (#1768) served only the per-value heap draw this backend no longer makes on a host. The slab pool’s classes have no such cliff: a 1024 B value’s 1072 B block is one 1152 B class block, from the same cache a 1000 B one comes from. bench_forward_heap’s allocs= pins count the small-value case.

Public Functions

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

One block of this build’s per-value draw — nothrow.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

Return a block try_alloc drew, sized.

inline virtual view::segment_t *alloc(std::size_t size, alloc_hint_t) override

A segment is ONE block, the header and the payload together (RFC-0028 §4.9).

One block is the producer-own row of bench_lean_value_path (slice 10). Draws straight from detail::value_block_alloc rather than through the virtual try_alloc, so the hot path pays no virtual call.

inline virtual void destroy(view::segment_t *seg) noexcept override

Return the block alloc drew — sized, as drawn.

inline virtual std::size_t alignment() const noexcept override

The payload follows a header padded to kBlockAlign.

inline virtual backend_tag tag() const noexcept override

The build-time-closed module-set tag (default UNKNOWN, ADR-0047 §2).

A backend that participates in the fast destroy dispatch overrides this to return its backend_tag; segments read it once at construction (like space). A backend that leaves the default is dispatched through its virtual destroy.

Public Static Attributes

static constexpr std::size_t kBlockAlign = segment_block_align(alignof(std::max_align_t))

The block alignment a heap segment is drawn at.

static constexpr bool needs_cache_ops = false

No DMA cache maintenance (host RAM).

static constexpr bool is_isr_safe = false

A class lock or a slab draw from the heap — not ISR-safe.

static constexpr bool is_nonblocking = false

A class lock or a slab draw may wait or syscall (#928).

static constexpr bool owns_bytes = true

Owns its blocks — durably storable.

mem_backend_t &tr::mem::heap_backend() noexcept

The process-wide heap backend (function-local static — no init-order trap).

enum class tr::mem::mem_space_t : std::uint8_t

The address space a backend’s bytes live in.

HOST bytes are CPU-addressable; DEVICE bytes are not (e.g. GPU/accelerator device memory — docs/adr/0024). The codec must never CPU-dereference a DEVICE link: such a segment may back only an opaque VALUE payload, with the header/trailer kept in a HOST segment (a heterogeneous host+device rope).

Values:

enumerator HOST

CPU-addressable bytes.

enumerator DEVICE

Non-CPU-addressable bytes (GPU/accelerator); codec must not deref.

enum class tr::mem::backend_tag : std::uint8_t

Which build-time-closed backend a segment came from — the module-set tag (ADR-0047 §2).

A segment carries its backend’s tag so the per-segment-release destroy dispatch (segment_ptr_t::reset → destroy_dispatch) is a switch → devirtualized direct call rather than a vtable indirect — foldable to a single direct call when a target links only one backend. An unrecognized tag (UNKNOWN, or any backend outside the fast set — every out-of-core device backend is) routes to the backend’s virtual destroy, so dispatch is correct regardless.

The set is closed over the backends core/ itself compiles. A vendor backend from the backends/ tier does not get an enumerator: it is identified by its mem_backend_t object (register_device_backend), which is what destroy already routes on, so core never has to name it.

Values:

enumerator UNKNOWN

No fast-path tag → virtual destroy fallback.

enumerator HEAP

mem_heap (mem_heap.hpp).

enumerator POOL

mem_pool (mem_pool.hpp).

enumerator BORROWED

mem_borrowed (mem_borrowed.hpp).

enumerator BORROWED_DEVICE

mem_borrowed device-space variant.

void tr::mem::destroy_dispatch(view::segment_t *seg) noexcept

Reclaim seg through its backend — the module-set destroy dispatch (ADR-0047 §2), called by segment_ptr_t::reset at refcount zero.

Switches on the segment’s backend_tag to a devirtualized direct call for a linked fast-set backend, and falls back to the backend’s virtual destroy for any other tag, so the result is identical to seg->backend->destroy(seg) for every backend. Defined in backend_set.cpp (the one TU that sees the concrete backend types), keeping this L0 seam free of an upward dependency.

bool tr::mem::transfer(view::segment_t *seg, std::span<std::byte> host, io_dir_t dir) noexcept

Move host.size() bytes between segment seg and host memory host in direction dir — the module-set host↔device transfer (ADR-0047 §2), bracketed by the backend’s cache hooks.

The single tag-dispatched byte-mover the codec routes a copy through, which replaced the vendor-named per-device copy pair the module set retired:

• io_dir_t::CPU_TO_DEVICE copies host into seg (host is the source);

• io_dir_t::DEVICE_TO_CPU copies seg out to host (host is the sink).

A host-addressable backend transfers with a memcpy, bracketed by before_io/after_io only when its static constexpr needs_cache_ops trait is set — so a cacheless backend (every one today) folds the hooks away at compile time (they are the traits’ first in-tree consumer, review finding #8). A DEVICE-space segment takes the registry arm instead: its backend’s register_device_backend hook, or false when nothing is registered for it — no host arm ever sees a pointer the CPU may not dereference (#928). That is where after_io gets its first caller, in the backends/ tier module that owns the device copy (docs/adr/0024). Defined in backend_set.cpp (the module-set TU).

Parameters:

• seg – The segment to read from or write to; nullptr yields false.

• host – CPU-addressable bytes; a .size() larger than seg's yields false.

• dir – Which way the bytes move (also the cache-hook direction).

Return values:

false – Null segment, an over-long host, or a device copy failure.

Shared pools and their synchronization policy

A pool shared by more than one thread needs a guard, and the guard is a compile-time parameter rather than a runtime flag so a single-threaded target pays nothing for it. synchronized_pool_t<Sync> takes any tr::guard (the one trait the LKV slot reads too, RFC-0028 §5.6) and defaults to the build’s guard_t, so synchronized_pool_t<> is the spelling for the common case on every target. A guard that declares may_spin is refused at the instantiation on a build that sets kSpinWaitSafe = false.

The guard vocabulary lives in the layer-neutral tr namespace (#1703), because the memory layer and the graph layer both bind it: libtracer/guard.hpp is freestanding (the tr::lockable and tr::guard concepts, tr::no_guard_t, tr::guard_scope_t and tr::rmw_counter_t), and libtracer/guard_mutex.hpp holds the hosted tr::mutex_guard_t. The tr::graph spellings and tr::mem::sync_none_t are aliases for one release.

The pool_source_t seam takes any tr::lockable, defaulting to tr::no_guard_t, and has one deliberate non-answer: sync_mutex_t lives in a separate header because the L0 seam is compiled into a freestanding footprint sentinel where <mutex> does not exist, and because a mutex is the right instrument only for a source shared at wiring frequency. It is not a way to make a per-frame source thread-safe — see failable allocation and backpressure for what a shared free list costs under contention.

template<class G>
concept guard

#include <guard.hpp>

The one critical-section contract a build binds per target (RFC-0028 §5.5, §5.6).

A guard is a LOCK OBJECT: lock() / unlock() open and close one short section. The same type serves every critical section the library opens —

• the single-writer LKV slot takes the guard that covers the slot’s address, G::for_address(slot), around its pointer swap and its handle copy;

• tr::mem::synchronized_pool_t<G> owns one G and takes it around its free-list edit;

• tr::rmw_counter_t takes it around its bump on a core with no atomic read-modify-write.

The build binds it once, as tr::graph::config_t::guard_t. It was spelled reader_guard_t until #1703, a name that undersold it: the guard serializes writers, the pool and the write-sequence bump, not only readers.

The traits are what a caller reasons with, in place of prose:

• is_isr_safe — the section may be entered from an interrupt;

• is_nonblocking — entering never waits on the OS (no heap, no syscall, no sleep);

• may_spin — a contender can SPIN-wait on the holder, unboundedly. Refused where kSpinWaitSafe is false (#1158, #1618): a spinner that outranks the holder never yields the CPU the holder needs.

• name — a stable name, which a pool built over the guard reports as its own.

template<class L>
concept lockable

#include <guard.hpp>

The lock-object half of the contract: lock() and unlock(), both noexcept.

What a pool that owns its one lock asks of it (tr::mem::pool_source_t<Sync>, ADR-0067: the seam “asks only for `lock()`/`unlock()`”). A tr::guard is always one.

template<class T, class G, bool kNative = std::atomic<T>::is_always_lock_free, std::memory_order kOrder = std::memory_order_seq_cst>
class rmw_counter_t

A wrapping counter, bumped by many writers and read without a lock, whose bump is chosen at compile time from what the target’s hardware can do (#1621, RFC-0028 D6).

• Native (std::atomic<T>::is_always_lock_free): the bump is one fetch_add — lock xadd on x86-64, amoadd.w on rv32imac (ESP32-C6), ldrex/strex on Cortex-M3/M4/M7 (both cores of an STM32H7), s32c1i on Xtensa. No call, no masked interrupt.

• Guarded (no atomic RMW: rv32imc such as the ESP32-C3, Cortex-M0/M0+): the bump is a load and a store inside one section of G, the build’s ONE critical-section type (config_t::guard_t, RFC-0028 §5.5) — an interrupt mask on a single-core chip, and on a dual-core one the cross-core lock G already is for the LKV slot. Plain aligned loads and stores stay single instructions on these targets (rv32imc compiles them to fence; lw; fence), so only the bump pays, and it pays the same section libatomic would open for it, without the call.

A guarded bump that skipped the guard would be unsound with two writers: one that loaded n and was preempted can store n + 1 after a later writer’s n + 2, rewinding the counter to a value a reader already snapshotted, and that reader then misses the later change. The guard serializes the writers; the reader needs no guard, because every store is a whole aligned word.

Both bindings give the bump, the load and the preset the ordering kOrder names. The default is seq_cst, so a caller’s Dekker pair (bump, then read a flag; set the flag, then read the counter) holds on either: the write sequence relies on it. A diagnostic tally that orders nothing names relaxed (core/STYLE.md §Introspection, rule 5), so its native bump is the plain AMO (amoadd.w, not amoadd.w.aqrl, on rv32imac; ldadd, not ldaddal, on aarch64). The counter only ever moves by one and wraps at T’s width, so it is for EQUALITY tests (now != then), never <.

Template Parameters:

• T – An unsigned integer, at most a machine word wide.

• G – The build’s critical-section guard (a tr::guard), taken only by the guarded binding.

• kNative – Which binding. Defaults to what the target supports; a test names it to drive the guarded binding on a host that has atomic RMW.

• kOrder – The memory order of every access. seq_cst unless the count orders nothing.

Public Functions

inline void bump() noexcept

Move the counter on by one, at kOrder, wrapping at T’s width.

inline void bump(const void *anchor) noexcept

Move the counter on by one; the guarded binding takes the guard covering anchor rather than the one covering the counter.

For a counter whose bump is sometimes made inside a section another object already opened (bump_in_section): every guarded bump of ONE counter must take the SAME guard, so the caller names that object’s address here too. Two anchors on one counter are two locks, and the writers would no longer exclude each other. The native binding ignores anchor.

inline void bump_in_section() noexcept

The guarded bump’s body, for a caller that ALREADY holds the guard every other bump of this counter takes (#1715): a load and a kOrder store, no section.

The precondition is the whole of its soundness: the load + store is atomic against other bumpers only because they all serialize on that one guard. The LKV slot’s fused publish calls it from inside its own section, which covers the anchor the vertex names on its other bumps. Guarded binding only: a native counter has no section to share.

inline T load() const noexcept

The current count, at kOrder. Lock-free on both bindings.

Public Static Attributes

static constexpr bool is_native = kNative

Whether the bump is one hardware RMW (true) or a guarded load + store.

template<::tr::guard Sync = graph::guard_t>
class synchronized_pool_t : public tr::mem::mem_backend_t

A thread-safe pool_t whose SYNCHRONISATION IS A COMPILE-TIME POLICY (ADR-0060 §2), guarding the O(1) free-list with Sync.

Any mem_backend_t injected at a shared seam MUST be thread-safe: a segment self-routes its reclaim on whatever thread drops the last ref — typically a reader/subscriber or transport receive thread, concurrent with a writer’s alloc (ADR-0060 §2; the same obligation holds for graph_t’s value_backend, the router’s flat, and transport_vertex_t’s rx_backend). A single thread-safe pool (never per-stripe sharding, which removes no race and adds partition imbalance) is the answer.

The policy is a tr::guard — the SAME critical-section trait the LKV slot binds (RFC-0028 §5.5, slice 10), so a target states its concurrency model once: graph::guard_t is the default, which is tr::mutex_guard_t on a host (one RMW to take; a contender naps rather than spins) and the interrupt-masked tr::esp::critical_guard_t on an ESP-IDF chip (tr::esp::critical_pool_t). The target knows its concurrency model at BUILD time, so the choice is a template argument, not a runtime knob: no branch, no vtable, no per-alloc indirection on a ~120 ns operation. The many-core lock-free index+tag CAS upgrade (the free list is already index-based) stays the recorded ADR-0060 §2 follow-up.

This is opt-in construction only — no seam defaults to it. heap_backend() remains the default everywhere; a target that wants its receive/value bytes inside its own slab constructs one of these and injects it.

Composition over pool_t: a freshly-alloc’d segment is re-pointed to this with a UNKNOWN tag, so destroy_dispatch routes reclaim through the virtual (locked) destroy here instead of the devirtualized POOL fast path (which would bypass the lock). pool_t::destroy recovers the slot from the segment’s slab offset, so the re-point is invisible to the inner pool. The re-point touches only the just-allocated segment, which no other thread can observe until the caller publishes it.

Public Functions

inline synchronized_pool_t(std::span<std::byte> slab, std::size_t slot_payload, std::size_t align = alignof(std::max_align_t)) noexcept

Carve slab into slot_payload-byte slots (see pool_t), thread-safe.

inline virtual void *try_alloc(std::size_t bytes, std::size_t align) noexcept override

pool_t::try_alloc inside Sync's critical section.

inline virtual void release(void *p, std::size_t bytes, std::size_t align) noexcept override

pool_t::release inside Sync's critical section.

inline virtual view::segment_t *alloc(std::size_t size, alloc_hint_t hint = alloc_hint_t::NONE) override

pool_t::alloc inside Sync's critical section; reclaim re-routed here.

inline virtual void destroy(view::segment_t *seg) noexcept override

pool_t::destroy inside Sync's critical section.

inline virtual std::size_t alignment() const noexcept override

The alignment (bytes) this backend guarantees for allocated bytes.

inline virtual std::size_t max_segment_size() const noexcept override

The largest single segment this backend can produce.

inline virtual backend_tag tag() const noexcept override

UNKNOWN so destroy_dispatch takes the virtual (locked) destroy, not the devirtualized POOL fast path that would bypass the critical section.

inline std::size_t capacity() const noexcept

Total slots (delegated to the inner pool_t).

inline std::size_t in_use() const noexcept

Slots handed out and not yet returned (delegated to the inner pool_t).

The wrapper used to forward capacity and STOP there, so wrapping a bounded resource in its thread-safe form lost half its census: a host could read the ceiling it had injected but not how much of it was gone — on precisely the pool that is shared, i.e. the one whose occupancy is hardest to reason about (#1503 finding 4).

Read without taking Sync, per the snapshot-coherence clause (core/STYLE.md §Introspection): locking here would put a critical section on a ~120 ns free-list op in order to serve a diagnostic, and the intended reading is the difference between two samples, not an instant.

inline std::size_t available() const noexcept

FREE slots (delegated). Free-polarity legacy spelling — new code reads in_use; see pool_t::available.

Public Static Attributes

static constexpr bool needs_cache_ops = false

Plain RAM slab.

static constexpr bool is_isr_safe = Sync::is_isr_safe

Whatever the sync policy guarantees.

static constexpr bool is_nonblocking = Sync::is_nonblocking

The pool’s section is O(1); the WAIT is the policy’s fact, so this forwards it rather than asserting it (#928).

static constexpr bool owns_bytes = true

Backend-managed, durably storable.

class sync_mutex_t

The hosted synchronization policy — a plain std::mutex.

For a source shared across threads at wiring frequency: a graph’s control source, where registration runs once per connection and an uncontended mutex is unmeasurable.

It stays a std::mutex rather than becoming an alias of the host guard tr::mutex_guard_t (#1703): a contender here parks in the kernel instead of napping, and the shared-pool arms of the store-topology benches measure exactly this type.

Note

Also not for a single-core FreeRTOS target’s per-frame path: a blocking mutex there invites the priority inversion ADR-0063 erratum 1 records. Such a target supplies an interrupt-disable policy of its own; the seam only asks for lock()/unlock() (tr::lockable, guard.hpp).

Warning

Do NOT reach for this to make a per-frame source thread-safe. ADR-0060 erratum 1 measured a shared free-list pool collapsing to ~1/15 of its single-thread rate on a 12-core host while the platform heap scaled; guarding the shared list is the problem, not the guard’s flavour. Give each receiver its own pool_source_t with the default tr::no_guard_t instead.

Public Functions

inline void lock() noexcept

Acquire. noexcept by policy: a mutex that cannot lock is a programming error.

inline void unlock() noexcept

Release.

Device memory — the registration seam

Core keeps the interface; the vendor backend lives in the backends/ tier. A DEVICE-space backend registers the pair {backend, transfer hook} and tr::mem::transfer routes that backend’s segments to it — the L0 mirror of tr::net::transport_vertex_t::register_transport_type (ADR-0024 Amendment 1). The key is the backend object, not the space tag, so a second vendor (ROCm, an NPU, dmabuf) plugs in without adding a name — or an enumerator — to core. A backend nobody registered gets a clean false, exactly as every unrecognized device segment did before the registry existed.

using tr::mem::device_transfer_fn_t = bool (*)(view::segment_t *seg, std::span<std::byte> host, io_dir_t dir) noexcept

The device byte-move a DEVICE-space backend registers with register_device_backend — transfer’s out-of-core arm.

Same contract as transfer, narrowed to one backend’s segments: move host.size() bytes between seg and host in direction dir, false on refusal. A plain function pointer, not a std::function: the seam must cost a pointer and never allocate (ADR-0047 §2).

bool tr::mem::register_device_backend(const mem_backend_t &backend, device_transfer_fn_t fn) noexcept

Register fn as the byte-mover transfer routes backend's DEVICE-space segments through.

The L0 mirror of tr::net::transport_vertex_t::register_transport_type: a module outside core supplies the value, the composition root wires it in, and core never names the module (docs/adr/0024 Amendment 1; the module seam is docs/adr/0043 §1). The backends/ tier holds the first in-tree caller.

Keyed by the backend object, not by mem_space_t — DEVICE is one enumerator shared by every accelerator, and the segment’s backend pointer is already the identity destroy routes on — so a second vendor plugs in without adding a name to core.

Bounded and allocation-free: the table holds tr::mem::kDeviceBackendSlots entries (config.hpp), so registration cannot fail for lack of heap, only for lack of a slot. Registering the same backend twice replaces its hook (insert_or_assign semantics), so a backend and its hook can never disagree.

Note

Call at setup, before frames flow, from one thread — the same contract register_transport_type carries. Concurrent lookups by transfer are safe against a completed registration.

Return values:

false – fn was null, or the table is full — nothing was registered.

The GPU module’s own entry points (built only in the tier):

mem_backend_t &tr::mem::cuda_backend() noexcept

The process-wide CUDA device backend (cudaMalloc/cudaFree; space() == DEVICE).

Constructing it also registers it (see register_cuda_backend), so a caller that only ever allocates — tr::view::cuda_alloc — can never meet a tr::mem::transfer that does not know where to route its segments.

bool tr::mem::register_cuda_backend() noexcept

Register cuda_backend with core’s device-backend registry, so tr::mem::transfer routes its DEVICE segments to cuda_transfer.

The tr::net::quic_transport_factory of this tier: the module supplies the value, the composition root (or, here, cuda_backend’s own first construction) wires it in, and core never names CUDA. Idempotent — calling it twice replaces the same slot’s hook.

Return values:

false – Core’s bounded table is full (tr::mem::kDeviceBackendSlots).

bool tr::mem::cuda_transfer(view::segment_t *seg, std::span<std::byte> host, io_dir_t dir) noexcept

The device byte-move behind tr::mem::transfer for a CUDA (DEVICE) segment: cudaMemcpy in direction dir, bracketed by the backend’s cache hooks (after_io == the CUDA stream barrier).

Declared here but defined in mem_cuda.cpp so cudaMemcpy stays TU-local. It is what register_cuda_backend hands core. Not called directly — use tr::mem::transfer, which routes this backend’s segments here.

The seam

        classDiagram
    class mem_backend_t { <<interface>> +alloc() +destroy() +before_io() +after_io() +alignment() }
    mem_backend_t <|-- heap_backend_t
    mem_backend_t <|-- borrowed_backend_t
    mem_backend_t <|-- pool_t
    mem_backend_t <|-- YourBackend
    segment_t --> mem_backend_t : backend*
    note for YourBackend "bind a DMA ring,\nlwIP pbuf, MMIO, …"
    

Consequences

• The same protocol runs against a heap, a caller-sized MCU slab or a live register, because the substrate is selected by binding a backend rather than by a build variant of the core.

• Memory use with mem_pool is exactly the caller’s slab: the free list is threaded through the slab, so there is no auxiliary heap allocation, and exhaustion is an alloc returning nullptr rather than an OOM.

• mem_borrowed puts a segment over bytes the caller already holds, so live data reaches the wire with no copy and no CRC imposed; the cost is that those bytes are outside libtracer’s lifetime control.

• A substrate that cannot allocate at all is still bindable, because alloc is permitted to return nullptr unconditionally — MMIO windows and hardware FIFOs bind as read-only borrowed segments.

• Two seams rather than one means two failure contracts to hold in mind: a mem_backend_t failure is a refcounted-segment allocation that failed, a block_source_t failure is a single-owner block that failed. Neither throws.

Pitfalls

• A raw segment_t* that is never adopted leaks. alloc hands back a pointer at refcount 1 and the backend does not track it; the value is only safe once segment_ptr_t::adopt owns it. The tr::view helpers (heap_alloc, borrow, borrow_const) exist so that the common paths cannot get this wrong.

• Borrowed bytes must outlive every segment over them. borrowed_backend_t::destroy deletes the control block and nothing else (core/include/libtracer/mem_borrowed.hpp:borrowed_backend_t::destroy), so a borrow over a stack buffer or a scratch frame becomes a dangling read the moment that storage goes away. Durable storage of a value wants an owning backend.

• bump_source_t is scope-lifetime only. Blocks carved from its buffer are never individually reclaimed, so a bump source wired as a long-lived seam fills monotonically and then refuses everything. Construct it per operation, or reset it between operations; a long-lived bounded seam wants pool_source_t, which recycles.

• A bump_source_t buffer is not a hard bound by default. Its upstream defaults to heap_source(), so overflow spills to the platform heap. Passing null_source() as the upstream is what makes the buffer the limit and turns overflow into a rejection.

• source_resource_t is placement, not failability. The std::pmr adapter draws its bytes from an injected block_source_t, but std::pmr’s only exhaustion signal is a throw — so its boundary is std::bad_alloc, and std::abort() under -fno-exceptions. Do not move a peer-provoked store onto a std::pmr container because the adapter exists; migrate it onto block_array_t instead. The adapter runs one direction only, and the reverse (a memory_resource used as a block_source_t) must never be added.

• block_source_t::release is sized. The bytes and align passed to release must match the originating try_alloc call — that is what lets a bump or pool source carry no per-block header. A mismatched pair corrupts the source’s accounting rather than failing loudly.

See: segment, views, interface map, failable allocation and backpressure.