The configuration space¶
Scope: the build configuration of the C++23 reference implementation under
core/— which knobs exist, what each costs on a given target, and which constants are not knobs. Not the standard: the spec constrains bytes on the wire, not build systems, and a second implementer needs none of this. Every number here is a measurement with its target and flags named. Standard-level companions:../../reference/10-module-catalog.mdfor module responsibilities,../../reference/12-deployment-profiles.mdfor which combinations serve which deployments.
Framing¶
A libtracer node is not one artifact with runtime switches. It is a chosen set of modules, sized for one target, and both choices are made at build time. That is a deliberate consequence of serving two targets that share no sensible middle: a single-core microcontroller with tens of kilobytes of RAM, and a many-core host. A runtime flag would force both to carry the other’s cost.
The configuration space therefore has exactly three kinds of axis, and no fourth:
axis kind |
what it selects |
mechanism |
|---|---|---|
module set |
which translation units compile at all |
CMake |
buffer sizes |
how big the fixed tables are |
|
policy types |
which implementation of a named seam is bound |
|
All three are plain C++ or build-system state — never a preprocessor feature macro (ADR-0068 — build configuration is plain C++). A knob is a constant or an alias, so a wrong value is a compile error in the build that set it rather than a silent behavioural fork between translation units.
The sizes and policies are members of one named type, default_config_t
(core/include/libtracer/config.hpp:default_config_t), bound once by using config_t = default_config_t;
(core/include/libtracer/config.hpp:config_t). An application declares its own by inheriting and overriding what differs (core/include/libtracer/config.hpp:struct my_node_config_t : default_config_t {):
struct my_node_config_t : tr::graph::default_config_t {
static constexpr std::size_t kCacheLineBytes = 0; // single-core: no false sharing
};
using config_t = my_node_config_t;
Inheriting means a knob added later does not break the preset — it inherits the new default
rather than failing to compile. The rest of the library names the derived spellings re-exported
below the traits type (core/include/libtracer/config.hpp:config_t), each of which is exactly its traits member, so introducing
config_t moved no call site.
It is bound once, not threaded as a template parameter, and ADR-0070 — configuration is a named traits type records why with measurements: a parameter produces byte-identical machine code, so it buys no latency; its one unique capability — two configurations in one binary — would fork the process-global stripe and hazard tables and so cost the RAM the configuration exists to save; and an app-declared traits type cannot reach the library’s out-of-line translation units anyway.
The delivery mechanism¶
One header, <libtracer/config.hpp>, carries every size and policy. It is ordinary
hand-written C++ and the only place a default is spelled — a consumer with no build-system
participation at all (a raw -Icore/include compile, a vendored source drop, the Cortex-M0
footprint gate) gets stock settings and builds.
A target that wants non-default values supplies an override fragment,
libtracer/config_override.hpp, earlier on the include path. config.hpp picks it up
automatically and uses the config_t it binds. One file per build is what makes a size
agreement impossible to break: a bare -D set on some TUs and not others would be an ODR
violation the linker is not obliged to notice.
The fragment inherits default_config_t and states only what differs:
// libtracer/config_override.hpp
namespace tr::graph {
struct esp_config_t : default_config_t {
static constexpr std::size_t kCacheLineBytes = 0;
};
using config_t = esp_config_t;
} // namespace tr::graph
That inheritance is why there is no drift gate any more: a knob added to config.hpp reaches
every override with its new default, because no override restates the knobs it does not change.
Earlier revisions generated the whole header from a CMake template and byte-compared a default
render against the checked-in copy — a gate that existed only because the defaults were spelled
twice (ADR-0068 §Erratum 1).
The consequence worth stating plainly: the values are chosen by whoever builds libtracer’s
sources. For every real integration path that is the application’s own build — the ESP-IDF
component, PlatformIO, a FetchContent/add_subdirectory CMake consumer — so an application
sets its own values. A consumer linking a prebuilt libtracer archive gets the values that
archive was built with; changing them means rebuilding it.
The module set¶
Selection is by which sources compile, not by conditional compilation inside them. A disabled transport contributes no TU and no registration call, so nothing in the image references it. Every option defaults ON — modularity is opt-out, so the default build is a full node.
CMake option |
module |
ESP-IDF counterpart |
what dropping it removes |
|---|---|---|---|
|
FWD routing plane ( |
none — the component has no counterpart |
inter-node forwarding; a pure in-process graph still works |
|
|
|
— |
|
|
|
— |
|
|
|
— |
|
|
|
— |
|
|
none |
off by default |
LIBTRACER_WITH_CUDA is gone (#1381): the GPU backend is a tier module with its own CMake
project, backends/cuda/, so “do I want it” is answered by whether you configure that directory
and link libtracer_cuda — not by a core option, and not by a compile definition that used to
reach every consumer TU of a CUDA-enabled build. It plugs into core through
tr::mem::register_device_backend, the memory layer’s mirror of register_transport_type.
The always-compiled core is the L2/L3 wire codec, the L0/L1 substrate, path, and the L4
graph runtime. See 10-module-catalog.md for what each module is
responsible for and 12-deployment-profiles.md for the profiles
these combinations serve.
The socket transports sharing one dependency (posix_endpoint) is handled by the build, not
by the modules: it compiles when any of them is on.
One option, two implementations. LIBTRACER_TRANSPORT_WS names the WebSocket module,
not a fixed pair of types, and on ESP-IDF the target decides which pair it builds. A chip
target gets the IDF-native links (httpd_ws_link_t, esp_ws_client_link_t) and does not
compile transport_ws_* at all, so no ws entry is registered in the built-in catalog there
and a kind=ws SPEC with no staged link answers SCHEMA_NOT_FOUND; the linux target and
every non-IDF build get the portable pair. This is still selection-by-which-TU-compiles — the
same rule as socketcan_link.cpp versus its stub — and it is a correctness split, not a size
one: the portable pair’s gather egress asks sendmsg for MSG_NOSIGNAL, which lwIP rejects
with EOPNOTSUPP, so on silicon it silently drops every data frame. See
the ESP-IDF integration README.
The one module that is not a TU list — kBusLinks¶
Every module above is selected by which sources compile. The bus module — ADR-0044’s
peer-named addressing tier: a mount’s bus shape, per-peer resolution, in-band peer enumeration,
the peer-named receiver and the two peer-lifecycle notifiers — cannot be, and is the deliberate
exception (#375). Whether a tcp or ws
listener is peer-named is a wiring-time choice made inside a TU that a bus-less target still
compiles for its point-to-point half, so there is no file to leave out. It is therefore stated as
a configuration member, kBusLinks, and reached through one gate, tr::net::bus_of, so the
closure is a compile-time fold at every consumer rather than a run-time branch. Measured on rv32
(-Os -fno-exceptions -fno-rtti, GCC 15.2, per-TU .text), binding it false removes 1,400 B
from fwd_router.cpp and 678 B from transport_vertex.cpp — 2,078 B of flash, 0 B of .bss, because
the tier is code and per-instance state rather than a static table. false is the default since
v0.17.0 (#1670): the lean choice, so a
node that needs the tier opts in (kBusLinks = true; ESP-IDF CONFIG_LIBTRACER_BUS_LINKS).
Closing it is a refusal, never a silent downgrade. A peer_named=true listener SPEC is
answered TYPE_MISMATCH (permanent — no retry grows a build a bus facet), a directly-constructed
peer-named listener reports ok() == false, and compiling LIBTRACER_TRANSPORT_CAN — a bus by
construction — is a static_assert. Serving such a configuration as flat would be worse than
either: the listener’s own per-frame tier select would keep delivering peer-named into a sink the
router never installed.
The module that is BOTH — kSelfHealLinks¶
The RFC-0014 §4 S5 link-liveness engine is the third shape, and the one this space did not have
before #1470: it does have a TU of its
own (core/src/self_heal_link.cpp), so it is dropped by a build switch like every other module —
and it also needs a configuration member, because the one place it is minted sits inside
transport_vertex.cpp, a TU every net-plane node compiles. Without the member, dropping the TU
is a link error rather than a trim.
So it carries both, and they are held in step from opposite ends. LIBTRACER_SELF_HEAL_LINKS
(CMake) / CONFIG_LIBTRACER_SELF_HEAL_LINKS (Kconfig) decides whether the TU compiles;
kSelfHealLinks decides whether anything reaches it, folding the mint site and the four calls
into the engine away at compile time so the linker never pulls the archive member back in. The
ESP-IDF component drives both from the one Kconfig symbol, so they cannot disagree there; a core
build pairs them by hand, and self_heal_link.cpp opens with a static_assert(kSelfHealLinks, …)
that catches the mismatch in the direction a link error would not explain. The saving is the
reporter’s C6 measurement: 4,336 B of reachable self_heal_link_t symbols, 0 B of
.dram0.bss. Both halves default OFF since v0.17.0
(#1670, re-ruling #1548’s default): an
opt-in turns the switch on AND binds kSelfHealLinks = true, which the core test build and
bench/ do and the Kconfig symbol does in one step.
Closing it is a refusal, never a silent downgrade, on the same principle as the bus tier:
register_transport_type does not catalogue a self_heal_dial kind on such a build (so a SPEC
naming it answers SCHEMA_NOT_FOUND, and a debug build asserts at the registration), rather than
registering it with the trait cleared — which would bring the connection up eagerly, with no
redial and no liveness publishing, and say nothing. Today the whole question is latent: no
in-tree kind sets self_heal_dial, so the engine is unreachable on every stock target and the
knob is a pure saving until #1548 flips
the built-in point-to-point kinds onto it.
The required-modules footprint ceiling¶
The minimum-feature module set — frame, tlv_arena, backend_set, mem_pool, mem_source,
rope, path — targets ≤ 16 KiB of stripped flash on
arm-none-eabi-g++ -std=c++23 -Os -fno-exceptions -fno-rtti -mcpu=cortex-m0 with
--specs=nano.specs (tools/cortexm0_footprint.py:REQUIRED_MODULES = ( for the module list, tools/cortexm0_footprint.py:cxx_flags = [ for the
compile flags, tools/cortexm0_footprint.py:"--specs=nano.specs" for the link spec). One committed sentinel
measures against it: tools/cortexm0_footprint.py, driven by .github/workflows/footprint-cortexm0.yml
over the core/tests/footprint/sentinel_node.cpp fixture. A second tool, tools/esp_size_gate.py,
reports the component’s flash and static-RAM contribution to the esp32c3/c6 full-node image
(all socket transports plus CAN — far more than the required modules) without setting a ceiling on
either: that image is one deployment’s composition, not a budget the library owes, and a ceiling
there would constrain how thin a client this library can serve. Its numbers are published to the
job step summary and a footprint-<target> artifact, and reviewed run to run.
The Cortex-M0 sentinel runs in warn mode because the measured node is 20,937 B, about 4.5 KiB
over the 16 KiB budget (re-measured 2026-07-27; .github/workflows/footprint-cortexm0.yml:governs the BUDGET VERDICT only).
The overage is not attributed to a module. An attribution of roughly 2.7 KiB to std::pmr
soft-float reaching the image through the arena decoder’s std::pmr::memory_resource& seam does
not survive the removal of that seam — the decoder takes a tr::mem::block_source_t&
(ADR-0065 — failable allocation gets its own seam)
— and the residue has not been re-measured against that seam. The gate is a referee for
the compile-time doctrine
(ADR-0047 — build-time closed module sets §5):
templating techniques are admissible exactly as far as it stays green, because it catches
type-erasure bloat and template-instantiation bloat alike.
The sized and bound axes¶
Every knob is overridden the same way — as a member of the fragment’s traits type. The column below names the member; a knob the fragment does not state keeps the default beside it.
The ESP-IDF column is that component’s fragment: it derives six knobs from its Kconfig
values and chip facts and writes them into a generated libtracer/config_override.hpp, listed
ahead of core/include on the component’s include path. Everything the column calls inherited
is a knob the fragment does not state at all (#1244).
knob |
kind |
default |
ESP-IDF |
|---|---|---|---|
|
count |
16 |
menuconfig |
|
padding width |
64 |
derived from |
|
count |
64 |
inherited — unused on a chip, which binds the single-writer slot and never builds the domain |
|
count |
32 |
set to 8 ( |
|
RAM ratchet |
88 / 64 |
the preset — deliberately not overridable |
|
size |
4,096 — RFC-0028 §11 ruling 2 |
|
|
size-class table |
80 rows: 16–128 B by 16, then eight per doubling to 64 KiB — the host slab pool’s classes (#1777) |
inherited — unused on a chip, which has no slab pool |
|
allocator choice |
|
|
|
size |
65,536 — the base slab |
inherited — unused on a chip |
|
count |
2 — fully free slabs a class keeps |
inherited — unused on a chip |
|
policy type |
|
inherited — the full policy is not selectable |
|
policy type |
|
|
|
target fact |
|
derived from |
|
module presence |
|
menuconfig |
|
module presence |
|
menuconfig |
|
size |
|
menuconfig |
|
ingress budget |
32 frames / 32,768 B per drain; |
menuconfig |
|
instrumentation |
|
inherited — only the core test build and |
|
instrumentation |
|
inherited — only the core test build binds it, through |
|
capability |
|
inherited — only the |
No knob has a CMake variable. The last two, -DLIBTRACER_ACL_FULL and -DLIBTRACER_LKV_SLOT,
were removed after their one transition release (#1722): configuring with a value that would
change the build is an error naming the trait to bind instead. The two compile definitions that
duplicated a trait went with them — LIBTRACER_NO_ATOMIC (the segment refcount now follows the
target and guard_t) and LIBTRACER_PIN_INSTRUMENT (folded into kInstrumentCounters) — and
config.hpp refuses either with an #error. The five other cache variables this table used to
list were deleted with the template (#1142).
Each is documented at its declaration with what it costs and when to move it; that header is
the reference, not this table. What matters here is the shape: twenty knobs, all named, all
finite. Four are counts (kVertexLockStripes, kHazardReaderSlots, kEdgePinSlots,
kSlabClassCap), one is a padding width, one is a per-target RAM ceiling, four are sizes
(kShareThresholdBytes, the slab pool’s size-class table kSizeClasses, its base slab
kSlabBytes, and the thread stack size kSelfHealWorkerStackBytes), two are an ingress budget
(kRxDrainFrames and kRxDrainBytes, the frames and bytes a receive context reads before it
waits for its core to idle), one chooses the host allocator (kSlabPool), two are type
bindings, one is a
target fact rather than a preference, and two — kBusLinks, below, and kSelfHealLinks (the
RFC-0014 S5 link-liveness engine, #1470) — state whether a module is present at all. kSpinWaitSafe says whether a task on this target may spin
for a lock another task holds, and the guard in mem_pool.hpp reads it to refuse
synchronized_pool_t<spin_sync_t> where the answer is no (#1158). The memory-ordering fact
that used to sit beside it, kWeaklyOrdered, was removed (#1717): the guard in vertex.hpp
now refuses a kDeliverySkipOrder weaker than seq_cst on every target (#1143), with the
ubuntu-24.04-arm CI leg (#1140) as the evidence half, so no build can waive it.
kInstrumentCounters is the one knob that selects no behaviour at all: it decides whether
graph_t carries the two counters that only tests and benches read (ancestor_walks(),
target_canonical_resolves()), each a relaxed 64-bit fetch_add — a libatomic call on rv32 —
on a node-wide line. The shipped default compiles them out; the core test build opts in through
the checked-in preset fragment core/tests/instrumented/, which yields to any fragment a CI leg
supplies (#1664). kFaultInjection is its twin for the test-only fault-injection hooks
(probe_fail_hook and three transport seams): closed out, the default allocation path carries
no hook branch and no hook variable reaches the archive (#1719).
Several of the twenty carry no build-system variable at all. kMaxVertexBytes64 / kMaxVertexBytes32,
kShareThresholdBytes, kSizeClasses, kSlabPool, kSlabBytes and kSlabClassCap are preset members: an application moves them by declaring its own
traits type, not by passing -D. kShareThresholdBytes is the copy-or-share threshold of
RFC-0028
§5.3 — a trailer-less written value of at least that many bytes is stored as a subview of the
inbound frame (shared, zero copy); a smaller one is copied into the value’s own block. Both
branches are correct, so the threshold selects which correct branch is cheaper rather than
imposing a limit: sharing holds the whole owning RX segment for the value’s lifetime, which
is the consumer’s retention budget to price. It replaced the RFC-0022 §3.D amplification ratio
K (D3: the variable measured was the absolute size). The host default is 4,096 B; the ESP-IDF
component binds SIZE_MAX (copy always), the posture it shipped before the knob. It is also the
one knob with a per-vertex value (vertex_policy_t::share_threshold_bytes), for an owner that knows
one vertex’s traffic differs from the target’s default.
The ESP-IDF component exposes exactly five options — LIBTRACER_TRANSPORT_{UDP,TCP,WS,CAN} and
LIBTRACER_VERTEX_LOCK_STRIPES — so an integrator reaching for the full ACL policy, the hazard
slot, or the net-plane switch through menuconfig will not find them. Those choices are made by
editing the component’s render site or by building libtracer’s sources from a CMake consumer.
Where the bytes are¶
The lock-stripe table is the only global mutable buffer libtracer links into a node. Its cost divides into a part reserved at link time and a part that really is lazy. The table’s cost is commonly described as lazy in full; only the platform primitive behind each handle is:
Static, not lazy.
kVertexLockStripesstripe structs plus the same number of condvar handles, in.bss, present whether or not a graph is ever constructed.Lazy. The platform primitive behind each handle. On FreeRTOS a stripe’s mutex costs roughly 90 B of heap on that stripe’s first lock, so a stripe no vertex has hashed onto costs its struct and no heap.
Most of the struct is false-sharing padding, which is what kCacheLineBytes governs.
Measured on rv32 (-Os -fno-exceptions -fno-rtti, GCC 15.2, compiling the real
core/src/graph.cpp), 16 stripes:
|
stripe table |
TU |
|---|---|---|
64 |
1,024 B |
1,200 B |
0 |
128 B |
304 B |
896 B of a single-core node’s static RAM, spent against a hazard it does not have — there is no second core for two stripes on one line to contend over.
The same knob governs the hazard domain’s cells and retire lists, where at 64 slots it is worth
a further 6.5 KB. Measured on rv32 at N = 64, same instrument (-Os, real core/src/graph.cpp,
GCC 15.2):
|
hazard registry |
TU |
|---|---|---|
64 |
8,384 B |
11,649 B |
0 |
1,828 B |
4,197 B |
The TU column is not the registry plus the stripes: binding hazard_slot_t also pulls in
roughly 2 KB of libstdc++ atomic::wait back-end (__waiter_pool_base) .bss beyond the
registry itself.
kCacheLineBytes is an optimization axis and never a correctness one: 0 on a multi-core target
costs control-plane throughput under concurrent verb traffic and changes nothing observable.
The ESP-IDF component therefore derives it from CONFIG_FREERTOS_UNICORE rather than exposing
it — a unicore build has no second core by construction, so the right value is not a question
an integrator should be asked.
The LKV slot contract¶
lkv_slot_t is the one knob whose value is a name the integrator supplies, so it is the one
knob with a contract attached. The declaration instructs that the named type must satisfy the
policy contract in lkv_slot.hpp (config.hpp:default_config_t::lkv_slot_t, and the instruction itself at config.hpp:contend across many cores. Override fragment:) —
a header that is absent from core/Doxyfile’s INPUT list, so the generated API site does not
serve the page that instruction points at. The contract, stated here, is three operations over
value_ptr_t = std::shared_ptr<const view::rope_t> and one declaration:
operation |
signature |
rule |
|---|---|---|
publish |
|
Sequentially consistent by default: |
drop |
|
Cannot fail, and says so in the return type: a clear releases resources rather than acquiring any. Only |
read |
|
Returns an owning handle. |
spin |
|
Mandatory. |
Owning is not negotiable. The composed branch read graph_t::read_subtree_folded
(core/include/libtracer/graph.hpp:graph_t::read_subtree_folded) stashes one LKV per node into a vector that outlives
the map lock and spans three passes, so N values are held simultaneously. A reclamation
scheme that can protect only one value per reader at a time — hazard pointers, as classically
stated — therefore cannot hand back a pinned pointer; it must promote the pin to a counted
reference before releasing it, and that promotion is a read-modify-write on the one cache line
every reader shares. That promotion, not the scheme, is what the measured win is net of; the
numbers and their conditions are in
../concurrency/00-scaling-and-serialization.md,
which is where they belong rather than repeated here.
The default is single_writer_slot_t, whose reclamation is the refcount, so there is no scheme
to implement and no registry to size. Its one wait is guard_t, which on a chip is an
interrupt-masked critical section and on a host one of 64 address-striped mutexes. A host can
bind hazard_slot_t instead, the lock-free slot, at a heap node per published value. The refcount
slot that used to be the default, std::atomic<std::shared_ptr<T>>, was removed because
libstdc++ spin-locks both its load and its store, and on a priority-preemptive core that spin
can hang the target (#1618;
ADR-0069 — the LKV slot is a compile-time policy).
Per-target nuances¶
Four differences that surprise people, each a property of the target rather than a choice:
The stripe table is a different object on a microcontroller. libstdc++ makes
std::mutex’s constructorconstexpronly where its gthreads port supports static initialization. ESP-IDF’s does not, so the host gets aninline constinittable indexed with no init-guard check, while the MCU gets a guarded function-local static — one predicted branch per control-plane verb. Prose describing “theconstinitstripe table” is describing the host only.The hazard domain costs nothing until it is bound. Its storage lives inside an
inlinefunction whose static is never named under the single-writer slot, so a chip build emits no registry at all. Bindinghazard_slot_talso pulls in roughly 2 KB of libstdc++atomic::waitback-end.bssbeyond the registry itself.sizeof(vertex_t)is gated in the header, not in a test. The ceilings areconfig_tmembers and the assertions sit invertex.hppbeside the type they constrain (core/include/libtracer/vertex.hpp:static_assert(sizeof(void*) != 8,core/include/libtracer/vertex.hpp:static_assert(sizeof(void*) != 4), so every build on every target checks its own binding, for free. A test-resident gate covers only the configurations CI actually builds: one, in practice, and never the 32-bit arm, because no CI leg cross-compiles that test while the ESP-IDF legs compilevertex_titself on every change. That distinction has teeth here — both arms are ratchets pinned to the measured size, so neither has headroom by construction (config.hpp:default_config_t::kMaxVertexBytes32): 88 B on 64-bit, 64 B on rv32, and the next added member is a build failure on both. They were ceilings held above the measurement until 2026-08-10, which is why 16 B reclaimed on the 64-bit arm and 8 B on the 32-bit one went unnoticed — a ceiling answers “did you regress past a fixed point”, never “did this get leaner”. The stripe carries a companion assertion of a different kind:alignof(vertex_stripe_t) == kStripeAlign(vertex_stripe.hpp:static_assert(alignof(vertex_stripe_t) == kStripeAlign), which catches analignasthat asked for less than the payload’s natural alignment and was therefore ignored — silently, by GCC, per[dcl.align]/5.A single-core target’s constraint is RAM; a many-core host’s is the read path. The two policy-type knobs exist because of that split: see 15-concurrency-and-scaling.md for which shapes scale and
../concurrency/for this implementation’s measured costs.
What is deliberately not configurable¶
Some constants look exactly like sizing knobs and are not:
The address bounds — maximum segment length, maximum path length, maximum segment count, maximum field depth (03-addressing.md). These are normative and incorporated by the spec: making them per-target would let one node accept a path another must reject, which is an interoperability failure dressed as a RAM saving. A node that wants a smaller bound is asking for a profile, and that is a spec question.
Anything derived from an injected resource. Capacities that depend on how much memory the application handed the node come from that resource, not from a constant — bounds are injected or per-target configuration, never a magic number (RFC-0006 — resource-bounded nesting depth). Field depth is the worked example: resource-keyed rather than configurable, precisely so it cannot diverge between peers.
Adding an axis¶
A knob is added as a default_config_t member in config.hpp, with its default as an ordinary
C++ literal. That is the whole procedure: there is one copy, so nothing has to be regenerated
and nothing can drift.
Nothing else has to be touched. Every override fragment inherits default_config_t, so an
existing override — the ESP-IDF component’s, an application’s — picks up the new knob at its
new default without an edit. Deliberately, a knob does not get a CMake cache variable: it is
set in the fragment, as C++. (-DLIBTRACER_ACL_FULL and -DLIBTRACER_LKV_SLOT, the last two
that had one, were removed in #1722.)
Put the knob in default_config_t even when it states an L0 fact — kSpinWaitSafe is a
tr::mem concept whose spelling is derived from config_t. A knob outside the one named type
cannot be reached by a fragment at all, which is exactly how that one ended up stranded in the
build system.