One listener, many slots — and why p<slot> is not an identity (transport plane)

tcp_transport_t’s LISTEN constructor accepts one peer at a time — the board↔board shape. A node that fans out to browser tabs or to a fleet needs the other one: transport_tcp_server (and its RFC 6455 sibling transport_ws_server, sharing the same slot/poll machinery since #871) runs one poll thread over a slot table, so steady-state memory is bounded by the concurrent-peer high-water mark or by max_peers, whichever is smaller.

With peer_named, that slot table is exposed through the same bus_link_t facet CAN uses — and that is where the two naming regimes part company.

What to notice

  • CAN names by IDENTITY, a slot server names by POSITION. n<node-id> is derived from the peer’s own bus id, so the name, the table key and the endpoint are one thing no other peer can inherit. p<slot> is a position: after that peer departs, a pointer resolved for it addresses whatever session inherits the slot, and the endpoint’s own liveness check is satisfied by that stranger. The pointer never dangles; it silently changes who it means (#1153).

  • So: resolve per use. child_registry_t resolves and sends in one expression, which is why no shipping caller is exposed, and a remote subscriber edge stores the peer NAME rather than the pointer.

  • max_peers is an injected bound, not a backlog (RFC-0006). A connection past the cap is accepted and immediately closed — a clean refusal rather than a hung SYN. It is also resolved once at construction (a request of 0 takes the liveness window’s own ceiling), so the example reads back what the server enforces instead of assuming it got what it asked for.

  • One object, two addressing surfaces. server.send(frame) fans out to every open peer; facet.peer_link("p1")->send(frame) reaches exactly one. The example checks the directed send arrived at p1 and that p0 and p2 got nothing.

  • A name outside the live slot set resolves to nullptr. Endpoints are not synthesized on demand, so a stale name cannot be sent to.

  • This target needs the TCP transport (LIBTRACER_TRANSPORT_TCP, the default).

The one run-time skip in this group, and why it is not silent

The subject here is the ADR-0044 peer-named tier, and a target can compile that out with kBusLinks = false — a C++ binding in config_override.hpp (ADR-0068), invisible to CMake, so no build guard can express it. Neither move the earlier domains used is available: there is no second arm to name (with the module closed out there is no peer-named tier at all), and a return 0 would make ctest record a pass for an example that ran nothing — the exact defect the index warns about.

So the example states the skip and exits 77, and its add_test carries SKIP_RETURN_CODE 77, which makes ctest report it as Skipped. That is strictly better than the two older run-time skips in this tree, which exit 0 and are therefore indistinguishable from a real pass. Verified: in a kBusLinks = false build ctest -R example_ reports example_net_multi_peer_listener (Skipped) and passes the rest.

Source

  1/*
  2 * SPDX-License-Identifier: Apache-2.0
  3 * SPDX-FileCopyrightText: Copyright 2026 avatarsd LLC
  4 */
  5
  6/**
  7 * @file
  8 * @brief ONE CONCEPT — a LISTEN link is not one link per peer: `transport_tcp_server` holds
  9 *        many peers in recycled SLOTS behind one `transport_t`, names them POSITIONALLY as
 10 *        `p<slot>`, and that positional naming is precisely why a resolved peer endpoint
 11 *        must be re-resolved per use instead of cached.
 12 *
 13 * `tcp_transport_t`'s LISTEN constructor accepts one peer at a time — the board↔board
 14 * shape. A node that fans out to browser tabs or to a fleet needs the other one:
 15 * `transport_tcp_server` (and its RFC 6455 sibling `transport_ws_server`) share one
 16 * poll thread over a slot table, so steady-state memory is bounded by the concurrent-peer
 17 * high-water mark or by `max_peers`, whichever is smaller — an injected bound (RFC-0006),
 18 * never a synthetic backlog. A connection past the cap is accepted and immediately closed,
 19 * which is a clean refusal rather than a hung SYN.
 20 *
 21 * With `peer_named`, that slot table is exposed through the same @ref tr::net::bus_link_t
 22 * facet CAN uses, and this is where the two naming regimes part company:
 23 *
 24 *  - CAN names a peer `n<node-id>` — an IDENTITY. The name, the table key and the endpoint
 25 *    are one thing that no other peer can inherit.
 26 *  - A slot server names a peer `p<slot>` — a POSITION. The endpoint is scoped to the SLOT,
 27 *    so after that peer departs a pointer resolved for it addresses whatever session
 28 *    inherits the slot, and the endpoint's own liveness check is satisfied by that stranger.
 29 *    The pointer never dangles; it silently changes who it means (#1153). Resolve, send,
 30 *    discard — `child_registry_t` does exactly that in one expression, which is why no
 31 *    shipping caller is exposed.
 32 *
 33 * The subject of this example is the ADR-0044 peer-named tier, so a target that closed the
 34 * bus module out (`kBusLinks = false`) does not contain it. That is stated and SKIPPED with
 35 * exit code 77 — which ctest reads as `Skipped`, not as a pass — rather than returning 0
 36 * from a `main` that demonstrated nothing.
 37 *
 38 * Needs the TCP transport (`LIBTRACER_TRANSPORT_TCP`, on by default). Runs under ctest as
 39 * `example_net_multi_peer_listener`; returns non-zero on any failed check.
 40 */
 41
 42#include <arpa/inet.h>
 43#include <netinet/in.h>
 44#include <poll.h>
 45#include <sys/socket.h>
 46#include <unistd.h>
 47
 48#include <algorithm>
 49#include <chrono>
 50#include <cstddef>
 51#include <cstdint>
 52#include <cstdio>
 53#include <memory>
 54#include <span>
 55#include <string>
 56#include <string_view>
 57#include <thread>
 58#include <vector>
 59
 60#include "libtracer/config.hpp"
 61#include "libtracer/transport_tcp.hpp"
 62
 63namespace {
 64
 65using namespace std::chrono_literals;
 66
 67/**
 68 * @brief The exit code ctest reads as SKIPPED, via the target's `SKIP_RETURN_CODE` property.
 69 *
 70 * 77 is the autotools convention ctest documents. The same value `tr::testing::kSkipExitCode`
 71 * spells on the test side; the `add_test` site in `core/examples/CMakeLists.txt` and this
 72 * constant have to agree, so changing one means changing both.
 73 */
 74constexpr int kSkipExitCode = 77;
 75
 76/** @brief Report expectation @p what and record a failure on @p ok. */
 77void check(bool& ok, bool cond, const char* what) {
 78    std::printf("  [%s] %s\n", cond ? "ok" : "FAIL", what);
 79    ok = ok && cond;
 80}
 81
 82/** @brief A raw POSIX TCP client — one peer of the listener, with its own eye on the wire. */
 83class raw_peer_t {
 84   public:
 85    /** @brief Connect to `127.0.0.1:@p port`; @ref ok reports whether it succeeded. */
 86    explicit raw_peer_t(std::uint16_t port) {
 87        fd_ = ::socket(AF_INET, SOCK_STREAM, 0);
 88        sockaddr_in peer{};
 89        peer.sin_family = AF_INET;
 90        peer.sin_port = htons(port);
 91        ::inet_pton(AF_INET, "127.0.0.1", &peer.sin_addr);
 92        if (::connect(fd_, reinterpret_cast<sockaddr*>(&peer), sizeof(peer)) < 0) {
 93            ::close(fd_);
 94            fd_ = -1;
 95        }
 96    }
 97    ~raw_peer_t() {
 98        if (fd_ >= 0) ::close(fd_);
 99    }
100
101    raw_peer_t(const raw_peer_t&) = delete;
102    raw_peer_t& operator=(const raw_peer_t&) = delete;
103
104    /** @brief True iff the connect succeeded. */
105    [[nodiscard]] bool ok() const noexcept { return fd_ >= 0; }
106
107    /** @brief Read up to @p want bytes within @p budget, answering whatever arrived. */
108    [[nodiscard]] std::vector<std::byte> read_within(std::size_t want,
109                                                     std::chrono::milliseconds budget) {
110        std::vector<std::byte> got;
111        const auto deadline = std::chrono::steady_clock::now() + budget;
112        while (got.size() < want) {
113            const auto left = std::chrono::duration_cast<std::chrono::milliseconds>(
114                deadline - std::chrono::steady_clock::now());
115            if (left.count() <= 0) break;
116            pollfd p{fd_, POLLIN, 0};
117            if (::poll(&p, 1, static_cast<int>(left.count())) <= 0) break;
118            std::byte buf[256];
119            const ssize_t n = ::recv(fd_, buf, sizeof(buf), 0);
120            if (n <= 0) break;
121            got.insert(got.end(), buf, buf + n);
122        }
123        return got;
124    }
125
126   private:
127    int fd_ = -1;
128};
129
130/** @brief @p n bytes counting up from @p seed — a stand-in for an encoded frame. */
131std::vector<std::byte> frame_of(std::size_t n, unsigned seed) {
132    std::vector<std::byte> f(n);
133    for (std::size_t i = 0; i < n; ++i) f[i] = static_cast<std::byte>(seed + i);
134    return f;
135}
136
137/** @brief `u32-LE length ++ @p payload` — what one record looks like on this kind's wire. */
138std::vector<std::byte> record(std::span<const std::byte> payload) {
139    const auto len = static_cast<std::uint32_t>(payload.size());
140    std::vector<std::byte> out;
141    for (unsigned shift = 0; shift < 32; shift += 8)
142        out.push_back(static_cast<std::byte>((len >> shift) & 0xFFu));
143    out.insert(out.end(), payload.begin(), payload.end());
144    return out;
145}
146
147/** @brief The peer names @p facet currently lists, sorted so the order is this example's. */
148std::vector<std::string> peers_of(tr::net::bus_link_t& facet) {
149    std::vector<std::string> names;
150    facet.enumerate_peers([&](std::string_view n) { names.emplace_back(n); });
151    std::sort(names.begin(), names.end());
152    return names;
153}
154
155/**
156 * @brief Poll until @p facet lists @p n peers, or @p budget expires.
157 *
158 * A bounded poll, not a wait: acceptance happens on the server's own poll thread and the
159 * seam publishes no completion edge to wait on. The loop is the honest shape for observing
160 * a state that becomes true asynchronously with no notification.
161 */
162bool wait_for_peers(tr::net::bus_link_t& facet, std::size_t n, std::chrono::milliseconds budget) {
163    const auto deadline = std::chrono::steady_clock::now() + budget;
164    while (std::chrono::steady_clock::now() < deadline) {
165        if (peers_of(facet).size() >= n) return true;
166        std::this_thread::sleep_for(2ms);
167    }
168    return peers_of(facet).size() >= n;
169}
170
171}  // namespace
172
173int main() {
174    // The subject is the peer-named tier itself. On a target that compiled it out there is
175    // nothing here to demonstrate, and a `return 0` would make ctest record a pass for an
176    // example that ran nothing — the failure mode this whole batch exists to avoid.
177    if constexpr (!tr::net::kBusLinks) {
178        std::printf(
179            "net_multi_peer_listener: SKIPPED — this build closed the ADR-0044 bus module "
180            "out (kBusLinks = false), so a listener has no peer-named tier to show.\n");
181        return kSkipExitCode;
182    }
183
184    bool ok = true;
185    std::printf("kBusLinks = true — the peer-named tier is present in this build\n");
186
187    // max_peers is the injected admission bound. It is resolved once at construction (a
188    // request of 0 takes the liveness window's own ceiling), so the value the server ENFORCES
189    // is read back rather than assumed to be what was asked for.
190    constexpr std::size_t kRequestedPeers = 4;
191    tr::net::transport_tcp_server server(std::uint16_t{0}, &tr::mem::heap_backend(),
192                                         /*max_frame=*/0, kRequestedPeers, /*peer_named=*/true);
193    check(ok, server.ok(), "the multi-peer listener bound an ephemeral port");
194    check(ok, server.max_peers() == kRequestedPeers, "the admission cap is the one requested");
195    check(ok, server.bus() != nullptr, "peer_named=true exposes the bus facet");
196
197    tr::net::bus_link_t& facet = *server.bus();
198    check(ok, peers_of(facet).empty(), "no peers before anyone connects");
199
200    // Three peers on ONE listener, one poll thread, one transport_t.
201    std::printf("three peers on one listener:\n");
202    std::vector<std::unique_ptr<raw_peer_t>> clients;
203    for (int i = 0; i < 3; ++i)
204        clients.push_back(std::make_unique<raw_peer_t>(server.local_port()));
205    check(ok, clients[0]->ok() && clients[1]->ok() && clients[2]->ok(), "all three connected");
206    check(ok, wait_for_peers(facet, 3, 2s), "the listener accepted all three into slots");
207
208    const auto names = peers_of(facet);
209    check(ok, names.size() == 3, "…and lists exactly three peers");
210    check(ok, names == std::vector<std::string>{"p0", "p1", "p2"},
211          "…named p0/p1/p2 — by SLOT, which is a position and not an identity");
212
213    // A DIRECTED send: one named peer's socket, and nobody else's.
214    std::printf("a directed send:\n");
215    const auto only_for_p1 = frame_of(6, 0x10);
216    tr::net::transport_t* to_p1 = facet.peer_link("p1");
217    check(ok, to_p1 != nullptr, "peer_link('p1') resolved a directed endpoint");
218    if (to_p1 != nullptr) to_p1->send(only_for_p1);
219    const auto want = record(only_for_p1);
220    check(ok, clients[1]->read_within(want.size(), 2s) == want,
221          "the addressed peer got the record, prefix and all");
222    check(ok, clients[0]->read_within(1, 200ms).empty(), "…and peer p0 got nothing");
223    check(ok, clients[2]->read_within(1, 200ms).empty(), "…and peer p2 got nothing");
224
225    // The FLAT surface of the same object: `send` on the server itself fans out to every open
226    // peer. One link, two addressing surfaces — the flat one for "this link", the facet for
227    // "that peer on this link".
228    std::printf("the flat broadcast on the same object:\n");
229    const auto for_everyone = frame_of(4, 0x20);
230    server.send(for_everyone);
231    const auto expect = record(for_everyone);
232    check(ok, clients[0]->read_within(expect.size(), 2s) == expect, "p0 got the broadcast");
233    check(ok, clients[1]->read_within(expect.size(), 2s) == expect, "p1 got it too");
234    check(ok, clients[2]->read_within(expect.size(), 2s) == expect, "and so did p2");
235
236    // A name outside the live slot set resolves to nothing — the endpoint is not synthesized
237    // on demand, so a stale name cannot be sent to.
238    check(ok, facet.peer_link("p9") == nullptr, "a name no slot holds resolves to no endpoint");
239
240    check(ok, server.dropped_tx() == 0, "nothing was shed on the way out");
241
242    std::printf("one listener, %zu peers in slots, cap %zu, 1 directed + 1 broadcast\n",
243                names.size(), server.max_peers());
244    return ok ? 0 : 1;
245}

See also: transport module · concurrency and scaling · the identity-named bus · the single-peer pair.