A stale label is dropped, NACK’d, and re-advertised (L4 routing)

Compaction makes a delivery depend on state the two ends must agree about. Anything that can desynchronise it — a reconnect, a restart, an eviction to stay inside a bounded table — leaves an upstream happily streaming onto a label the downstream has forgotten.

The design answer is that a stale label is never dereferenced and never guessed at: the frame is dropped, a HANDLE_NACK goes back, and receiving that NACK makes the producer re-advertise (RFC-0004 §E.1, ADR-0035, ADR-0030).

Re-advertising IS the self-heal. There is no separate repair protocol, no sequence numbers to reconcile and no teardown handshake — which is why clear_link is a safe thing for a transport to call on every (re)connect: the worst case is one dropped frame and one round trip.

What to notice

  • The whole repair is counted, not narrated. One frame lost, one NACK, one re-advertise, and the delivery count resumes climbing. Those are the four assertions.

  • The refusal is observable. on_stale_label carries the inbound link name and the refused label, so an operator can see a desynchronised flow instead of inferring it from a gap in the data. Dropping silently would be the same behaviour with none of the evidence.

  • The refused frame changed nothing. The delivery count does not move — a dropped frame, not a misrouted one. That distinction is the entire reason the label is not looked up best-effort.

  • clear_link on an unknown link is a no-op, deliberately, so a transport can call it unconditionally from its connect path. The example checks that too.

  • On a mid-chain node clear_link reaches further than one link. It also drops every ingress binding whose downstream half crossed the cleared link (#716) — without that, an upstream that never saw the reconnect keeps streaming onto a dead out-label and the flow drops silently forever. The two-node example cannot show that; it is the reason the hook is not simply “forget my own table”.

  • link_down is the bigger hammer. It runs the subscriber-edge eviction as well as this label clear, and it is what add_child installs behind every child’s departure notifier.

  • This target needs the FWD net plane. It is built only when LIBTRACER_NET_PLANE is on (the default). Nothing in it is conditional at run time.

Source

  1/*
  2 * SPDX-License-Identifier: Apache-2.0
  3 * SPDX-FileCopyrightText: Copyright 2026 avatarsd LLC
  4 */
  5
  6/**
  7 * @file
  8 * @brief ONE CONCEPT — a label the receiver no longer knows is DROPPED and NACK'd, and the NACK
  9 *        is what puts the flow back together.
 10 *
 11 * Compaction (route_label_compact) makes a delivery depend on state the two ends must agree
 12 * about. Anything that can desynchronise it — a reconnect, a restart, an eviction to stay
 13 * inside a bounded table — leaves an upstream happily streaming onto a label the downstream has
 14 * forgotten. The design answer is that a stale label is never dereferenced and never guessed
 15 * at: the frame is dropped, a `HANDLE_NACK` goes back, and receiving that NACK makes the
 16 * producer re-advertise (ADR-0035, ADR-0030).
 17 *
 18 * Re-advertising IS the self-heal. There is no separate repair protocol, no sequence numbers to
 19 * reconcile and no teardown handshake — which is why `clear_link` is a safe thing for a
 20 * transport to call on every (re)connect: the worst case is one dropped frame and one round
 21 * trip.
 22 *
 23 * `clear_link` below stands in for the reconnect. It is the same call a transport makes from
 24 * its connect/disconnect hook, and calling it for a live or unknown link is deliberately safe.
 25 *
 26 * Runs under ctest as `example_route_label_stale`; returns non-zero on any failed check.
 27 */
 28
 29#include <cstddef>
 30#include <cstdint>
 31#include <cstdio>
 32#include <initializer_list>
 33#include <span>
 34#include <string>
 35#include <string_view>
 36#include <vector>
 37
 38#include "libtracer/fwd_router.hpp"
 39#include "libtracer/route_handle.hpp"
 40#include "libtracer/tlv_emit.hpp"
 41#include "libtracer/tracer.hpp"
 42
 43namespace {
 44
 45using tr::graph::graph_t;
 46using tr::graph::path_t;
 47using tr::graph::role_t;
 48using tr::wire::opt_t;
 49using tr::wire::type_t;
 50
 51/** @brief Report expectation @p what and record a failure on @p ok. */
 52void check(bool& ok, bool cond, const char* what) {
 53    std::printf("  [%s] %s\n", cond ? "ok" : "FAIL", what);
 54    ok = ok && cond;
 55}
 56
 57/** @brief A `transport_t` that keeps every frame handed to it — the "wire", made inspectable. */
 58struct recording_link_t : tr::net::transport_t {
 59    std::vector<std::vector<std::byte>> sent; /**< @brief Frames emitted on this link, in order. */
 60    void send(std::span<const std::byte> frame) override {
 61        sent.emplace_back(frame.begin(), frame.end());
 62    }
 63    void send(std::span<const std::span<const std::byte>> iov) override {
 64        std::vector<std::byte> flat;
 65        for (const auto part : iov) flat.insert(flat.end(), part.begin(), part.end());
 66        sent.push_back(std::move(flat));
 67    }
 68};
 69
 70/** @brief What the consumer observed — deliveries taken, and labels refused. */
 71struct observer_t {
 72    std::size_t deliveries = 0;   /**< @brief COMPACTs that resolved to a local terminus. */
 73    std::size_t stale = 0;        /**< @brief COMPACTs dropped for an unknown label. */
 74    std::uint16_t last_stale = 0; /**< @brief The last refused label. */
 75    std::string last_stale_link;  /**< @brief The link it arrived on. */
 76};
 77
 78/** @brief A `PATH` TLV over @p segs — RFC-0018 packed segment records, `opt.PL` clear. */
 79std::vector<std::byte> path_tlv(std::initializer_list<std::string_view> segs) {
 80    std::vector<std::byte> body;
 81    for (std::string_view s : segs) (void)tr::wire::emit_path_segment(body, s);
 82    std::vector<std::byte> out;
 83    tr::wire::emit_tlv(out, type_t::PATH, opt_t{}, body);
 84    return out;
 85}
 86
 87/** @brief A one-byte `VALUE` TLV carrying @p v. */
 88std::vector<std::byte> value_tlv(std::uint8_t v) {
 89    const std::byte b{v};
 90    std::vector<std::byte> out;
 91    tr::wire::emit_tlv(out, type_t::VALUE, opt_t{}, std::span<const std::byte>(&b, 1));
 92    return out;
 93}
 94
 95}  // namespace
 96
 97int main() {
 98    bool ok = true;
 99
100    graph_t graph_p, graph_c;
101    tr::net::fwd_router_t producer(graph_p), consumer(graph_c);
102    recording_link_t p_to_c, c_to_p;
103    if (!producer.add_child("c", p_to_c) || !consumer.add_child("p", c_to_p)) {
104        std::fprintf(stderr, "route_label_stale: add_child failed — nothing registered\n");
105        return 1;
106    }
107    (void)graph_c.register_vertex(path_t("/mirror"), role_t::STORED_VALUE);
108
109    observer_t obs;
110    consumer.on_compact_delivery(
111        [](void* ctx, std::span<const std::byte>, std::span<const std::byte>) {
112            ++static_cast<observer_t*>(ctx)->deliveries;
113        },
114        &obs);
115    consumer.on_stale_label(
116        [](void* ctx, std::string_view inbound, std::uint16_t label) {
117            auto* o = static_cast<observer_t*>(ctx);
118            ++o->stale;
119            o->last_stale = label;
120            o->last_stale_link.assign(inbound);
121        },
122        &obs);
123
124    // An established flow: advertise, then stream.
125    const std::vector<std::byte> route = path_tlv({"mirror"});
126    const std::vector<std::byte> payload = value_tlv(0x2A);
127    const std::uint16_t label = producer.advertise("c", route);
128    if (label == 0 || p_to_c.sent.size() != 1) {
129        std::fprintf(stderr, "route_label_stale: the flow would not establish\n");
130        return 1;
131    }
132    consumer.on_frame("p", p_to_c.sent[0]);
133    p_to_c.sent.clear();
134    producer.send_compact("c", label, payload);
135    consumer.on_frame("p", p_to_c.sent.front());
136    check(ok, obs.deliveries == 1 && obs.stale == 0, "the flow is established and delivering");
137
138    // The reconnect. The consumer forgets every label on this link; the producer, which never
139    // saw it, keeps streaming onto the label it still holds.
140    consumer.clear_link("p");
141    check(ok, consumer.handles().ingress_count() == 0, "clear_link dropped the link's bindings");
142
143    p_to_c.sent.clear();
144    c_to_p.sent.clear();
145    producer.send_compact("c", label, payload);
146    consumer.on_frame("p", p_to_c.sent.front());
147    check(ok, obs.stale == 1 && obs.last_stale == label,
148          "the unknown label was REFUSED — not dereferenced, not guessed at");
149    check(ok, obs.last_stale_link == "p",
150          "and the refusal is reported against the link it came on");
151    check(ok, obs.deliveries == 1, "nothing was delivered: a dropped frame, not a misrouted one");
152
153    // The refusal is not silent. A HANDLE_NACK goes back on the same link.
154    check(ok, c_to_p.sent.size() == 1, "a HANDLE_NACK went back toward the producer");
155    if (c_to_p.sent.size() != 1) return 1;
156
157    // Handing the NACK to the producer is the whole repair: it re-advertises by itself.
158    p_to_c.sent.clear();
159    producer.on_frame("c", c_to_p.sent[0]);
160    check(ok, p_to_c.sent.size() == 1,
161          "receiving the NACK made the producer re-advertise — no repair protocol");
162
163    // The re-advertise rebinds the consumer, and the next COMPACT lands again.
164    consumer.on_frame("p", p_to_c.sent[0]);
165    check(ok, consumer.handles().ingress_count() == 1, "the consumer is bound again");
166    p_to_c.sent.clear();
167    producer.send_compact("c", label, payload);
168    consumer.on_frame("p", p_to_c.sent.front());
169    check(ok, obs.deliveries == 2, "and the flow resumed");
170    check(ok, obs.stale == 1, "with exactly ONE frame lost to the desynchronisation");
171
172    // Calling the hook for a link that has no label state at all is a no-op, which is what
173    // lets a transport call it unconditionally from its connect path.
174    consumer.clear_link("no-such-link");
175    check(ok, consumer.handles().ingress_count() == 1, "clearing an unknown link changed nothing");
176
177    std::printf("reconnect cost: %zu dropped frame, 1 NACK, 1 re-advertise, %zu deliveries kept\n",
178                obs.stale, obs.deliveries);
179    return ok ? 0 : 1;
180}

See also: fwd-router module · network formation · CAN transport · the label plane · the child table.