Restart and Outdated States

The report asks how outdated messages become recognisable after a connection loss. It contrasts three restart rules using a synthetic trace and states the limits of the fault model. The full text is the readable report.

U01 · research report · not peer-reviewed

Summary

After a connection loss, buffered messages can arrive that look like current states. This report compares three recovery rules for a single state receiver: a rule based on the sequence number, one based on message age (timestamp), and a combination of both with an explicit restart rule. The comparison uses a small synthetic trace and a qualitative table. This is a report from the scenario; it is not scientific evidence and contains no measurements.

Research question

In the project Recovery of distributed state messages, the guiding question is: which combination of sequence number, timestamp and recovery rule makes stale states recognisable? Stale means here that a message is delivered after an interruption although the sender has since reported a newer state, or the message is too old to describe the present state. For a receiver that displays or processes states, two errors must be distinguished:

  • a stale message is wrongly accepted and overwrites the current state;
  • a fresh message is wrongly discarded, so that the receiver stays on an old state indefinitely.

Set-up

We consider a sender that reports the state of a single object (“OPEN” or “CLOSED”) and a receiver that holds exactly one stored state. Every message carries three fields: an epoch number e, which the sender increases by 1 at each restart and stores durably; a sequence number s, which starts at 1 within an epoch and rises by 1 per message; and the send time t in seconds. The age of a message is its arrival time minus t; sender and receiver clocks are taken to be identical in the example. The maximum age is set to 15 s. The three rules are:

  1. Rule A (sequence number only): a message is accepted if s is greater than the sequence number of the last accepted message.
  2. Rule B (age only): a message is accepted if its age is at most 15 s.
  3. Rule C (combination with restart rule): a message is accepted if its age is at most 15 s and the pair (e, s) is lexicographically greater than that of the last accepted message. A higher epoch therefore always counts as newer, even though s starts again at 1.

The approach follows the comparison logic planned in the project; a test set-up on the network test bed has not been evaluated for this report. The ordering of possible fault cases underlying the project is described in the method note Fault catalogue for state messages.

Assumptions

This report is a document; the numbers below are constructed and support no statement about real systems. The assumptions are: one sender and one receiver, with no further participants. The transmission path buffers messages during an interruption and delivers them once the connection is restored, occasionally in swapped order. Messages are neither lost nor duplicated. Clocks are identical and the epoch number survives a restart. The maximum age of 15 s is a freely chosen parameter without technical justification. A decision counts as correct if the receiver accepts exactly those messages that, at arrival time, are neither too old nor superseded by a newer message from the same sender.

Illustrative analysis

Constructed worked example, not a measurement. In epoch 1 the sender reports four states. Between second 12 and second 60 the connection is interrupted; messages 3 and 4 are buffered and delivered only at second 61. At second 55 the sender restarts (epoch 2, sequence starting at 1 again). Messages 5 to 7 arrive afterwards, with message 7 overtaking message 6. The table lists the messages in order of arrival.

Synthetic trace in order of arrival (constructed; times in seconds)
MessageEpoch eSequence sSend time tArrival timeAgeState
M111011OPEN
M21210111CLOSED
M313206141OPEN
M414406121OPEN
M52158624OPEN
M72367681CLOSED
M62265694OPEN

Applying the rules from the set-up section gives the decisions in the next table. Correct decisions are: accept M1, M2, M5 and M7; discard M3 and M4 (too old) and M6 (superseded by M7). The most recently sent state is CLOSED (M7).

Decisions of the three rules on the trace (A = accepted, D = discarded; constructed worked example, not a measurement)
MessageCorrectRule ARule BRule C
M1AAAA
M2AAAA
M3DADD
M4DADD
M5ADAA
M7ADAA
M6DDAD
Correct decisions7 of 73 of 76 of 77 of 7
Final displayed stateCLOSEDOPENOPENCLOSED

Rule A accepts the buffered stale messages M3 and M4 because their sequence numbers exceed that of M2. After the restart it discards every fresh message, since the sequence starts again at 1; the receiver stays at OPEN until the sequence exceeds 4. Rule B recognises the stale messages by their age but has no notion of order, so it accepts M6 after M7. Rule C avoids both errors because the epoch expresses the restart and the sequence number expresses the order within an epoch.

Qualitative comparison of the rules (assessment on the constructed trace, not a measurement)
RuleDetects stale messages after interruptionDetects swapped orderSurvives sender restartPrerequisite
A: sequenceNoYesNoDurable counter
B: ageYesNoYesSufficiently aligned clocks
C: combinationYesYesYesCounter, epoch and clocks

For rule C the communication overhead grows only by the additional fields per message; no figure is given here because nothing was measured.

Limitations

The fault model is narrow. It covers only buffering, restart and reordering, with no lost or duplicated messages, no diverging clocks and no multiple senders. If the sender’s epoch storage fails and the epoch falls back after a restart, rule C fails in the same way as rule A. The maximum age of 15 s was chosen without reference to any real deployment; a larger value would accept M4, a smaller one would discard fresh messages on slow paths. The trace has seven messages and permits no statement about frequencies, time to a consistent state or performance. Only modelled message paths are considered; suitability for control rooms or real-time operation does not follow.

Conclusion

On the constructed trace, sequence number and age each cover a different class of error, and the restart has to be handled as a rule of its own. The combination of epoch, sequence and maximum age decides all seven cases correctly, but only under the idealised assumptions stated above. The value of the example lies in its traceable comparison logic, not in the numbers. A robust assessment would require varied loss patterns, diverging clocks and independent verification.

Background literature

Real, verified sources. They are not part of the report and do not substantiate its content.

  • Lamport, L. (1978). Time, clocks, and the ordering of events in a distributed system. Communications of the ACM, 21(7), 558–565. https://doi.org/10.1145/359545.359563 Relevant for why timestamps alone do not fix the order of events in a distributed system and what role logical counters play.
  • Banks, A., Briggs, E., Borgendale, K., & Gupta, R. (Eds.). (2019). MQTT Version 5.0 (OASIS Standard, 7 March 2019). OASIS. https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html Relevant as an example of how a widely used messaging standard describes session state across connection loss (section 3.1.2.11.2, Session Expiry Interval) and stored state messages (RETAIN flag, section 3.3.1.3).
  • Eddy, W. (Ed.). (2022). Transmission Control Protocol (TCP) (RFC 9293). IETF. https://www.rfc-editor.org/info/rfc9293 Relevant for the use of sequence numbers for ordering and duplicate detection (section 3.4), although the report does not model any TCP behaviour.