3 DXSpiderWeb Orthogonal Communications Protocol
9 <Origin>,<TimeSeq>,<Hop>,<FrmUser>,<To>,<ToUser>|<Tag>,<Data>...
13 For many years DX Clusters have used a protocol which was designed
14 for a non-looped tree of nodes. This environment has probably never, reliably,
15 been achieved in practice; certainly not recently.
17 There have always been loops, sometimes bringing the network to its
18 knees. In modern usage, both in order to get some resilience and also
19 to expedite information flow, we use internet based, deliberately
20 looped networks with filtering. Whilst this works, after a fashion, there
21 are all sorts of problems that the current PC protocol can never
25 describes a complete replacement for the PC protocol. It allows a
26 fully looped network, is inherently extensible and should be simple
27 to implement (especially in perl).
29 All implementations of this protocol shall B<only> use this protocol
30 for inter-node communications.
34 This protocol is encoded in UTF8 with HTTP style escaping. It is
35 designed to be an extensible basis for any type of one to many
36 "instant" line-based communications tasks.
38 This protocol is designed to be flood routed in a meshed network in
39 as efficient a manner as possible.
41 Each message consists of a L</Routing Section> and a L</Command Section>.
42 The two sections are separated with the '|' character and the whole
43 message is terminated in the standard RFC/Internet manner with the
44 ascii <carraige return><linefeed> characters. It follows that these
45 characters (as well as a small number of other reserved characters)
46 can only be sent escaped. This is described further in the
49 Most of this document is concerned with the L</Routing Section>, however
50 some L</Standard Commands> which all implementation should issue and
51 must accept are described.
53 =head1 Routing Section
55 The application that implements this protocol is essentially a line
56 oriented message router. One line equals one message. Each line is
57 effectively a datagram.
59 It is assumed that nodes are connected to
60 each other using a "reliable" streaming protocol such as TCP/IP or
61 AX25. Having said that: in context, messages in this protocol could be
62 multi/broadcast, either "as is" or wrapped in some other framing
65 Because this is an unreliable, best effort, "please route my packets
66 through your node" protocol, there is no guarantee that a message
67 will get to the other side of a mesh of nodes. There may be a
68 discontinuity either caused by outage or deliberate filtering.
70 However, as it is envisaged that most messages will be flood routed or,
71 in the case of directed messages (those that have L</To> and/or
72 L</ToUser> fields) down some/most/all interfaces showing a route for that
73 direction, it is unlikely that messages will be lost in practice.
75 =head2 Field Description
77 Only the first three fields in the L</Routing Section> are compulsory
78 and indicate that this is a broadcast to be sent to all nodes coming
79 from the L</Origin>. If the message needs to be identified as coming
80 from a user on a node, then the L</FrmUser> field is added.
82 Adding a L</To> and/or L</ToUser> field will restrict the destinations
83 or recipients that receive this message.
85 The L</Hop> field is incremented on receipt of a message on a node.
87 Fields are separated by the comma ',' character with the last field
88 required followed by the vertical bar '|' character.
90 If trailing fields are missed out then superfluous commas can also
91 be left out. If intervening fields are missing then no space needs
92 to be left for the separating comma.
94 The characters allowed in the routing section are restricted. Any
95 invalid characters in any field will cause the whole message to be
98 More detailed descriptions of the fields follow:
104 This is a compulsory field. It is the name of the originating node.
105 The field can contain up to 12 characters in the set [-A-Z0-9_] in
106 any order. Higher layers may restrict this further.
108 The field must not be changed by any other node.
112 This is a compulsory field. It is a 10 hexadecimal digit string which
113 consists of a day no (1-31),
114 a flag to indicate NTP syncronisation in use,
115 seconds within that day (0-86399) [total of 6 hex digits]
116 that are concatenated with a sequence number (0-65535)
117 [4 hex digits] making the total of 10 hexadecimal digits.
119 The date portion is constructed as:
121 my $date = ((((gmtime)[3] < 1) | $ntpflag) < 18) | (time % 86400);
123 The sequence number is simply an unsigned short (or 16 bit) number
126 Each message originated at this node will increment the sequence
131 This is a compulsory field. It is the number of hops from the
132 originating node. It is incremented immediately on receipt and
133 before determining its value.
135 So the originating node sends a message with a L</Hop> of 0, the
136 neighbouring nodes must increment this field before passing
137 it on to higher layers for onward processing.
139 Implementations may have an upper limit to this field and may
140 silently drop incoming messages with a L</Hop> count greater than the
145 This field is optional. It is the identifier of the originating
146 user. If it is missing then the message is
147 assumed to come from the originating node itself.
149 It can consist of up to 12 characters in the set [-A-Z0-9_]
150 in any order. Higher layers may restrict this further.
154 This field is optional. It is a string of up to 12 characters
155 in the set [-A-Z0-9_] in any order.
157 This field is used either to indicate particular node destination
158 or to differentiate this broadcast in some way by making this
159 message as a member of a L</Channel>. Any message can be sent
160 down any L</Channel>. The names of L</Channel>s and their usage
161 is entirely up to the implementor.
163 It is assumed that node names can be differentiated from user
164 names and L</Channel> names.
166 If the field is set to a particular node destination, it will
167 be routed (rather than broadcast) to that node. However, any
168 intervening nodes are free to duplicate the message and send
169 it down more than one, likely looking, interface - depending on any
170 network policies that may pertain.
174 This field is optional. It is a string of up to 12 characters
175 in the set [-A-Z0-9_] in any order. Higher layers may restrict
178 Conventionally this field is used to indicate the user to whom
179 this message is directed. In an ideal world the L</To> field
180 will be set, by the originating node, to the identifier of the node
181 on which this user resides.
183 If the L</To> field is not set then this message will be
184 broadcast. However, should a node become apparent (on route)
185 then nodes are free to fill in the L</To> field and proceed
186 with a more directed approach.
188 If it becomes apparent (on route) that there may be more than
189 one possible L</To> destination for a L</ToUser> then a node
190 may duplicate the message (keeping the same L</TimeSeq>) and
191 route it onwards. Because of the L</DeDuplication> inherent in
192 the system, it is indeterminate as to which destination will
193 receive the message. It is possible for all or just some
194 destinations to receive the message. The tuple (L</Origin>,
195 L</TimeSeq>) will determine uniqueness.
197 This field can, in the case where L</To>
198 is set to the name of a node, be set to a L</Channel>. If this
199 is the case then this will cause this message to be sent to
200 a L</Channel> on the L</To> node only.
206 Channels are a concept very similar to that on IRC. It is a
207 way of segregating data flows in a network. In principle, subject
208 to local policy or application requirements, any data (or
209 L</Command Section>) can be sent down any channel.
211 It is up to the implementation whether to use this feature or not.
215 It is assumed that nodes will be connected in a looped network with
216 more than one route available (in many cases) to another node.
218 In anycase, most traffic is not directed, but broadcast to all users
221 Each message is uniquely identified by the (L</Origin>,L</TimeSeq>)
222 tuple. The basic system will learn which interfaces can see what nodes
223 by looking at the tuple and merging that with the L</Hop> count.
224 Each interface remembers the latest L</TimeSeq> with the lowest L</Hop>
225 for each L</Origin> that arrives on that interface. It also remembers
226 the number of messages for that L</Origin> that has been received on
229 Any message for onward broadcast is duplicated and sent out on all
230 interfaces that it did not come in on.
232 Any message that is directed to a particular node will be sent out on
233 the "best" interface based on routing information gathered so far. If there
234 is more than one possible route then, depending on network or local
235 policy, the message may be duplicated and sent on other interfaces
240 On receipt of a message, its unique tuple (L</Origin>,L</TimeSeq>) is
241 checked against a hash table. If it exists: the message is silently
242 dropped. If it does not exist in the hash table then the tuple is
245 The hash table is periodically cleaned, removing tuples that
246 have expired. The length of time a tuple remains in the hash table
247 is implementation dependant but could easily be several days, if
250 This mechanism only ensures that a message broadcast around the network
251 travels the least distance and through the fewest nodes possible. It
252 is up to higher layers to make sure that data carried is not, itself,
257 # on link startup from GB7BAA (both sides hello)
258 GB7TLH,3D02350001,0,GB7BAA|HELLO,Aranea,1.2,24.123
259 GB7BAA,3D02355421,1,GB7TLH|HELLO,Aranea,1.1,23.245
261 # on user startup to GB7TLH
262 GB7TLH,3D042506F2,0,G1TLH|HELLO,PClient,1.3
264 # on user disconnection
265 GB7TLH,3D9534F32D,0,G1TLH|BYE
267 # a talk (actually 'text') message to a user (some distance away
268 # from the origin node)
269 GB7TLH,3D03450019,3,G1TLH,GB7BAA,G8TIC|T,Hiya Mike what's happening?
271 # a talk/chat/text message to a channel or group
272 GB7TLH,0413525F23,2,G1TLH,VHF|T,2m is opening on MS
274 # a ping to find the whereabouts and distance of a user from a node
275 # the hex number on the end is the ping ID
276 GB7TLH,1512346543,0,,,G7BRN|PING,9F4D
278 # the same from a user on GB7TLH
279 GB7TLH,1512346543,0,G1TLH,,G7BRN|PING,23
281 # this effectively asks whether the user is on-line on a particular node
282 GB7TLH,1512346543,0,G1TLH,GB7DJK,G7BRN|PING,35DE
284 # A possible reply, same ID as ping followed by the no of hops on the
285 # ping that was received
286 GB7DJK,1512450534,3,G7BRN,GB7TLH,G1TLH|PONG,35DE,3
289 =head1 Command Section
291 The L</Command Section> of the message contains the actual data being
292 passed. It is called the Command Section because all commands
293 are identified with a L</Tag> each of which is implemented by
294 the software using this protocol. Each </Tag> (usually) is followed by one
299 The L</Tag> consists of string of uppercase letters and digits, starting
300 with a leading, uppercase, letter. Tags should be as short as is meaningful.
308 Invalid tags include:
314 The L</Tag> is separated from its data L</Fields> by a comma ','.
319 in any subsequent data shall be separated by a comma ','.
321 be HTTP encoded such that reserved characters (comma ',',
325 and non printable characters less than 127 (or %7F in hex)
326 [including newline and carraige return] are tranlated to
327 their two hex digit equivalent preceeded by the percent '%' character.
331 "%0D%0A" is "<carriage return><linefeed>".
332 "hello%2C there" is "hello, there"
334 This is not standard CSV, fields are not quoted (delimited with either
337 All national characters above 127 are UTF8 encoded in the
338 standard perl 5.8.x way. It follows that all (perl) programs that
339 are written according to this specification must say:
343 A message (or line) is terminated with <carriage return><linefeed>
344 0x0d 0x0a. Incoming messages must be accepted even when terminated
345 with just <linefeed>.
347 Care must be taken to make sure that fields have any reserved characters
348 encoded. In particular: it is perfectly permissible to have <linefeed>
349 characters in a field - so long as they are escaped.
351 Fields come in two styles: either simple fields (just containing
352 data) or B<key>=B<value> pairs. Each pair must be separated from
353 the next by a comma ','. The B<key> must consist of the set of
354 characters [a-z0-9_] (ie lowercase letters, digits and underscore),
355 with a leading letter. The B<value> must be HTTP encoded as
356 specified above and can otherwise contain any character.
358 There is no maximum size specified for a message. It is up to each
359 implimentation to enforce one (if only for their own protection).
361 =head2 Standard Commands
363 There are a number of L</Standard Commands> which must be accepted by
370 HELLO,<software name>,<version>,<build>,<comments>
372 Command sent on connection to another node. Both sides send their information
373 to the other. All the possible arguments are optional, although some of the
374 arguments should be sent in order to help diagnose problems. This command is
381 Command sent to all connections when the software is shutting down. This is sent
382 by the node just before shutdown occurs. This is really only used to help the
383 network prune its routing tables. It isn't a requirement. The <comment> field
388 DISC,<node name>,<comments>
390 Command sent when a node has disconnected from this node. This message is sent when
391 an interface shuts down. It need not be sent if a L<BYE> from an interface for
392 that node has just been received. This command should be broadcast.
394 The <node name> is mandatory and is the name of the interface that has just
401 Command to send a ping to a node or user. This command is used both by the software
402 and users to determine a) whether a node or user exists and b) how good the path is
405 The <ping id> is a unique string which is usually the hexadecimal equivalent of an
406 integer that is incremented every time it is used. But it can be anything that
407 will identify this ping using the tuple (L<Origin>,<ping id>) as unique.
411 PONG,<ping id>,<no of hops on ping>
413 Command to reply to a ping. This is sent as a reply to an incoming ping command.
414 The <ping id> is the one supplied and the <no of hops on ping> is the number of
415 hops it took for the ping to arrive.
421 All implementations must be able to send "text" (encoded as specified in
422 L</Fields>). There would be little point in doing all this otherwise!
428 Dirk Koopman, G1TLH, E<lt>djk@tobit.co.ukE<gt>
430 =head1 COPYRIGHT AND LICENSE
432 Copyright 2004 by Dirk Koopman, G1TLH
434 This library is free software; you can redistribute it and/or modify
435 it under the same terms as Perl itself.