The system for incorporating dynamic data into messages on the current controller has worked well, but (due to the hardware it had to work under) it has several limitations:
To solve the above limitations, I propose to create the following networked standard for data sources:
Each controller will run a Source Daemon which acts as the central repository for data sources. All data sources which wish to offer a source to a display on the controller will register with the Source Daemon. Each display driver on the controller, upon loading a sequence, will contact the Source Daemon and request connections from any sources required to display its sequence. Any software for message composition will be able to query the Source Daemon for a list of available sources and their characteristics. Also, the scheduler for each display will contact the Source Daemon and arrange notification of the special events that it wants to know about.
All protocols in this proposal are assumed to be layered over TCP/IP (or some equivalent protocol, such as unix domain sockets) to provide sequenced, reliable communication with multiple ports per host.
Since this proposal defines a set of network protocols, it is important to make sure that everything is secure. Therefore, every protocol is initiated with a challenge-response authentication. Each controller will be assigned an administrator-changeable "network password" that will be used as the initial authentication for every network connection made to the box. (This only applies to the standards which we have defined; obviously the web daemon and the NTCIP daemon will authenticate to their clients in a manner appropriate to them.)
All programs on the local controller will have access to the "network password", so it won't be necessary to configure them specially (and moreover, they will always have guaranteed access to the controller). Only external network sources will need to be told the network password.
Note: the network password is separate from any individual password; it is presumed that any software client which might run on a user's computer will cache this password so that it doesn't need to ask them again.
There will be one source daemon per controller which will coordinate all data sources for that controller. It will listen for connections from Data Sources on port 30101 and use Data Source to Source Daemon Protocol (DSSDP) to communicate (documented below). It will listen for connections from display drivers on port 30201 and use Display Driver to Source Daemon Protocol (DDSDP) to communicate (documented below). It will listen for connections from schedulers on port 30301 and use Scheduler to Source Daemon Protocol (SSDP) to communicate (again, documented below). It will listen for connections from clients who want a list of the available data sources on port 30401 and use Source Query Protocol (SQP) to communicate (documented below).
When a Display Driver is loading a sequence which contains dynamic messages, it will contact the Source Daemon and request connections from the Data Sources that it needs to display the sequence. If any of the Data Sources are not available, for the affected messages the Display Driver should display the alternative message specified in the message, or blank if no alternative message is specified. If the Data Source is not available, or if at any time the Display Driver loses contact with the Data Source, it should periodically attempt to request a new connection from the Source Daemon.
Data sources will be able to specify which of two modes of operation they will use. In the first mode, when a Display Driver is about to display a dynamic message, it will ask the Data Source for the data to use in the display. The Data Source will only have a few milliseconds to respond. This mode is suitable for local data sources which do not take long to generate their data, or sources over a very fast network (such as ethernet).
In the other mode, the Data Source will send a data packet whenever it wants to. The Display Driver will maintain a 1 datum FIFO for data packets and whenever it requires a datum to display a dynamic message, it will use the contents of the FIFO. This mode is more suitable for data sources which do not frequently generate data, or which are over a slower connection.
Each data source has an associated signed 32 bit integer which is used to indicate events. The meaning of this integer — if it has a meaning at all — is up to the individual source. Each Data Source is required to describe whatever meaning that it has in its description text. Events will also be time-stamped and carry an indication of low long they are valid for. (The duration of validity will be dependent on how the data is read, etc.)
A Scheduler may indiciate to the source daemon, and the Source Daemon in turn to the Data Source, what data sources it's interested in. Only Data Sources with interested parties should bother to send their events. When a Data Source generates an event it then sends an Event Packet to the Source Daemon, which then forwards it to the appropriate Scheduler(s). The Scheduler then does whatever it wants with this data. (Presumably, it will cause a new sequence to be displayed.)
An integer value was chosen because most data types should be mappable to an integer range. The way to map speed from a radar gun and temperature should be obvious. Other numeric values have obvious mappings, such as the count from a car counter, the weight from a truck scale, or the intensity from a light meter. However, other values should be mappable to an integer by special meanings. For example, if two messageboards are being used in place of flag men, a data source could send 1 for the "let trucks pass" event and 0 for the "tell your trucks to stop" event. (that example is a stretch, since this would be better handled by a single source generating the data sources and a fairly high refresh rate, but hopefully you get the idea.)
The scheduler will store the most recent event packet from every Data Source it's got an event-driven schedules for. The scheduler will use this to display an event for which there is valid data in the correct threshold (unless a time-based sequence has a higher priority). The scheduler is responsible for always examining all of the valid event data to determine which if any events are active.
Various clients which present a user interface will want to know what data sources are available in order to present them to the user in a friendly manner. The will connect to the Source Daemon and maintain the connection for as long as they run; the source daemon will notify them of new data sources which become available or disconnected ones which are no longer available as that happens.
There are fundamentally two different types of messages which can use data sources: text messages and graphical messages (which might include text in text areas). Dynamic text messages will be specified like normal messages with special tags embeded in them. Images will be specified in an XML-like format.
Note: the following describes data sources mostly in their implementation; it is assumed that they will typically be entered and edited via a GUI which hides the complexity involved behind a user-friendly front-end.
Data sources will be available to use in text messages by using a special escape sequence and naming the data source. For example, if a radar gun registers a data source called Radar (which provides a string of length 2), a message source which uses it might look like:
YOUR SPEED IS <Radar> MPHWhich might be displayed as:
YOUR SPEED IS 43 MPH
If a source requires an argument, it may be passed by adding a colon to the source ID and appending the comma-separated arguments, either a string which does not contain a > character, or by enclosing the argument in quotation marks and using standard C-style backslash escaping. While this is available, it is not the recommended idiom, and sources should supply multiple source IDs in preference to using an argument.
An example argument:
YOUR SPEED IS <Radar:units=kph> KPH EH?Which might be displayed as:
YOUR SPEED IS 69 KPH EH?
When a Data Source registers a source ID, it must specify the default behavior if it becomes unavailable. If the default is an alternate message, the Data Source must specify a text message to display. The defaults may be overwridden by the message using the source.
To override the default failure behavior of a message, one supplies a text message in square brackets after the source ID, but before any arguments. For example:
YOUR SPEED IS <Radar["DRIVE SAFELY"]> MPHor, if an argument is also required:
YOUR SPEED IS <Radar["DRIVE SAFELY"]:units=kph> KPH EH?This should typically not be necessary, as a source should specify an appropriate failure behavior, but it is available to the user if they want something else.
If a user simply wants to skip the message entirely if the source fails, and this is not already the default behavior, they should just supply a blank message. E.g.
YOUR SPEED IS <Radar[]> MPHThis will replace the current message with a brief blank if the data source is unavailable. If the message is the only one in the sequence, that will result in a blank sign. (In most cases, that would be the desired behavior in case of failure.) Note: the same will happen if the data source specifies a blank message as its default alternative message.
Dynamic graphical messages will use an XML-like syntax (xml-like because it will not use a DTD or allow recursion, so that parsing does not require a recursive descent parser (or stronger)).
A dynamic graphic must begin with:
<graphic sourceID="sourceID">Next, it may contain an override of the alternate message to display, in the form:
<alternate>alternate text</alternate>Alternates, even for graphical sources, are restricted to static text messages for simplicity. Alternates should never actually be used — they represent a failure condition — so they don't need to be fancy; they only need to be functional.
Next, it may contain a parameter in the form:
<param name="key">value</param>After this, one may specify the text to insert into one or more text areas defined by the source, in the form:
<textarea>text for textarea 1</textarea>Textareas are ordered by their upper-left coordinate, from top-to-bottom, right-to-left. Any text specified for a textarea which is not defined by the source is simply ignored. (I.e. if a source defined 2 textareas and a message specifies 3 textarea strings, the third is silently ignored.) (Note: silently ignoring extraneous strings is not recommended for the input gui, only for the Display Driver.)
Finally, a dynamic graphic must be terminated by:
</graphic>The Data Source family of protocols are versioned, variable-sized packet protocols. They all begin with a challenge-response authentication phase (prior to any handshaking or version exchange) where the listener issues a unique 20 byte challenge. The correct response is the 20 byte SHA1 hash of the concatenation of the challenge and the secret password. (i.e. SHA1(challenge + password).) The initiator of the connection only gets one chance; if they do not return the correct response, the connection is terminated. Also, the connection is terminated if the complete response is not recieved within 10 seconds. (That should never be an issue since virtually all connections which don't come from the local machine will come via a high-speed network (ethernet, 802.11b, etc.), but this is here to deter DOS attacks.)
Note: unless otherwise specified, all protocols use network ordered (i.e. big-endian) orderings of multi-byte integeral types. Further, the data type String is defined as an unsigned short indicating length, followed by that many bytes of UTF-8 data.
The following packets may be used in more than one protocol:
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 0 | Packet Type |
2-65,537 | String | The source ID | |
2-65,537 | String | A human-readable description of what data the source provides, as well as what its event number means (if it generates any meaningful event numbers). | |
1 | unsigned byte | The minimum length of the supplied text. | |
1 | unsigned byte | The maximum length of the supplied text. |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 1 | Packet Type |
2-65,537 | String | The source ID | |
2-65,537 | String | A human-readable description of what data the source provides, as well as what its event number means (if it generates any meaningful event numbers). | |
2 | unsigned short | The minimum width of the supplied graphic. | |
2 | unsigned short | The minimum height of the supplied graphic. | |
2 | unsigned short | The maximum width of the supplied graphic. | |
2 | unsigned short | The maximum height of the supplied graphic. |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 2 | Packet Type |
2-65,537 | String | The source ID |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 5 | Packet Type |
2-65,537 | String | The source ID |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 3 | Packet Type |
2-65,537 | String | The source ID | |
4 | signed integer | The numerical value of the event. What this means is source-dependent. | |
8 | signed long | the timestamp in epoch milliseconds | |
4 | unsigned integer | the duration (in milliseconds) which this packet is valid for. |
Byte Count | Data Type | Value | Description | ||||||||
1 | Unsigned Byte | 4 | Packet Type | ||||||||
2-65,537 | String | The URI for the Data Source to connect to. | |||||||||
1 | Unsigned Byte | Data Source block Count | |||||||||
variable | The data sources desired. Each block is of the form:
|
||||||||||
The source daemon is responsible for forwarding this packet to as many data source providers as necessary. Note: the data source will be responsible for processing the argument; the display driver should send it unaltered. |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 12 | Packet Type |
2-65,537 | String | The source ID for the source which is no longer available. |
The protocol is a versioned, variable-length packeted protocol which begins with a handshake. The handshake packets do not contain a packet type because their required order make it redundant.
The handshake packets are, in order:
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The maximum version of DSSDP supported. |
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The version of DSSDP to be used. This must be <= the maximum supported version sent by the Source Daemon. If the Source Daemon cannot support a version great enough, the Data Source should simply drop the connection and report an error. |
At this point, the Data Source may send the following packets at any time:
And the Source Daemon may send any of the following packets at any time:
If there are ever zero Event Filters registered for a source, it should not send any events. This will be the case on startup until the Source Daemon sends an Event Filter Packet, and any time all Event Filters have been canceled with Event Filter Cancellation Packets.
The protocol is a versioned, variable-length packeted protocol which begins with a handshake. The handshake packets do not contain a packet type because their required order make it redundant.
The handshake packets are, in order:
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The maximum version of DDSDP supported. |
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The version of DDSDP to be used. This must be <= the maximum supported version sent by the Source Daemon. If the Source Daemon cannot support a version great enough, the Data Source should simply drop the connection and report an error. |
At this point, the Display Driver may send the following packets:
Note: There is no notification from the Source Daemon to the Display Driver as to what sources are available because the Display Driver can't do anything with this information anyway. All it cares about is whether it has a connection from a data source which is offering the sourceID that it wants. It will try to get these connections, but so long as it doesn't have them, it's entirely irrelevant to the Display Driver why it doesn't have them.
It is, of course, possible for the Source Daemon to generate warning messages about requests for non-existant sources to help detect stolen radar guns, but that behavior is outside of the scope of this specification.
The protocol is a versioned, variable-length packeted protocol which begins with a handshake. The handshake packets do not contain a packet type because their required order make it redundant.
The handshake packets are, in order:
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The maximum version of SSDP supported. |
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The version of SSDP to be used. This must be <= the maximum supported version sent by the Source Daemon. If the Source Daemon cannot support a version great enough, the Scheduler should simply drop the connection and report an error. |
At this point, the Scheduler may send the following packets at any time:
The Source Daemon may send the following packets at any time:
The protocol is a versioned, variable-length packeted protocol which begins with a handshake. The handshake packets do not contain a packet type because their required order make it redundant.
The handshake packets are, in order:
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The maximum version of SQP supported. |
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The version of SQP to be used. This must be <= the maximum supported version sent by the Source Daemon. If the Source Daemon cannot support a version great enough, the client should simply drop the connection and report an error. |
The client may not send any packets. The Source Daemon may send any number of Text Source Information Packets, Graphics Source Information Packets, or Source Unavailable Pakets.
The protocol is a versioned, variable-length packeted protocol which begins with a handshake. The handshake packets do not contain a packet type because their required order make it redundant.
The handshake packets are, in order:
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The maximum version of DSDDP supported. |
Byte Count | Data Type | Value | Description |
1 | unsigned byte | 0 | The version of DSDDP to be used. This must be <= the maximum supported version sent by the Display Driver. If the Display Driver cannot support a version great enough, the Data Source should simply drop the connection and report an error. |
The Display Driver will then immediately send a Display Driver Description Packet. After receiving a Data Source Description Packet, the Display Driver may optionally send an Argument Packet to it.
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 7 | Packet Type |
2 | unsigned short | The width of the display (in pixels) | |
2 | unsigned short | The height of the display (in pixels) | |
1 | unsigned byte | The bits-per-pixel used by this display (note: this is not a restriction; images can be up- or down-converted as necessary; this is just to allow the transfer to be maximally efficient.) | |
1 | unsigned byte | The bits-per-color used by this display (note: this is not a restriction; images can be up- or down-converted as necessary; this is just to allow the transfer to be maximally efficient.) |
Byte Count | Data Type | Value | Description | ||||
1 | Unsigned Byte | 6 | Packet Type | ||||
2-65,537 | String | The source ID | |||||
1 | unsigned byte | Data type:
|
|||||
1 | unsigned byte | Transfer mode:
|
|||||
2-65,537 | String | The text of the alternate message |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 8 | Packet Type |
2-65,537 | String | The source ID | |
2-65,537 | String | The argument string. Note: the data source will be responsible for processing the argument; the display driver should send it unaltered. |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 9 | Packet Type |
2-65,537 | String | The source ID |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 10 | Packet Type |
2-65,537 | String | The source ID | |
2-65,537 | String | The text data |
Byte Count | Data Type | Value | Description |
1 | Unsigned Byte | 11 | Packet Type |
2-65,537 | String | The source ID | |
variable | Image | The image data |
Note: If the Data Source selected request-as-needed mode, it has 10ms to respond to a Data Request Packet before it is considered unavailable and the alternate message is used. If it sends its Data Packet more than 10ms after it recieves the Data Request Packet, the data packet will be silently ignored.