Revision 8

Librarian Protocol

Each controller will have a single librarian which is responsible for storing all messages and sequences. All other programs will request the sequences that it needs from the Librarian, using the sequence's title.

This arrangement will minimize disk usage and maximize reliability and flexibility.

The Librarian

(Note: because the storage formats for sequences and messages both indicate what sort they are as their first byte, either may be stored in a library. Therefore in this document libraries will be said to store "items", which will refer to messages or sequences. This is not meant to imply that any such terminology should be used with users, who would probably find it confusing.)

The Librarian will maintain a minimum of two libraries: one called "Sequences" and one called "Sequences-reference". The librarian will also support user-defineable libraries. Users will be able to create named libraries and store items in them. At any given time, one Library will be the "active" Library. When selecting a sequence for a scheduler, the user will only be presented with the active library. When the Display Driver wants a sequence, it pulls it from the active library. Etc. Users will be able to set the active library from among all of the libraries present.

In general, when a new user Library foo is created, a library called "foo-reference" should also be created. The main library should be for sequences which might be displayed, whereas the foo-reference library is meant to store components for building up sequences. (The name "foo-reference" should probably not be presented to the user exactly like that.) Furthermore, every library will be able to specify one or more auxiliary Libraries. These do not directly affect the list of items available in a library, but a program may query the auxiliary libraries and in some cases present them to the user along with the items from the library being used. This is primarily intended to allow auxiliary reference libraries (which are set read-only) to aid in creating sequences for display.

Every library may be set read-only, so that it cannot be modified until it this flag is turned off.

Users will also be able to set information dependent on the active library. For example, one can restrict the fonts which are available from the font manager based on the active library. Also, one can specify whether special effect transitions are permitted based on the library. (note: the restrictions are only enforced by the active library.)

(note: all of the above is about the back-end. There's no requirement that we expose custom libraries to actual users, but by maintaining it, we give ourselfs a lot of flexibility at virtually no cost.)

This will allow users to create libraries for particular venues that a sign alternates between. For example, if a portable sign is time-shared, the users could create one library per time-sharer, so that they can maintain separate libraries without causing confusion or accidentally overwriting each other's sequences (as easily). A fixed sign might have seasonal libraries, so that there would be a Christmas Sequence Library, an Easter Sequence Library, a Summer Sequence Library, a Back To School Sequence Library, etc.

Also, a portable sign which is used part of the year for road construction and part of the year for advertising might keep its sets of messages in two separate libraries. (Since we will be emphasizing remote access, especially by laptops, PDAs, cell phones, etc. it is important to keep all information on the sign so that any device has access to the data that it needs, and the user isn't chained to one computer to do all his sign control.)

The Protocol

The Librarian Protocol is a variable-length packet based protocol which is initiated by a challenge-response authentication phase, followed by a versioning handshake.

The Librarian will listen for connections using this protocol on port 40101.

Note: unless otherwise specified, all multi-byte integral types are in network-order (i.e. big-endian). The data type String is defined as an unsigned short indicating the number of bytes in the string, followed by that many bytes of UTF-8 data.

Handshake

The protocol handshake works according to the Solartech Protocol Handshake & Authentication specification. The current protocol version is 1.

Normal Operation

This is normally a synchronous protocol, where the librarian only sends packets in response to packets from the client. The packets should be sufficiently well-named and described so that their use is obvious.

Library Revisions

In order to facilitate low bandwidth communications, a global libraries revision number was added to the librarian and associated protocols.

The motivating problem is that every time one connects to a unit, it is impossible to know with certainty what changes might have happened since the last time one was connected without requesting the entire library. This is both time consuming and expensive in terms of bandwidth. The solution is for each unit to have a global revision number which increments every time any message in any library changes. On reconnection, the client can simply query the library revision number to get a guarantee that it's idea of what is in the library is correct.

The revision number is a 32 bit (unsigned) number, since it will be incremented rather than randomly assigned. Even in the pathological case of 1 update per second, this number would only wrap around every 136 years. Additionally, wrap-around is not significant, since the client does not care about the difference between revision numbers, only whether the current revision number is the same as the last revision number it saw. Thus the only real requirement to handle even this pathological case is for the client to check the unit at least once every 135 years. This should not be overly burdensome.


Appendix A. Packet Definitions

Item Insertion Packet
Byte Count Data Type Value Description
1 unsigned byte 0 Packet Type
variable Sequence or Message This packet embeds the item which is to be inserted into the active library.
Library Item Insertion Packet
Byte Count Data Type Value Description
1 unsigned byte 1 Packet Type
2-65,537 String The name of the library into which to insert the sequence info
variable Sequence or Message This packet embeds the item which is to be inserted into the named library.
Item Request Packet
Byte Count Data Type Value Description
1 unsigned byte 2 Packet Type
2-65,537 String The title of the sequence to be fetched from the active library.
Library Item Request Packet
Byte Count Data Type Value Description
1 unsigned byte 3 Packet Type
2-65,537 String The Library to fetch the sequence from.
2-65,537 String The title of the sequence to be fetched from the active library.
Item List Request Packet
Byte Count Data Type Value Description
1 unsigned byte 4 Packet Type
Library Item List Request Packet
Byte Count Data Type Value Description
1 unsigned byte 5 Packet Type
2-65,537 String The name of the Library that this request is for.
Item List Packet
Byte Count Data Type Value Description
1 unsigned byte 6 Packet Type
4 unsigned integer (count) The number of sequences in the library
variable String[count] The item titles.
Library Item List Packet
Byte Count Data Type Value Description
1 unsigned byte 7 Packet Type
2-65,537 String The Library Name
4 unsigned integer (count) The number of items in the library
variable String[count] The item titles.
Insertion Acknowledgement Packet
Byte Count Data Type Value Description
1 unsigned byte 18 Packet Type
2-65,537 String The name of the library into which the sequence has been inserted (blank if it's the active library)
2-65,537 String The name of the sequence which has just been inserted
Error Packet
Byte Count Data Type Value Description
1 unsigned byte 20 Packet Type
4 integer Error code
2-65,537 String The error message, encoded in UTF-8.
Deletion Acknowledgement Packet
Byte Count Data Type Value Description
1 unsigned byte 19 Packet Type
2-65,537 String The name of the library from which the sequence has been deleted (blank if it's the active library)
2-65,537 String The name of the sequence which has just been deleted.
Item Deletion Packet
Byte Count Data Type Value Description
1 unsigned byte 8 Packet Type
2-65,537 String The title of the sequence to be deleted.
Library Item Deletion Packet
Byte Count Data Type Value Description
1 unsigned byte 9 Packet Type
2-65,537 String The Library Name
2-65,537 String The title of the item to be deleted.
Active Library Query Packet
Byte Count Data Type Value Description
1 unsigned byte 12 Packet Type
Notes:
Requests that the librarian sends a Active Library Packet.
Active Library Packet
Byte Count Data Type Value Description
1 unsigned byte 13 Packet Type
2-65,537 String Library name.
Notes:
When sent to the Librarian, this sets the named library to be the active library. When sent by the librarian, this indicates which the active library is.
Library Creation Packet
Byte Count Data Type Value Description
1 unsigned byte 10 Packet Type
2-65,537 String Library Name
On success, the server sends back a Library Creation Packet. On error, it sends back an error packet.

NOTE: creating an existing library has no effect, but is not an error, and is treated as a success condition.
Library Deletion Packet
Byte Count Data Type Value Description
1 unsigned byte 11 Packet Type
2-65,537 String Library Name
On success, the server sends back a Library Deletion Packet. On error, it sends back an error packet.

NOTE: deleting a library which doesn't exist results in an error.
Library Font Restriction List Packet
Byte Count Data Type Value Description
1 unsigned byte 14 Packet Type
2-65,537 String Library Name
2 unsigned short (count) The number of fonts in this packet.
variable String[count] The restriction list for this library.
Notes:
When sent to the Librarian, this sets the specified library's restriction list. When sent by the Librarian, it indicates what it's current restriction list is. The restriction list specifies which fonts are permitted (one may think of it as an opt-in list). Note: as a special case, a zero-element restriction list means that any font is allowed.
Library Font Restriction List Insertion Packet
Byte Count Data Type Value Description
1 unsigned byte 15 Packet Type
2-65,537 String The Library's name.
2-65,537 String The font's name.
Notes:
The librarian will send this packet back as a confirmation of deletion, or an Error Packet if there was an error.
Library Font Restriction List Deletion Packet
Byte Count Data Type Value Description
1 unsigned byte 16 Packet Type
2-65,537 String The Library's name.
2-65,537 String The font's name.
Notes:
The librarian will send this packet back as a confirmation of deletion, or an Error Packet if there was an error.
Library Font Restriction List Query Packet
Byte Count Data Type Value Description
1 unsigned byte 17 Packet Type
2-65,537 String Library name.            
Rename Library Packet
Byte Count Data Type Value Description
1 unsigned byte 21 Packet Type
2-65,537 String The name of the library to rename
2-65,537 String What to rename it to.
Notes:
The change is confirmed by sending this packet back. If there is an error, it is reported with an Error Packet.
Library Read-Only Flag Packet
Byte Count Data Type Value Description
1 unsigned byte 22 Packet Type
2-65,537 String Library name
1 unsigned byte The value of the read-only flag:
0False
1True
Notes:
When sent to the Librarian, this sets the flag. When sent by the Librarian, this indicates the current value of the flag. When the flag is set to true, the library may not be modified. (When it is set to false, the library may be modified.
Library Read-Only Flag Query Packet
Byte Count Data Type Value Description
1 unsigned byte 23 Packet Type
2-65,537 String The name of the library to query.
Special Effects Enabled Packet
Byte Count Data Type Value Description
1 unsigned byte 24 Packet Type
1 unsigned byte The value of the special effects enabled flag:
0False
1True
Notes:
When sent to the Librarian, this sets the flag. When sent by the Librarian, this indicates the current value of the flag. When the flag is set to true, the Display Driver should render any special effect transitions indicated in the sequence. When the flag is set to false, the Display Driver should skip all special effects transitions.
Special Effects Enabled Query Packet
Byte Count Data Type Value Description
1 unsigned byte 25 Packet Type
2-65,537 String The name of the library to query.
Auxialiary Library Attachment Packet
Byte Count Data Type Value Description
1 unsigned byte 26 Packet Type
2-65,537 String The name of the library to which the auxiliary library is to be attached.
2-65,537 String The name of the auxiliary library which is to be attached.
Notes:
If successful, it is confirmed by recieving an appropriate detachment packet. If unsuccessful, an Error Packet is sent instead.
Auxiliary Library Detachment Packet
Byte Count Data Type Value Description
1 unsigned byte 27 Packet Type
2-65,537 String The name of the library from which to detach the auxiliary library.
2-65,537 String The name of the auxiliary library to detach.
Notes:
If successful, it is confirmed by recieving an appropriate detachment packet. If unsuccessful, an Error Packet is sent instead.
Auxliary Library List Query Packet
Byte Count Data Type Value Description
1 unsigned byte 28 Packet Type
2-65,537 String The name of the library to query.
Auxiliary Library List Packet
Byte Count Data Type Value Description
1 unsigned byte 29 Packet Type
2-65,537 String The name of the library for which this packet is a listing.
2 unsigned short (count) The number of auxiliary libraries listed in this packet.
variable String[count] The names of the auxiliary libraries for this library, arranged in no particular order.
Notes:
When sent by the Librarian, this indicated which libraries are the auxiliary libraries. When sent to the Librarian, it sets the list of auxiliary libraries for the specified library.

If this packet is used to set the auxiliary library list, it is confirmed with an Insertion Acknowledgement Packet. As always, an error is reported with an Error Packet.

Library List Request Packet
Byte Count Data Type Value Description
1 unsigned byte 30 Packet Type
Library List Packet
Byte Count Data Type Value Description
1 unsigned byte 31 Packet Type
4 integer (count) The number of libraries kept by this librarian
variable String[count] The names of the libraries kept by this librarian.
No Such Sequence Packet
Byte Count Data Type Value Description
1 unsigned byte 32 Packet Type
2-65,537 String Library Name
2-65,537 String Sequence Name
Notes:
This packet is send in response to either type of request for a sequence which does not exist in the library. Note: if it's in response to an Item Request Packet, the library field in this packet will be "".
Revision Number Packet
Byte Count Data Type Value Description
1 unsigned byte 33 Packet Type
4 unsigned int Current Revision Number
This packet will be send out to all clients immediately following the notification of any change to a library which would result in changing the revision number.

NOTE: this is a version 11 packet.

Appendix B. Packet types

Packet Type # Packet Type
0 Item Insertion Packet
1 Library Item Insertion Packet
2 Item Request Packet
3 Library Item Request Packet
4 Item List Request Packet
5 Library Item List Request Packet
6 Item List Packet
7 Library Item List Packet
8 Item Deletion Packet
9 Library Item Deletion Packet
10 Library Creation Packet
11 Library Deletion Packet
12 Active Library Query Packet
13 Active Library Packet
14 Library Font Restriction List Packet
15 Library Font Restriction List Insertion Packet
16 Library Font Restriction List Deletion Packet
17 Library Font Restriction List Query packet
18 Insertion Acknowledgement Packet
19 Deletion Acknowledgement Packet
20 Error Packet
21 Rename Library Packet
22 Library Read-Only Flag Packet
23 Library Read-Only Flag Query Packet
24 Special Effects Enabled Packet
25 Special Effects Enabled Query Packet
26 Auxiliary Library Attachment Packet
27 Auxiliary Library Detachment packet
28 Auxiliary Library List Query Packet
29 Auxiliary Library List Packet
30 Library List Request Packet
31 Library List Packet
32 No Such Sequence Packet
33 Revision Number Packet

Last modified: Thu Nov 11 10:52:43 Eastern Standard Time 2004