The connection mutiplexer protocol allows a single streaming (serial or TCP) connection to be multiplexed to carry multiple outgoing or incoming TCPv4 streams. This allows relatively complex multi-connection protocols to be carried over a single connection, which is possible to establish over a serial port, and easier to establish through NATted firewalls over the internet. (The TCP port that the multiplexer runs on is 39186.)
The connection multiplexer protocol is divided into two parts. The framing protocol is responsible for taking the stream, dividing it into 254 channels, and ensuring that data is delivered reliably to those channels. The control protocol runs on channel 0, and is responsible for opening and closing connections. This separation allows the control protocol to benefit from the connection reliability provided by the framing protocol.
Whenever a type longer than one byte is used (two-byte shorts or four-byte longs), the bytes should be sent in network (big-endian) byte order.
The framing protocol is responsible for dividing the stream into up to 254 channels, and ensuring that data is delivered reliably to those channels, even in the case of errors in the underlying data stream.
The errors we handle are byte insertion, byte removal, and byte corruption. We assume that a lower-level protocol is feeding us bytes, so we don't handle bit-level corruption. Since we use checksums, our response to errors is probabalistic. There is around a 1 in 65536 chance an error will yield a valid packet.
Creating a framing protocol connection begins by the client establishing a stream connected to the server. The precise details of establishing this connection vary depending on the underlying transport mechanism.
Once connected, the client sends the string "SOLARTECH-TCPMUX-CLIENT-1\r\n", where \r and \n represent the ASCII CR and LF characters, respectively. The server responds with the string "SOLARTECH-TCPMUX-SERVER-1\r\n", and the protocol is now active in normal operation mode.
Byte Count | Data Type | Value | Description |
1 | byte | 0-254 | channel - The channel number this packet is associated with. |
1 | byte | 0-255 | number - The packet number of this packet. |
1 | byte | 0-255 | ack_number - The packet number of the last packet successfully received from the other host. |
1 | byte | 0-255 | length - The length of this packet's data, in bytes. |
0-255 | bytes | data - The data this packet contains. The length of this data is given by the previous field. | |
2 | short | crc - The CRC-16 of all prior fields in this packet. |
Each packet sent by the local host contains the current local packet number, modulo 256. The packet number starts with 1, increments each time a non-zero-length data packet is sent, and wraps around after 255 has been sent. Packet number must not be sent until packet (number+192)%256 has been acknowleged by the other end of the connection.
When a data packet is sent out, the ack_number should be set to last_number, which is the number of the last packet successfully received from the remote host. This is initialized to 0, and is updated as packets are successfully received.
If a non-empty packet has been received, and ack_timeout has elapsed since a packet has been sent, then an empty packet is sent. An empty packet has channel=0, length=0, number=0, data empty, and crc and ack_number set appropriately.
If a packet has not been acknowledged by the remote host in retransmit_timeout, seconds have elapsed, that packet should be retransmitted.
If no data is recieved for more than 1 second once we have read the first byte of a data packet, a framing protocol error has occured. See the next subsection for how to handle a framing protocol error.
When a data packet has been received with an invalid CRC, a framing protocol error has occured. See the next subsection for how to handle a framing protocol error.
When a data packet has been recieved, the ack_number is looked at to determine which of our packets can be considered acknowledged. For all i in the range 128 to 256 inclusive, packet (ack_number+i)%256 should be considered acknowledged. This occurs even if the packet is then discarded.
If the length field of the packet is 0, the packet is discarded without further processing. (This discards empty packets after their ack_number has been considered.)
When packet number is received, we check to see if number is equal to (last_number+1)%256. If not, the packet is discarded. If so, last_number is set to number, and the data from the packet is fed into the appropriate channel.
A protocol error occurs when data is transmitted incorrectly. For example, when a byte is corrupted, or a byte is lost. When this condition is detected, we have to somehow restore synchronization.
The lost_sync and gained_sync packets are special because the form in which they are recognized is not the form in which they are sent. To transmit a lost_sync packet, send 15 0xffs, followed by an 0xfe. To transmit a gained_sync packet, send 15 0xffs, followed by an 0xfd. To recognize an lost_sync packet, look for between 1 and 15 0xffs, followed by an 0xfe. To recognize a gained_sync packet, look for between 1 and 15 0xffs, followed by a 0xfd.
Byte Count | Data Type | Value | Description |
15 (send), 1-15 (recv) | byte | 0xff | Indicates this is a framing protocol message, allows for synchronization. |
1 | byte | 0xfe | Indicates this is a lost_sync packet. |
Byte Count | Data Type | Value | Description |
15 (send), 1-15 (recv) | byte | 0xff | Indicates this is a framing protocol message, allows for synchronization. |
1 | byte | 0xfd | Indicates this is a gained_sync packet. |
When a host realizes that synchronization has been lost, it begins sending lost_sync packets. It then begins trying to recognize a lost_sync or gained_sync packet. Once one of these packets has been recieved, the host has regained synchronization. It transmits a gained_sync packet, then proceeds to handle the packet as described in the next paragraph.
When a host (that has not lost synchronization, or that has just regained synchronization) receives a lost_sync packet, it transmits a gained_sync packet. It also stops transmitting data packets until retransmit_timeout has elapsed. (This timeout is reset each time a lost_sync packet is received.) Upon receiving a gained_sync packet, or the timeout elapsing, a host can once again begin transmitting data packets. All outstanding packets that have not been acknowledged should be re-transmitted when a gained_sync packet is received.
The control protocol is responsible for creating and destroying channels, and transfering control and configuration information. As it runs atop the framing protocol, as long as the connection remains alive the control packets will eventually be reliably delivered, so we do not need to worry about reliablity in the control protocol.
The control protocol is automatically assigned channel 0 when the connection is first established. There is no way to open or close the control protocol, and there's no need to.
There are three types of packets in the control protocol:
Request packets and response packets contain a request_id field, which is a byte. The response to a given request will contain the same value for the request_id field. While the client is allowed to used the same request_id value for multiple outstanding requests, it probably isn't a good idea, unless the response is expected to be ignored.
As will be indicated in the description of the requests below, most requests can produce an error packet. Some requests can also produce a generic success response, with no additional data. We represent these with generic error and success responses.
Byte Count | Data Type | Value | Description |
1 | byte | 0x01 | Packet ID |
1 | byte | request_id | |
2 | short | errno - An error code, taken from Linux's errno.h. | |
1 | byte | error_string_length - The length of a string giving a textual description of the error code. | |
error_string_length | byte data | error_string - A textual description of the error code. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x02 | Packet ID |
1 | byte | request_id |
The framing configuration request is used to configure parameters for the framing protocol. It should be sent as the first packet of a connection, as the first packet will be delivered even if the various timers are set wrong. The framing configuration response is used to confirm that the request was recieved and acted upon. A success response will be sent once the configuration change has occured.
Byte Count | Data Type | Value | Description |
1 | byte | 0x03 | Packet ID |
1 | byte | request_id | |
4 | long | The new value of ack_timeout, in milliseconds. | |
4 | long | The new value of retransmit_timeout, in milliseconds. |
The open channel request is used to open a TCP connection to a port on the local host. We limit connections to the local host so that we cannot be used as an open proxy. When a connection has been established, the open channel response is packet is send to report the channel number corresponding to the connection. If opening fails, an error response is given.
Byte Count | Data Type | Value | Description |
1 | byte | 0x04 | Packet ID |
1 | byte | request_id | |
2 | short | 1 - 65535 | port - the TCP port to connect to on the local host. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x05 | Packet ID |
1 | byte | request_id | |
1 | byte | 1 - 254 | channel - The id of the newly opened channel. |
The client can request the server listen on a port by sending a Start Listening request. If the request succeeds, the server sends a Start Listening response. Otherwise, an Error response is sent.
While the server is listening on a port, Incoming Connection interrupts are sent whenever a remote host connects to that port.
The client can request the server stop listening on a port by sending a Stop Listening request. The server will attempt to stop listening, and return either a Success response or an Error response.
Byte Count | Data Type | Value | Description |
1 | byte | 0x06 | Packet ID |
1 | byte | request_id | |
2 | short | port - the port to listen on. | |
If port is 0, then the server will randomly chose a port to listen on, and return that in the Listen on Port Success Response packet. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x07 | Packet ID |
1 | byte | request_id | |
2 | short | port - The port that we are listening on. | |
Port will be the same as in the request, unless the request was for port 0, in which case it will be the port actually chosen. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x08 | Packet ID |
2 | short | port - The listening port the incoming connection arrived on. | |
1 | byte | 1-254 | channel - The channel the incoming connection has been assigned. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x09 | Packet ID |
1 | byte | request_id | |
2 | short | port - The port to stop listening on. |
A channel can be closed by sending a Close Channel request. A success or error response will be sent in reply, as appropriate. In either case the channel should be considered closed by the client, and no further packets will be sent with that channel number.
Byte Count | Data Type | Value | Description |
1 | byte | 0x0a | Packet ID |
1 | byte | request_id | |
1 | byte | channel - The channel to close. |
A Channel Closed interrupt occurs when a channel can no longer be read from, because the other end has closed the connection.
A Channel Error interrupt occurs when an error occurs when reading to or writing to a channel fails. This is the case, for example, when writing to a channel that's closed.
Byte Count | Data Type | Value | Description |
1 | byte | 0x0b | Packet ID |
1 | byte | channel - The channel this error is associated with. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x0c | Packet ID |
1 | byte | channel - The channel this error is associated with. | |
2 | short | errno - An error code, taken from Linux's errno.h. | |
1 | byte | error_string_length - The length of a string giving a textual description of the error code. | |
error_string_length | byte data | error_string - A textual description of the error code. |
Many of the commands above requie a connection be authenticated before they can be issued. Authentication proceeds by the client sending a Challenge Request to the server. The server will then respond with a Challenge Response. The challenge response will contain a string between 0 and 64 bytes in length. The client should contatenate the shared secret and this string, and compute the MD5 hash of those strings. It should then send the digest (as 16 bytes, not as hex digits) of this hash to server. The server compares the digest sent by the client to the digest it has computed itself, using the same method. If they match, a Success Response is sent out and the connection is considered authenticated. If not, an Error Response is sent out.
Byte Count | Data Type | Value | Description |
1 | byte | 0x0d | Packet ID |
1 | byte | request_id |
Byte Count | Data Type | Value | Description |
1 | byte | 0x0e | Packet ID |
1 | byte | request_id | |
1 | byte | 0-64 | challenge_len - The number of challenge bytes. |
challenge_len (0-64) | byte | challenge - The challenge bytes. |
Byte Count | Data Type | Value | Description |
1 | byte | 0x0f | Packet ID |
1 | byte | request_id | |
16 | byte | MD5(secret + challenge) |
The client can teminate this protocol by dropping the underlying connection. When the connection drops, all resources are freed, and all TCP connections are closed.
The default values of the timeouts should vary based on the underlying transport mechanism. For unreliable terrestrial serial connections, the defaults should be:
For reliable connections (like TCP connections), there is no need for retransmission. We suggest setting the timers to the following large values:
These values may be changed by the client by sending a framing configuration request.