Goby v2
|
Table of contents for libdccl:
Return to goby-acomms: An overview of Acoustic Communications Library.
DCCL uses the Google Protocol Buffers (Protobuf) language to define messages. DCCL specific components are defined as extensions to the Protobuf language message and field options. You should familiarize yourself with basic Protobuf using before reading the rest of this document: see Google Protocol Buffers and http://code.google.com/apis/protocolbuffers/docs/overview.html.
Scenario 1: Send a string command to a vehicle:
We need to send an ASCII string command to an underwater vehicle. We thus make a Protobuf message with a single string field (let's call it "telegram") to hold our command:
message Simple { required string telegram = 1; }
The "= 1" indicates that this is the first field on the wire in our DCCL message. All fields must have a unique index, but otherwise these index values are not particularly important. "required" means a valid "Simple" message always contains something for "telegram" (could be an empty string).
To turn this Protobuf message into a DCCL message, we need to add a few options. All the options are defined in acomms_option_extensions.proto so we include that:
import "goby/common/protobuf/option_extensions.proto"; message Simple { required string telegram = 1; }
At a minimum we must give a unique ID for our DCCL message and a maximum number of bytes we allow the message to be before throwing an exception when it is loaded. This allows us to ensure that we are not creating messages larger than we can send with the physical hardware. We want to have the ability to use the lowest rate WHOI Micro-Modem message size, so we pick max_bytes to be 32. We are testing so we'll use an id of 124. See http://gobysoft.org/wiki/DcclIdTable for a list of the assigned DCCL IDs.
After these additions we have:
import "goby/common/protobuf/option_extensions.proto"; message Simple { option (dccl.msg).id = 124; option (dccl.msg).max_bytes = 32; required string telegram = 1; }
Finally, we need to pick an encoder/decoder (codec) for each field in our message. DCCL comes with defaults for all the Protobuf types. So if we don't specifically list a codec for a given field, the default is used. The default "string" codec is goby::acomms::DCCLDefaultStringCodec and is variable length. It uses one byte to list the length of the string and then up to 255 bytes to hold the contents. To ensure we stay within our bounds for the entire message ((goby.msg).dccl.max_bytes = 32
), we have to give a maximum allowed length for a string when using the DCCLDefaultStringCodec ((goby.field).dccl.max_length
).
import "dccl/option_extensions.proto"; message Simple { // see http://gobysoft.org/wiki/DcclIdTable option (dccl.msg).id = 124; // if, for example, we want to use on the WHOI Micro-Modem rate 0 option (dccl.msg).max_bytes = 32; required string telegram = 1 [(dccl.field).max_length = 30]; }
See dccl_simple.cpp for an example of how to use this message.
Scenario 2: Send a more realistic command and receive a status message from the vehicle:
We want to be able to command our vehicle (to which we have assigned an ID number of "2") to go to a specific point on a local XY grid (meters from some known latitude / longitude), but no more than 10 kilometers from the datum. We also want to be able to turn the lights on or off, and send a short string for other new instructions. Finally, we need to be able to command a speed. Our vehicle can move no faster than 3 m/s, but its control is precise enough to handle hundredths of a m/s (wow!). It's probably easiest to make a table with our conditions:
message variable name | description | type | bounds |
destination | id number of the vehicle we are commanding | int32 | [0, 31] |
goto_x | meters east to transit from datum | int32 | [0, 10000] |
goto_y | meters north to transit from datum | int32 | [0, 10000] |
lights_on | turn on the lights? | bool | |
new_instructions | string instructions | string | no longer than 10 characters |
goto_speed | transit speed (m/s) | float | [0.00, 3.00] |
Taking all this into account, we form the first message (named GoToCommand) in the file two_message.proto (see Two Message example)
We choose a dccl.id
of 125 to avoid conflicting with the message from Scenario 1 (simple.proto) and a dccl.max_bytes
of 32 bytes to again allow sending in the WHOI Micro-Modem rate 0 packet.
Now, for the second message in two_message.proto. We want to receive the vehicle's present position and its current health, which can either be "good", "low_battery" or "abort". We make a similar table to before:
message variable name | description | type | bounds |
nav_x | current vehicle position (meters east of the datum) | integer | [0, 10000] |
nav_y | current vehicle position (meters north of the datum) | integer | [0, 10000] |
health | vehicle state | enumeration | HEALTH_GOOD, HEALTH_LOW_BATTERY, or HEALTH_ABORT |
The resulting message, can be seen under Two Message example. An example of how to use this message is given under two_message.cpp.
You can run analyze_dccl
to view more information on your messages:
When I ran the above command I got:
read in: two_message.proto === Begin DCCLCodec === 2 messages loaded. = Begin GoToCommand = Actual maximum size of message: 18 bytes / 144 bits [dccl.id head: 8, user head: 0, body: 131, padding: 5] Allowed maximum size of message: 32 bytes / 256 bits == Begin Header == == End Header == == Begin Body == GoToCommand required int32 destination = 1; :: size = 5 bit(s) required int32 goto_x = 3; :: size = 14 bit(s) required int32 goto_y = 4; :: size = 14 bit(s) required bool lights_on = 5; :: size = 1 bit(s) required string new_instructions = 6; :: min size = 8 bit(s) :: max size = 88 bit(s) required double goto_speed = 7; :: size = 9 bit(s) :: min size = 51 bit(s) :: max size = 131 bit(s) == End Body == = End GoToCommand = = Begin VehicleStatus = Actual maximum size of message: 6 bytes / 48 bits [dccl.id head: 8, user head: 0, body: 36, padding: 4] Allowed maximum size of message: 32 bytes / 256 bits == Begin Header == == End Header == == Begin Body == VehicleStatus required double nav_x = 1; :: size = 17 bit(s) required double nav_y = 2; :: size = 17 bit(s) required .VehicleStatus.HealthEnum health = 3; :: size = 2 bit(s) :: size = 36 bit(s) == End Body == = End VehicleStatus = === End DCCLCodec ===
Besides validity checking, the most useful feature of analyze_dccl
is the calculation of the size (in bits) of each message variable. This lets you see which fields in the message are too big. To make fields smaller, tighten up bounds.
This section gives an overview of the DCCL message and field options available for use with DCCL and the default field codecs. The full list is available in option_extensions.proto (as messages DCCLFieldOptions and DCCLMessageOptions).
DCCL message options:
name | type | default | description |
id | uint32 | required | A unique ID for each DCCL message |
max_bytes | uint32 | required | Maximum allowed size in bytes for the encoded message |
codec | string | "_default" | Name of the codec to use for encoding the base message (add more codecs with goby::acomms::DCCLFieldCodecManager::add()) |
DCCL field options:
name | type | default | required for codecs | description |
codec | string | "_default" | optional | Name of the codec to use for encoding this field |
omit | bool | false | optional | Omit this field from all DCCL encoding (has_field() will be false on receipt) |
in_head | bool | false | optional | Set true for fields in the header (will not be encrypted, rather will be used to create the encrytion IV). |
precision | int32 | 0 | goby::acomms::DCCLDefaultNumericFieldCodec (double, float) | Number of decimal digits of precision to keep (can be negative). |
min | double | 0 | goby::acomms::DCCLDefaultNumericFieldCodec (double, float, int32, uint32, int64, uint64, fixed32, fixed64, sfixed32, sfixed64) | Minimum value that can be encoded in this field. |
max | double | 0 | goby::acomms::DCCLDefaultNumericFieldCodec (double, float, int32, uint32, int64, uint64, fixed32, fixed64, sfixed32, sfixed64) | Maximum value that can be encoded in this field. |
static_value | string | "" | goby::acomms::DCCLStaticCodec (any type) | The static value for use on decoding this placeholder field. |
max_length | uint32 | 0 | goby::acomms::DCCLDefaultStringCodec, goby::acomms::DCCLDefaultBytesCodec (string) | The maximum length of the string that can be stored in this field. |
max_repeat | uint32 | 0 | any repeated field | The maximum length of the repeated array (or vector). |
Using the goby::acomms::DCCLCodec is a fairly straightforward endeavor (this example uses dccl_simple.cpp). First you need to get a pointer to the DCCLCodec singleton:
Validate all messages with the DCCLCodec to ensure all bounding constraints are met:
Then, to encode a message, create a Protobuf message, set its fields and pass it to goby::acomms::DCCLCodec::encode():
bytes
will now contain the encoded message in the form of a byte string (each char will contain a single byte of the message).
You may now send this message through whatever channel you would like.
To decode a message (stored in bytes
as a byte string), simply pass bytes as a reference along with pointers to the Protobuf message to store the results.
For line by line interaction with the goby::acomms::DCCLCodec and for advanced use, investigate the code examples given in the Examples column of this table.
Encryption of all messages can be enabled by providing a secret passphrase to the goby::acomms::protobuf::DCCLConfig object passed to goby::acomms::DCCLCodec::set_cfg(). All parties to the communication must have the same secret key.
DCCL provides AES (Rijndael) encryption for the body of the message. The header, which is sent in plain text, is hashed to form an initialization vector (IV), and the passphrase is hashed using SHA-256 to form the cipher key. You will want to make sure the header (designate fields for the header with (goby.field).dccl.in_head = true
) is a nonce by including a constantly changing value such as time.
AES is considered secure and is used for United States top secret information.
This section provides a listing of DCCL example Protobuf messages used in the code examples and unit tests.
simple.proto
import "dccl/option_extensions.proto"; message Simple { // see http://gobysoft.org/wiki/DcclIdTable option (dccl.msg).id = 124; // if, for example, we want to use on the WHOI Micro-Modem rate 0 option (dccl.msg).max_bytes = 32; required string telegram = 1 [(dccl.field).max_length = 30]; }
two_message.proto
import "dccl/option_extensions.proto"; message GoToCommand { option (dccl.msg).id = 125; option (dccl.msg).max_bytes = 32; required int32 destination = 1 [ (dccl.field).max = 31, (dccl.field).min = 0, (dccl.field).precision = 0 ]; optional string type = 2 [(dccl.field).static_value = "goto", (dccl.field).codec = "_static"]; required int32 goto_x = 3 [ (dccl.field).max = 10000, (dccl.field).min = 0, (dccl.field).precision = 0 ]; required int32 goto_y = 4 [ (dccl.field).max = 10000, (dccl.field).min = 0, (dccl.field).precision = 0 ]; required bool lights_on = 5; required string new_instructions = 6 [(dccl.field).max_length = 10]; required double goto_speed = 7 [ (dccl.field).max = 3, (dccl.field).min = 0, (dccl.field).precision = 2 ]; } message VehicleStatus { option (dccl.msg).id = 126; option (dccl.msg).max_bytes = 32; required double nav_x = 1 [ (dccl.field).max = 10000, (dccl.field).min = 0, (dccl.field).precision = 1 ]; required double nav_y = 2 [ (dccl.field).max = 10000, (dccl.field).min = 0, (dccl.field).precision = 1 ]; required HealthEnum health = 3; enum HealthEnum { HEALTH_GOOD = 0; HEALTH_LOW_BATTERY = 1; HEALTH_ABORT = 2; } }
Test1 showing all Protobuf types (using default codecs):
import "dccl/option_extensions.proto"; enum Enum1 { ENUM_A = 1; ENUM_B = 2; ENUM_C = 3; } message EmbeddedMsg1 { optional double val = 1 [ (dccl.field).min = 0, (dccl.field).max = 126, (dccl.field).precision = 3 ]; optional EmbeddedMsg2 msg = 2; } message EmbeddedMsg2 { optional double val = 1 [ (dccl.field).min = 0, (dccl.field).max = 126, (dccl.field).precision = 2 ]; optional string sval = 2 [(dccl.field).max_length = 10]; optional Enum1 enum_default = 3; } message TestMsg { option (dccl.msg).id = 2; option (dccl.msg).max_bytes = 512; // test default enc/dec optional double double_default_optional = 1 [ (dccl.field).min = -100, (dccl.field).max = 126, (dccl.field).precision = 2, (dccl.field).in_head = true ]; optional float float_default_optional = 2 [ (dccl.field).min = -20, (dccl.field).max = 150, (dccl.field).precision = 3 ]; optional int32 int32_default_optional = 3 [(dccl.field).min = -20, (dccl.field).max = 3000]; optional int64 int64_default_optional = 4 [(dccl.field).min = -710, (dccl.field).max = 3000]; optional uint32 uint32_default_optional = 5 [(dccl.field).min = 0, (dccl.field).max = 3000]; optional uint64 uint64_default_optional = 6 [(dccl.field).min = 5, (dccl.field).max = 3000]; optional sint32 sint32_default_optional = 7 [(dccl.field).min = -60, (dccl.field).max = 3000]; optional sint64 sint64_default_optional = 8 [(dccl.field).min = -70, (dccl.field).max = 3000]; optional fixed32 fixed32_default_optional = 9 [(dccl.field).min = 0, (dccl.field).max = 400]; optional fixed64 fixed64_default_optional = 10 [(dccl.field).min = 0, (dccl.field).max = 3000]; optional sfixed32 sfixed32_default_optional = 11 [(dccl.field).min = 11, (dccl.field).max = 3000]; optional sfixed64 sfixed64_default_optional = 12 [(dccl.field).min = -12, (dccl.field).max = 3000]; optional bool bool_default_optional = 13; optional string string_default_optional = 14 [(dccl.field).max_length = 8]; optional bytes bytes_default_optional = 15 [(dccl.field).max_length = 9]; optional Enum1 enum_default_optional = 16; optional EmbeddedMsg1 msg_default_optional = 17; required double double_default_required = 21 [ (dccl.field).min = -100, (dccl.field).max = 126, (dccl.field).precision = 2, (dccl.field).in_head = true ]; required float float_default_required = 22 [ (dccl.field).min = -20, (dccl.field).max = 150, (dccl.field).precision = 3 ]; required int32 int32_default_required = 23 [(dccl.field).min = -20, (dccl.field).max = 3000]; required int64 int64_default_required = 24 [(dccl.field).min = -710, (dccl.field).max = 3000]; required uint32 uint32_default_required = 25 [(dccl.field).min = 0, (dccl.field).max = 3000]; required uint64 uint64_default_required = 26 [(dccl.field).min = 5, (dccl.field).max = 3000]; required sint32 sint32_default_required = 27 [(dccl.field).min = -60, (dccl.field).max = 3000]; required sint64 sint64_default_required = 28 [(dccl.field).min = -70, (dccl.field).max = 3000]; required fixed32 fixed32_default_required = 29 [(dccl.field).min = 0, (dccl.field).max = 400]; required fixed64 fixed64_default_required = 30 [(dccl.field).min = 0, (dccl.field).max = 3000]; required sfixed32 sfixed32_default_required = 31 [(dccl.field).min = 11, (dccl.field).max = 3000]; required sfixed64 sfixed64_default_required = 32 [(dccl.field).min = -120, (dccl.field).max = 3000]; required bool bool_default_required = 33; required string string_default_required = 34 [(dccl.field).max_length = 8]; required bytes bytes_default_required = 35 [(dccl.field).max_length = 9]; required Enum1 enum_default_required = 36; required EmbeddedMsg1 msg_default_required = 37; repeated double double_default_repeat = 101 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).precision = 3, (dccl.field).max_repeat = 4 ]; repeated float float_default_repeat = 102 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).precision = 3, (dccl.field).max_repeat = 4 ]; repeated int32 int32_default_repeat = 103 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated int64 int64_default_repeat = 104 [ (dccl.field).min = -100, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated uint32 uint32_default_repeat = 105 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).max_repeat = 4, (dccl.field).in_head = true ]; repeated uint64 uint64_default_repeat = 106 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated sint32 sint32_default_repeat = 107 [ (dccl.field).min = -60, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated sint64 sint64_default_repeat = 108 [ (dccl.field).min = -600, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated fixed32 fixed32_default_repeat = 109 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated fixed64 fixed64_default_repeat = 110 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated sfixed32 sfixed32_default_repeat = 111 [ (dccl.field).min = 0, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated sfixed64 sfixed64_default_repeat = 112 [ (dccl.field).min = -500, (dccl.field).max = 100, (dccl.field).max_repeat = 4 ]; repeated bool bool_default_repeat = 113 [(dccl.field).max_repeat = 4]; repeated string string_default_repeat = 114 [(dccl.field).max_length = 4, (dccl.field).max_repeat = 4]; repeated bytes bytes_default_repeat = 115 [(dccl.field).max_length = 4, (dccl.field).max_repeat = 4]; repeated Enum1 enum_default_repeat = 116 [(dccl.field).max_repeat = 4]; repeated EmbeddedMsg1 msg_default_repeat = 117 [(dccl.field).max_repeat = 4]; }
import "dccl/option_extensions.proto"; message CustomMsg { option (dccl.msg).id = 3; option (dccl.msg).max_bytes = 256; option (dccl.msg).codec = "custom_codec"; optional uint32 a = 1; optional bool b = 2; } message CustomMsg2 { option (dccl.msg).id = 4; option (dccl.msg).max_bytes = 256; optional CustomMsg msg = 1; repeated int32 c = 3 [ (dccl.field).max = 100, (dccl.field).min = 0, (dccl.field).max_repeat = 4, (dccl.field).codec = "int32_test_codec" ]; }
import "goby/common/protobuf/option_extensions.proto"; import "dccl/option_extensions.proto"; import "goby/test/acomms/dccl3/header.proto"; message GobyMessage { option (dccl.msg).id = 4; option (dccl.msg).max_bytes = 32; required string telegram = 1 [(dccl.field).max_length = 10]; required Header header = 2; }
import "goby/common/protobuf/option_extensions.proto"; import "dccl/option_extensions.proto"; // required fields will be filled in for you by ApplicationBase // if you choose not to do so yourself message Header { // // time // // result of goby::util::as<std::string>(goby_time()) // e.g. "2002-01-20 23:59:59.000" required string time = 10 [(dccl.field).codec = "_time", (dccl.field).in_head = true]; // // source // required string source_platform = 11 [ (dccl.field).codec = "_platform<->modem_id", (dccl.field).in_head = true ]; optional string source_app = 12 [(dccl.field).omit = true]; // // destination // enum PublishDestination { PUBLISH_SELF = 1; PUBLISH_OTHER = 2; PUBLISH_ALL = 3; } optional PublishDestination dest_type = 13 [default = PUBLISH_SELF, (dccl.field).in_head = true]; optional string dest_platform = 14 [ (dccl.field).codec = "_platform<->modem_id", (dccl.field).in_head = true ]; // required if dest_type == other }
import "dccl/option_extensions.proto"; import "goby/test/acomms/dccl3/header.proto"; message GobyMessage1 { option (dccl.msg).id = 4; option (dccl.msg).max_bytes = 32; optional int32 int32_val = 1 [(dccl.field).min = 0, (dccl.field).max = 20]; } message GobyMessage2 { option (dccl.msg).id = 5; option (dccl.msg).max_bytes = 32; optional bool bool_val = 1; } message GobyMessage3 { option (dccl.msg).id = 6; option (dccl.msg).max_bytes = 32; optional string string_val = 1 [(dccl.field).max_length = 10]; }
dccl5/test.proto
import "dccl/option_extensions.proto"; message ShortIDMsg { option (dccl.msg).id = 2; option (dccl.msg).max_bytes = 1; } message ShortIDMsgWithData { option (dccl.msg).id = 3; option (dccl.msg).max_bytes = 10; optional int32 in_head = 1 [ (dccl.field).in_head = true, (dccl.field).min = 0, (dccl.field).max = 100 ]; optional int32 in_body = 2 [ (dccl.field).in_head = true, (dccl.field).min = 0, (dccl.field).max = 100 ]; } message LongIDMsg { option (dccl.msg).id = 10000; option (dccl.msg).max_bytes = 2; } message TooLongIDMsg { option (dccl.msg).id = 32768; option (dccl.msg).max_bytes = 32; } message LongIDEdgeMsg { option (dccl.msg).id = 128; option (dccl.msg).max_bytes = 2; } message ShortIDEdgeMsg { option (dccl.msg).id = 127; option (dccl.msg).max_bytes = 1; }
import "dccl/option_extensions.proto"; message BytesMsg { option (dccl.msg).id = 10; option (dccl.msg).max_bytes = 32; required bytes req_bytes = 1 [(dccl.field).max_length = 8]; optional bytes opt_bytes = 2 [(dccl.field).max_length = 8]; }
import "dccl/option_extensions.proto"; import "goby/test/acomms/dccl3/header.proto"; message GobyMessage1 { option (dccl.msg).id = 4; option (dccl.msg).max_bytes = 32; optional int32 int32_val = 1 [(dccl.field).min = 0, (dccl.field).max = 20]; } message GobyMessage2 { option (dccl.msg).id = 5; option (dccl.msg).max_bytes = 32; optional bool bool_val = 1; } message GobyMessage3 { option (dccl.msg).id = 6; option (dccl.msg).max_bytes = 32; optional string string_val = 1 [(dccl.field).max_length = 10]; }
import "dccl/option_extensions.proto"; message MiniUser { option (dccl.msg).id = 1000001; option (dccl.msg).max_bytes = 2; required uint32 user = 1 [ (dccl.field).min = 0, (dccl.field).max = 0x03FF, (dccl.field).in_head = true ]; } message MiniOWTT { option (dccl.msg).id = 1000002; option (dccl.msg).max_bytes = 2; required uint32 clock_mode = 1 [ (dccl.field).min = 0, (dccl.field).max = 3, (dccl.field).in_head = true ]; required uint32 tod = 2 [ (dccl.field).min = 0, (dccl.field).max = 0x0F, (dccl.field).in_head = true ]; required uint32 user = 3 [ (dccl.field).min = 0, (dccl.field).max = 0x0F, (dccl.field).in_head = true ]; } message MiniAbort { option (dccl.msg).id = 1000003; option (dccl.msg).max_bytes = 2; required uint32 user = 1 [ (dccl.field).min = 0, (dccl.field).max = 0x03FF, (dccl.field).in_head = true ]; } message NormalDCCL { option (dccl.msg).id = 1; option (dccl.msg).max_bytes = 32; required int32 a = 1 [(dccl.field).min = 0, (dccl.field).max = 0xFFFF]; required int32 b = 2 [(dccl.field).min = 0, (dccl.field).max = 0xFFFF]; }