Best practice for determining the message type of protocol buffers
I need to serialize and deserialize a sequence of protocol buffers messages to and from a byte stream. There are several predefined message types. What is the recommended way to encode the type information so that my application can know what type it should read?
The most common way to do this is by using a message.
For example:
message AnyMessage {
optional Message1 msg1 = 1;
optional Message2 msg2 = 2;
...
}
Then all messages are encoded / decoded inside the container AnyMessage
. As of protobuf 2.6, you can also use a specifier oneof
that will ensure that only one of the subordinate messages is set.
My suggestions, in any particular order:
- Include the field that contains the protos name / id (and give it the same field number in all 1 protocols)
-
Use the self-description of the message (at the bottom of the page). In this case, you can either
- include FileDescriptorSet as a field (in your posts)
- In java (and some other implementations) you can write the FileDescriptor as the first message in a delimited file.
-
Keep the proto filenames small and use the proto name in the filename, eg.
salesProto_Store001.bin
This has several good points:
- It is immediately obvious that the file format
- You can scan shell scripts to find where the protocol is being used.
- This method can be used on its own or in combination with the above.
- This method can be used for any schema based files (like Cobol).
Finally ProtobufEditor
-
Supports messages about yourself where
- The file descriptor is the first message in the delimited file
- First field in message
-
Has a lookup function that will try to match fields in a Protobuf post with known Proto definition files and give you a possible match
Background: In case you didn't know, a proton protocol buffer file can be converted to a protocol buffer message FileDescriptorSet and saved
Self-description of messagew:
message SelfDescribingMessage {
// Set of .proto files which define the type.
required FileDescriptorSet proto_files = 1;
// Name of the message type. Must be defined by one of the files in
// proto_files.
required string type_name = 2;
// The message data.
required bytes message_data = 3;
}