Transmit Multiple Google Protocol Buffers Messages Between Java and C++

I am working on a project that needs to transmit data between an Android application written in Java and a Windows application written in C++, and naturally I chose to use Google Protocol Buffers over UDP as the solution. I did a prototype, it worked fine until I wanted to put multiple messages into a single UDP packet.

There are several good reasons I want to put multiple messages into a single UDP packet, like performance and simplicity. The first message is fixed, program will read it first, then will know what are the messages after it based on the information from first message. For example, first message is always “fruit”, it has data like type, color, weight, etc. Based on type, program will know there is an apple or orange after it.

After several hours of Google search and testing, finally I made it working to transmit multiple messages between Java and C++ using a single buffer, using Android emulator.

Send Messages From Java
Google Protocol Buffers Java API builds a function writeDelimitedToin each message, it can be used to write message into raw buffer with the size of the message will be written first:

ByteArrayOutputStream output = new ByteArrayOutputStream(1024);

msg1.writeDelimitedTo(output);
msg2.writeDelimitedTo(output);
... // write more messages if they can fit into the output buffer

byte buffer[] = output.toByteArray();
DatagramPacket packet = new DatagramPacket(buffer, output.size());
... // send packet out

Receive Messages In C++
On the C++ side, the size of the message needs to be read out first, then read the message itself, repeat it for each message in the packet:

unsigned char buffer[1024];
sockaddr_in addr_other = {0};
int naddrsize = sizeof(addr_other);
int size = recvfrom(s, (char*)buffer, sizeof(buffer), 0, (sockaddr*)&addr_other, &naddrsize);

google::protobuf::io::ArrayInputStream arr(buffer, size);
google::protobuf::io::CodedInputStream input(&arr);
	
UINT32 message1_size = 0;
input.ReadVarint32(&message1_size);
google::protobuf::io::CodedInputStream::Limit limit = input.PushLimit(message1_size);
msg1.ParseFromCodedStream(&input);
input.PopLimit(limit);

UINT32 message2_size = 0;
input.ReadVarint32(&message2_size);
limit = input.PushLimit(message2_size);
msg2.ParseFromCodedStream(&input);
input.PopLimit(limit);

// repeat if there are more messages

Send Messages From C++
Sending messages from C++ requires writing the message size before actually writing the message itself:

unsigned char buffer[1024];
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
google::protobuf::io::CodedOutputStream output(&arr);
			
output.WriteVarint32(msg1.ByteSize());
msg1.SerializeToCodedStream(&output);
			
output.WriteVarint32(msg2.ByteSize());
msg2.SerializeToCodedStream(&output);

// repeat if more messages can fit into the buffer

int nSent = sendto(s, (char*)buffer, output.ByteCount(), 0, (sockaddr*)&addr_other, sizeof(addr_other));

Receive Messages in Java
Again it is simpler in Java:

byte[] buffer = new byte[1024];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
... // receive packet
ByteArrayInputStream input = new ByteArrayInputStream(buffer);
Msg1 msg1 = Msg1.parseDelimitedFrom(input);
Msg2 msg2 = Msg2.parseDelimitedFrom(input);
// repeat if there are other messages

You can test the Java application in Android emulator, but don’t forget to add the port redirection in the emulator.

Advertisements

Posted on April 29, 2013, in C++, Google Protocol Buffers, Java, Network. Bookmark the permalink. 1 Comment.

  1. thanks for this! We can show full listing .cpp?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: