endianness and data serialization

A bit on endianness

Endianness refers to the ordering of bytes in memory that make up a larger chunk of memory.  There's plenty of more in depth reading out there, but here's a really simple summary.  Most computer systems are either big-endian or little-endian.  Here's a table copied from wikipedia's page on endianness (http://en.wikipedia.org/wiki/Endianness) that explains the difference:

When we write out a number in hex it looks like big-endian.  For example the number 4097 would be written as 0x1001 where the 0x10 is the MSB (most significant byte) and 0x01 is the LSB (least significant byte.  

The systems we are working on are all little endian though, so the number 4097 would be stored as 0x01:0x10 in memory if we saved it as a short.  Saving it as an int would add some 0s for padding in there lie so: 0x01:0x10:0x00:0x00.  

Serializing our data

Now say we want to send a uint16 (unsigned short) through acomms along with a bunch of other data.  We'll probably be constructing a long array of bytes and copying our data into it for transmission, then picking it back out at the other end.  Sticking with our simple uint16, let's look at how we can reconstruct the number given an array of bytes.  

The following code snippet illustrates two ways of converting an array of two bytes to an unsigned short.  

unsigned char data[] = {0x01, 0x10};

// method 1 - bitwise operations
unsigned short result1 = (data[0]<<8) + data[1];
cout << result1 << endl;

// method 2 - memcpy
unsigned short result2;
memcpy(&result2, &data[0],2);
cout << result2 << endl;

These two methods will yield different results.  The first produces 0x0110 = 272 as we would expect.  The second produces 0x1001 = 4097 because we're working on a little-endian system.  On a big endian system the second method would give us the same result as the first.  

Neither one of these solutions is very elegant though, especially if we start to deal with longer arrays or larger numbers.  One solution is to use structs to send your data.  A struct can be defined that holds all the data you want to send and then memcpy can copy the whole thing to or from an array in one go.  

#include <iostream>
#include <string.h>
#include <vector>
#include <math.h>

using namespace std;

// our structure for sending data
struct MY_DATA {
	unsigned char packet_id;
	int latitude;
	int longitude;
	unsigned short heading;
};

int main() {
	// fill a struct with data
	MY_DATA data_to_send;
	data_to_send.packet_id = 0x01;
	data_to_send.latitude = (int) (42.358456 * pow(10,7));
	data_to_send.longitude = (int) (-71.087589 * pow(10,7));
	data_to_send.heading = 185;

	// construct our array/vector and fill it with data
	vector<unsigned char> packet (sizeof(data_to_send), 0);
	memcpy(&packet[0], &data_to_send.packet_id, sizeof(data_to_send));

	// to get our data out, do the reverse
	MY_DATA data_received;
	memcpy(&data_received.packet_id, &packet[0], sizeof(data_to_send));

	// print out the data
	cout << "packet id: 0x" << hex << (int) data_received.packet_id << dec << endl;
	cout << "lat: " << data_received.latitude / pow(10,7) << endl;
	cout << "lon: " << data_received.longitude / pow(10,7) << endl;
	cout << "heading: " << data_received.heading << endl;

	return 0;
}

Posting to MOOS

See Binary acomms data in MOOS for information on getting your serialized data into or out of a MOOS message.