This page covers the basics of downloading and parsing logs for your own use, including building AlogParse under windows.
The files
On the vehicles
Log files on the vehicles are saved the "logs" folder inside the home directory, i.e. /home/josh/logs. pLogger will automatically create a new time stamped folder whenever it is started. Other MOOS applications that make their own log files should save their logs in the same folder by subscribing to the PLOGGER_DIRECTORY (I think) variable.
To download log files from the vehicle run the script multi-logfetch on terra, with the first letter of each vehicle name to get logs from as the argument. For example "multi-logfetch tni" will fetch logs from terra, nostromo, and icarus. The logs are compressed and moved to a date stamped folder on terra's data drive. From there they must be manually uploaded to AWS.
Downloading log files
Logs are kept in Amazon Web Services's S3 here: http://aws.amazon.com/console/. There is also a google doc with a quick listing of all kayak experimental operations. Email Josh to get access to both AWS and the google doc. Once you sign sign into AWS click on "S3" under the red section titled "Storage & Content Delivery".
Logs are compressed as .tar.gz or .tar.bz2 files. To open them under windows you can use WinRAR available for free at http://www.rarlab.com/download.htm. Download the "Graphic and Command line" version at the very top corresponding to your windows version (32 or 64 bit).
If you have trouble extracting the log files check the file extension. On some computers the files are downloaded with the wrong file extension, such as .gz instead of .tar.gz. Rename the extension to what it says in AWS and WinRAR should open it just fine.
Files you will find
Each time the software on the kayaks is restarted a new folder is created corresponding to the start time. You can expect to find the following extension types in each folder:
.blog - ignore this
.ylog - ignore this too
._moos - a copy of the MOOS configuration file the vehicle was started with.
.alog - the main asynchronous log file.
goby_logX.txt - a log of the communications between iAcommsDriver and the MicroModem
An asynchronous logfile means that when a variable is posted to in the MOOSDB, that posting is logged as a single line with at least its time stamp, source application name, variable name, and value. To import data into matlab we first need to convert our alog to a synchronous logfile.
Parsing
There are two main parsing programs, AlogParse and AcommsParse. Together they should fulfill most of your parsing needs. If not, consider rethinking how you log data.
AlogParse
AlogParse is a simple command line application that will extract the variables you request from an alog and create a delimited, synchronous text file that can be imported into Matlab or Excel.
Installation and Quick Usage
If using Linux, AlogParse will be built with the Hovergroup util applications. If using windows, you can compile AlogParse independently by downloading its source files from here: AlogParse SVN. It has no external dependencies so simply download the header and source file and use nmake to build.
To use alogParse follow these instructions:
1) Open up a command prompt go to Start -> Run, type in "cmd" and hit enter. A command prompt should open up in your user directory (e.g. C:\Users\Josh>)
2) Navigate to the directory where you saved alogParse
cd Documents\alogParse
3) Run alogParse
alogParse.exe --help
--help should tell you everything you need to know to use alogParse, though there are some more detailed notes below. It's easiest to pass an alog file to alogParse if you first copy the logfile to the same directory. Then you can run alogParse using a command such as:
alogParse.exe mylog.alog --sync_period=0.2 NAV_X NAV_Y
When dealing with long alog file names, remember that tab auto-completes. Once you start typing a name press tab and the command prompt will automatically try to fill in the rest. AlogParse will output a txt file with the same name as the input. This can be imported into Excel or Matlab using their import data functions.
More Detailed Usage
There are a couple of key concepts for using alogParse. Firstly, how is an asynchronous log file turned into a synchronous one? This is done either by specifying a "sync_period" or a "sync_variable". A sync_period specifies an interval in seconds at which alogParse will output a new line of data containing a posting of every variable. Setting a sync_variable means that alogParse posts a new line whenever the specified variable is posted to. Note that this variable does not necessarily need to be one of the variables you choose to output.
The syncing concept ties closely to how alogParse creates a line. By default it will look for the closest posting of each variable requested to fill the line - this posting could be in the future. Sometimes this is good, such as when we want to find the nearest nav fix at a given time. However it could be misleading if you were, for example, trying to find out what mission mode the vehicle was in when you sent an acomms transmission. alogParse might fill in a mission_mode posting one minute in the future, even though the actual mission_mode at the time was the one posted five minutes earlier.
For this reason there is the "--backsearch" option, which forces alogParse to look at only variable posting in the past when filling a line. You can even specify "--backsearch=0.5", which will allow alogParse to look up to 0.5 seconds in the future for closer variable postings. Backsearch, especially pure backsearch, can be much faster than the default mode. Regardless of the mode, alogParse always outputs and "age" along with every variable to tell you how far away that variable value was picked from. A negative age would indicate that variable came from the future.
The last capability of note is that of specifying the delimiter. The default delimiter is a comma, but if one of your variables is expected to contain commas then you can specify a different delimiter with the "--delimiter" flag.
AcommsParse
AcommsParse, as the name implies, is specifically for parsing acomms transmission and reception messages. It provides one line for each reception or transmission, splitting all the available information into separate columns.
Usage
AcommsParse has the same usage as AlogParse, except instead of defining a sync_variable or sync_period, you tell it to look for acomms receptions or acomms transmissions using the --receive and --transmit flags. In addition to the acomms information, any additional variables requested will be synced to the acomms receptions/transmissions. The backsearch option can be used in the same way as it is for alogParse.