Motivated by a recent post in our forum regarding data logging, I started writing a series of posts on data logging options for the NXT. I also began working on a Datalog NXT-G Block dedicated to this purpose. During the block development, however, I encountered some serious performance issues regarding Array usage in the LabView NXTToolkit compiler (and hence in NXT-G) which I’ll describe in this post. Future posts will describe existing and new solutions for data logging.

Contrary to Brian Davis datalog program (found in NXTLOG), I wrote my block to use a RAM array storage, writing it to Flash at the end of the execution. This approach is successfully implemented in-firmware for RobotC. When I ran my dedicated ‘Datalog Block’ I found it stores a record much slower then Brian Davis’s code (but without the occasional long delay ~70 millisecond intervals in Brian’s code when the Flash buffers are emptied). This made me really worried – RAM access should have been MUCH faster. I therefore wrote a simple test program in LabView shown below (click for full image):

For those unfamiliar with LabView, it is similar to Robolab/NXT-G in spirit but differs in icons… I initialize an empty array (knowing that dynamic allocation inside the loop, which is what happens when you concatenate arrays, is very memory ineffective), record the milliseconds timer, loop over the array replacing each element with the numerical index, and record the time at the end. The last few VIs (i.e. ‘blocks’) convert the time difference to a string, displays it and wait for a button press. For a 750 elements array the display showed that 780 milliseconds have passed. However, this actually took about 25 seconds on my watch!
I’ve asked the guys over in National Instruments (the developers of the firmware/LV compiler) and got an answer that there are indeed ‘serious performance issues’ with array opcodes (operations) in the released firmware. Namely, the firmware uses a complicated (polymorphic) method to write to memory even for simple numerical array data types. Furthermore, they admit that there is a bug in the timer, which cause it to ‘loss’ milliseconds when a single opcode takes more then 1 millisecond to be executes.
Here I got some help from Dick Swan (thanks!), who tested a similar code in NXC:

#include “NXCDefs.h”
long data[];
long idx;
long startTime;
long endTime;task main()
{
  ArrayInit(data, 0, 750);
  startTime = CurrentTick();
  for (idx 0; idx < 750; ++idx)
  {
    data[idx] = idx;
  }
  endTime = CurrentTick();
  NumOut(0, 8, endTime - startTime); 
Wait(5000);
}

Now this code both displayed and (apparently) ran in 279 milliseconds. A similar RobotC code took 12 milliseconds only. Since the NXC code compiles via NBC into a code executed by the same official LEGO firmware my code used, I was again puzzled at my slow program result.
Another email from NI solved the mystery – apparently the (version 1.0) compiler we have today is not very efficient in dataspace usage. In particular, most wire creates an extra copy of their input (except those that run through loop/case/sequence tunnels). In particular the shift register (the little up and down arrows in the figure above) which passes the array from one iteration to the other duplicate the array at each loop iteration. This is very counter intuitive, especially to embedded LabView programmers (NI recommends using shift registers for saving memory in embedded LV applications, see e.g. ftp://ftp.ni.com/pub/devzone/pdf/tut_3747.pdf).
Fortunately, the puzzle solution also gave me a hint for a solution. Instead of using shift register (i.e. the usual way in LV) I’ll use an array variable, read it at the beginning of the loop iteration, replace and element and then write it back. Here’s the modified code:

This code both measured and (apparently) executed in 167 milliseconds !

So – what is the “take home message”? When using arrays in LV/NXT-G block code DO NOT pass it in shift registers, but rather use variable read/write ‘local variable’ VIs to manipulate it within loops etc.

Guy Ziv
NXTasy.org