NI Week is National Instruments annual sales conference held in Austin, TX. There was a very strong LEGO and Mindstorms flavor at this year’s conference with several significant Mindstorms announcements. There was great attendance by many influential people in the Mindstorms community.

The contents of this post include not only the “formal” NI Week activities but also describe advanced Mindstorms “sensor” technologies that I was quietly demonstrating at NI week — including  interfaces to a wireless SONY Playstation Controller and (an almost complete) video camera connected to the NXT high speed sensor port.

People Networking
This conference was a unique opportunity to meet almost all the movers and shakers behind the Mindstorms development.

• Soren Lund and several senior Lego executives from Denmark were in attendance.
• Everybody from NI that was associated with the NXT was there.
• Several members of the MDP came from out of town.
• There were several attendees from Tufts and Carnegie-Mellon. Both of these universities have been developing terrific educational materials for the NXT. They’ve spent all summer running pilot trials on a variety of new NXT training and education classes. They have classes for both students and educators.

Here’s a picture of MDPers Brian Davis, Steve Hassenplug and Chris Smith at lunch with most members of the National Instruments NXT development team. It was a bit sad, but not unexpected, to learn that many of the NI team have finished their work on NXT and are transitioning to other projects.

 

A personal highlight was unexpectedly meeting and having a long chat with Fred Martin. I’ve read about Fred’s early work at MIT but never dreamed that I’d ever meet him. Fred designed the Handyboard — the first autonomous robotics controller for LEGO bricks and motors.  It was the Handyboard’s success over 15 years ago that eventually led to the Mindstorms products. Fred presented a paper on his next generation Handyboard which will be available later this fall. It’s a very powerful system based on the AMD Blackfin CPU. Naturally it can be programmed in “embedded LabVIEW” as well as a variety of conventional text based programming languages.

Buy a NXT at the Apple Store
Did you know that Apple is selling the NXT in their stores? Here’s Steve Hassenplug talking to a rep from the local Apple store (the picture resolution is too low to see the white blob on her T-shirt is the Apple logo]. You could buy a NXT at the Apple booth in the exhibit hall. Unfortunately there was no show discount!

 

Steve has so many NXTs I don’t think he needs another one. Steve’s holding his holonomic robot in his right hand; it has three equally spaced wheels which allows the robot to instantly travel in any direction without mechanical steering.

LabVIEW Toolkit for the NXT
Brian Davis partnered with John Field of NI on a conference presentation on the NXT. John’s talk explained the details on the newly announced LabVIEW toolkit for NXT.

Here’s Brian getting ready for his presentation. The second picture shows the packed audience.

   

I’ve had a chance to review an early version of the tool kit. It’s really powerful.  There are two main components.

1. The first component is for the experienced LabVIEW programmer. This toolkit allows you to use the standard LabVIEW environment rather that the MINDSTORMS NXT software to program the NXT brick. This gives you a lot more control in writing your programs. You’ll be able to write NXT programs at the individual opcode / instruction level; each opcode will represent a single “subVI” (i.e ICON) on your program block diagram. While this environment is much more powerful, the LabVIEW environment was not designed for a 10 year-old and has a steeper learning curve than MINDSTORMS NXT
2. The next component is the “build a block” capability. This is the really exciting part. It’s ideal for defining support for third party sensors; in fact, these were stated as the primary target for this functionality. With this toolkit, there are three steps you do in building your own block.
1. You write the low-level NXT instructions for the block. You start with a template that defines the input and output connectors to your block and fill in the details for interacting with your sensor.
2. You write the code for the configuration page (the GUI that is displayed in the lower left corner of MINDSTORMS NXT). You start with a template and build the GUI using regular LabVIEW. The toolkit has all the UI elements so that your config pane looks like the ones that were shipped by LEGO.
3. You start with another template to write code that displays the graphical icon that will be used for your block.

You’re done. Your block is complete. This is the same toolkit that NI used internally in developing the NXT-G blocks.

There has been a logjam in supporting new third party sensors on the NXT. Most (all?) of the available digital sensors have been constrained to behave like a LEGO UltraSonic (US) sensor so that NXT-G software can be “tricked” into supporting them. This means that any value they return has to fit into a single byte because that’s what the US sensor support.

So for example, a digital compass might return reading values of 0 to 180 representing even numbered (0, 2, 4, 6, ….,  358, 360) degrees. With the “build a block”, the return value could be the true higher resolution available with the sensor.  I’m sure the release of the toolkit will accelerate the availability of the new sensors.

Vernier Software and Technology – More Sensors for the NXT
One of the press announcements at NI Week was a new collaboration between LEGO, NI, Tufts University, Carnegie-Mellon University and Vernier Software/Technology to “to produce a continuum of robotics sets, programming software, sensors, and curriculum aimed at students”.  The first four companies/universities have always been visible in the NXT efforts. But who is Vernier? Vernier (http://www.vernier.com/probes) happens to be one of the biggest suppliers of sensors into the education market; the ability to use their sensors will add an incredible array of new sensors to the NXT.

Vernier makes dozens of different sensor types. The picture below shows four different Vernier sensors (force, magnetic field, current, acceleration). For their newer sensors, Vernier has standardized on the BT (“British Telecom”) connector. The little black rectangular box in the picture is an prototype adaptor that will convert between the Vernier and NXT cables/connectors.

A little more detail on the adaptor can be seen in the following picture.

 

One of the neat things about Vernier sensors is that all their newer sensors have an auto-identification feature. You simply plug the sensor in and the “host” system is able to determine the type of sensor.
 

Wireless SONY Playstation Controller Interface for the NXT
mindSensors has a really cool new “sensor”. It hasn’t been formally announced yet, but I had an early version that I was demonstrating. It’s an interface to a wireless SONY Playstation Controller. Here’s a picture showing the complete system.

The blue controller is a “MadCatz Predator” wireless controller for the Sony Playstation. I bought mine at Walmart for $14.95. The black rectangular shape is the wireless base station that comes with the MadCatz. There’s a small $5 cable that converts from the Sony 9-pin connector format to the sensor. The green circuit board is the ‘sensor’ from mindSensors. There are four analog joysticks and 12 buttons on the controller. The values of these can all be read by the NXT over the wireless link. It’s a bidirectional link so there’s even control over the ‘rumble’ motors in the controller handles – useful feedback for when your robot crashes! The actual device driver for the sensor is written using the RobotC system – it’s about 100 lines of code.

With the new Build a Block toolkit it will only be a matter of time before there will also be a NXT-G driver!

Here’s a close up of the sensor and the wireless base station.

 

Sensors Galore
Some people bring their robots to an event like this to show off. Brian, Steve and Chris brought a lot – I can’t complete with them. So I brought my collection of NXT digital sensors. I need a sensor multiplexer.

Counter-clockwise from the top left are two different infra-red sensors for communicating with RCXs and the existing infrared remote, a multiplexer for four legacy RCX motors, the SONY Playstation controller interface, a standard LEGO ultra-sonic sensor, the Vernier adaptor and a prototype video camera for the high-speed sensor port.
 

Video Cameras and the NXT
I’m working on support for a smart video camera connected to the NXT using the high speed RS-485 sensor port. The device drivers for it use the RobotC system. Video cameras are already available on several of the different robotics controllers supported by RobotC. For camera support on RobotC for NXT it’s a straight forward software porting and minor change to hardware from RS-232 to RS-485!

Prototype hardware is now complete. It was real easy. Start with a standard commercial camera (an AVRCAM was used but a CMUCAM would also have worked) that was then modified to change from an RS-232 to RS-485 interface required for the NXT high speed sensor port; the hardware change wasn’t much more than swapping an RS-232 signal driver IC with an RS-485 IC and accommodating pin out differences. The modified camera is shown in the following picture.

The biggest remaining challenges are:

1. Converting onboard camera firmware from full duplex RS-232 based signaling to half-duplex RS-485 signaling.
2. Testing the NXT high speed sensor port. This may be the first application for it.
3. Adding enhanced functionality to utilize the NXT’s Bluetooth signaling to provide near real-time frame upload to the PC. Hopefully you’ll get “live” camera pictures on your PC!
4. Finding and selecting a hardware partner to (hopefully) productize the prototype.

The initial development used an unmodified camera with a RS-232 interface. It’s connected to a standard ARM7 development board (i.e. same CPU family as the NXT) which is running the RobotC firmware. See picture below. The camera driver has now been developed for the RobotC system using this configuration. Once the on-board camera firmware is modified to support RS-485 it will then easy to add an RS-485 camera driver to the RobotC firmware and move it on to the NXT.