Co-op at Thalmic Labs

Published 2015/01/06

“When our founders, Stephen, Matthew, and Aaron, started thinking about the future of technology, they wanted to give users a way to interact with digital devices in a natural and mobile environment. They came up with the idea of a wearable device that would communicate based on EMG signals from the forearm, rather than touch, voice control, or camera-based control. And since the Myo™ armband doesn’t rely on cameras, it can travel with the user wherever they go.” -Thalmic Labs Blog

Thalmic was the first place I’ve been able to work on real dedicated hardware. The Myo is an awesome piece of wearable technology that allows you to use gestures to quickly trigger actions. Working on wearable technology forced me to learn a lot about asynchronous processing, and make a lot of ridiculous arm motions while testing. During my term, I served as a jack-of-all-trades in the software department; learning everything from iOS development to MIDI processing. Here I’ll talk about some of the various technologies I learned about in the process.

Re-engineering

Matlab is a popular language in the scientific programming community for its extensive library of algorithms and built-in plotting tools. The strength of the language lies in its support for matrices, and rapid manipulation thereof; something we used a lot of at Thalmic. My first task as a software engineer at the company, was to re-engineer parts of the Matlab signals analysis codebase into C++, the language that was primarily used in the distributed software. The intent was to rapidly prototype new processing techniques in Matlab, where engineers could take advantage of the data visualization tools, before porting the features to the true codebase.

Accordingly, I worked as a bridge between the Machine Learning (ML) and software teams. This involved a lot of research to understand both the Matlab language (with which I was inexperienced) and the underlying math (with which I was somewhat less inexperienced). Along the way I learned about some great things like quaternions (numerical system for manipulation of 3D objects), and some not-so-great things like Matlab’s index reference syntax.

Opinionated iOS

Dealing with iOS was another heartache. I wanted to badly to love the Core* libraries bundled with the XCode IDE, but the work that I was doing required fine tuning of low level bluetooth operations. Apple has developed very concise libraries intended to abstract all lowe level operations which proved to be a hindrance in this case. Granted, I may be missing something, as this was my first time working with Objective-C as well.

I really enjoyed the function naming in Objective-C; to avoid redundancy in the function signature, the function name is made up of a prefix and the argument names. It had been a long time since I’d worked in a dedicated IDE for long periods of time and I think that XCode has done a nice job of adding features without constantly getting in the programmers way. Despite the code completion features, I still felt that I worked much slower compared to my usual Vim configuration where I have more practice with keyboard shortcuts.

MIDI

My final task of the work term was to integrate MIDI functionality into the MYO control libraries. These libraries allow developers to create their own scripts that can react to events generated by the armband. Events are hand gestures and arm movements that the wearer performs using the device. MIDI is short for Musical Instrument Digital Interface and has been used by DJs, musicians, and producers since the late 80s to simulate instruments and sound effects.

The best part about this project was the simulations I was able to run once the initial functionality was in place. I experimented with various ways of generating notes based on the position, velocity and acceleration of my arm, meaning that testing consisted of flailing my arm about like a mad man at my desk. My co-workers looked on, horror-stricken. The end result was a basic program that mapped the position of the arm, to the pitch of a note. By opening and closing your hand, you could trigger the note to go on and off using Garageband. This led to a round of ridiculous music-making in the lounge area as one of my gifts before departure.