A trio of Multitouch Tables

I’ve built a few multitouch tables in the past while using different construction / illumination techniques each time & I learned different lessons on each build. My most recent effort was for 3 multitouch tables for a large tradeshow booth, I incorporated decisions made from all my past lessons, as well as some new tricks I had to come up with for the tradeshow floor. I was dealing with some serious “unknowns” and had to account for them (in addition to trying to imagine the ever present unforeseeable issues). A biggie was the lighting at the Philadelphia convention center – I assumed they had the same enormous arc lights I’ve seen at other convention centers. I had no idea what kind of infrared output these have and whether or not an arc light oscillates/strobes to a ccd camera. I had an interesting experience with this, see sidebar entitled “Strobing”.

Sidebar: Strobing The first time I set up the Pt.Grey Firefly with wide angle lens from Computar, it was a moment of anticipation to see if the math on paper was accurate in deciding which lens would give me a clear & non-distorted view of the table surface. Well that all worked out peachy but I saw a new problem through the camera. The fluorescent lighting overhead was giving off a considerable amount of noise, and even worse, the noise was oscillating/strobing in and out! The blob tracking software I use is CCV (community core vision), and it is simply great at what it does. Great because it’s got everything you need to tweak settings for your particular set-up. One new feature is called “learn” whereby CCV can sense a new addition to the environment which is not a touch and start to combine it with the background intended to be ignored, the “learn” is based on a time threshold. Sure, it works great when a lighting condition suddenly changes. Not so great for a rhythmic oscillation! Without Pt.Grey’s ability to dig into settings deep, I’d be stuck right then & there. I was able to access the shutter speed which is editable in very small steps… I stepped up the shutter speed gradually using the cursor keys and watched the oscillating light speed up until I matched that pesky glow overhead resulting in a “steady on”. I then had to tweak some of my other settings so they jive with the new shutter settings. Problem solved. Thank you Pt.grey!

I also wasn’t clear on the lighting in the booth. These unknowns can be detrimental, not only to a good user experience but to any functionality at all. So how could I deal with the possibility of non-working touches? Indeed, a very big question, the answer was about tiny levels of control… Granular control of all conditions to an almost ridiculous extent.

Hardware: Since I didn’t how tough the environment outside the table would be, I had to reduce the internal environment to the highest contrast view of touches possible. First I made the decision to use a Point Grey Research camera. The Sony PS3 Eye cameras are highly regarded for use in multitouch tables but I wanted the kind of access to driver settings that Pt.Grey provides. They are all about computer vision and the features of their drivers show it. This camera provides a C mount (one of the industry standards for a threaded lens mount, usually found on security cameras and microscopes) which brings me to the second part of my super ninjaskill solution.

getting painted white in Brooklyn.

I sourced a good glass lens with manual iris control made by Computar. Most lenses have manual focus and zoom but I wanted to be able to quickly tune out the environmental light. Third, I had to figure out a high quality way to filter out visible light. I have a lot of experience with Lee 87 IR filters as well as blank exposed 35mm film (an effective poor-man’s trick), but I needed a solution for mounting the filter without distortion at the super wide angle I was planning. In short, lenses that offer an extremely wide angle are bending a lot of light nearest the edges of the lens. You’re asking a filter (behind the lens) to do a different job at the lens center vs. the lens edge. Acrylic filters aren’t really good for this variance. I found a company that produces glass filters cut to custom sizes, they’re called Midwest Optical. They were able to explain in detail why coated glass will work in this situation and how acrylic may not. They also did the legwork of contacting PT.Grey for the CAD files of the fabrication inside the camera in order to cut a piece of glass filter to perfectly replace a layer of clear glass underneath the lens mount. They also knew Computar’s specifications aren’t always to be trusted and knew to stay away from producing a filter on the lens side because of this. These guys were fast, knowledgeable, friendly, and spoke to the camera manufacturer directly for me. I HIGHLY recommend them, they went above and beyond their obligation to make a sale with me. Please go to them with all your filter needs.

Moments before going live. Pictured with sound domes.

Ok, so I got the glass filters, opened up the camera, popped in my replacement and voila, the camera could no longer see visible light. I couldn’t tell if these were any better or worse than acrylic but I chose to trust the science and the vendor made the process easy so I’m happy about that path – plus, with such a big deal as the possibility of no touches on site as a risk, I had to err on the side of caution at every turn. Camera with lens and filters = done. Now it’s time to get my illumination straight. Since I planned to tune out the external environment completely, I needed some extreme illumination. Different methods exist – laser light plane (LLP), diffuse illumination (DI as front, rear, and side), frustrated total internal reflection (FTIR), and LEDLP. I think I may be the first builder to ever execute a LLP+FTIR hybrid. Individually, I was able to achieve touches via LLP and FTIR - with both types of illumination simultaneously active though, my touches were super bright and very responsive. The level of brightness I achieved enabled me to shrink the lens iris in a dramatic way and still see touches coming through. I essentially blacked out everything but the brightest blobs which were a great deal brighter than any noise that might be viewable from above. A hybrid of IR illumination methods - The trial was a success!

A mention about the sound domes - some of the content on the tables were video clips that had a narrated audio track and some subtle sound effects. In order to deliver crisp, clear audio to the user without competing with ambient tradeshow noise or the audio from the other tables, we localized audio by using sound domes. It was my first experience with them. They work remarkably well, although the ones we had were underpowered. It would start to "clip" when we hit the audio level that would have been ideal, but even so, we could achieve sound clarity and a usable volume in the most logical sweet spot for the user.

Software: We executed some special tricks on the software side as well. We had a lot of content provided by the client. All of it was to be redone for aesthetic purposes but we couldn't alter any of the user interface flow. In the world of pharmaceutical trade shows, once an experience has gone through it's long & arduous approval process, it is not to be changed (lest it go before lawyers for another round of approvals). Well, the approved interface flow did not lend itself well to a multitouch table designed for attendee education. It was decided that each table was to be staffed by a "driver" who could guide a user through the multifaceted and somewhat deep interface. So now we were faced with something new, an opportunity to create a new kind of interaction. Two people would stand opposite each other, both using the interface, with an opportunity for dialog through the entire experience. To help make the interface natural to use for both parties, we seized that chance to try something cool. We created a gesture (see Seth Sandler's blog post on AS3 Gestures) similar to the turning of a large dial which would rotate the whole screen 180degrees. As simple as that sounds, It was great fun to do. no matter where you spun your five fingertips, a huge 50"+ display rotation would ensue. I saw some people experimenting with the gesture recognition for the fun of it. I learned to always try to include a simple action with an unexpected payoff in the future. Everyone had a good smile to go with their new information - and that was from a trueblue utilitarian function of the interface, not a bit of eyecandy placed there purposely to add funvalue. I was very pleased with the accidental outcome.

Special thanks go to Seth Sandler for his expertise on the Flash Actionscript3 and the implementation of the Gesture. It's always smooth sailing with Seth, he's got skills, speed, problem solving, and a great enthusiasm and work ethic that gets the project executed with style, even if we're down to the final hours. Thanks again my friend.

Thanks also go to Sam Ewen and Interference Inc for the opportunity to build these Multitouch tables, it's always great working for progressive thinking and fast moving company.