At IDEO we’re pretty excited about multi-touch interfaces.

Most people who have heard of multi-touch know of it in the context of the iPhone. Being able to use more than one finger on the iPhone’s screen means support for some pretty intuitive interactions. Pinch a map to zoom out. Slide a finger to scroll through photos. Multi-touch is a big part of that little device.

But when you’ve got a large screen with multi-touch support, that’s when things start to get really interesting.

Most computer interfaces today assume a single user. Whether a screen is small or large, the computer powering it is usually designed for one user. There’s one keyboard and one mouse. And even if you were to add another, the software only allows for one cursor. Every interaction assumes a single user.

IDEO's love affair with post-it notes

But multi-touch, particularly on large displays, assumes multiple inputs at the same time. Multiple fingers, yes, or multiple hands from multiple people. In other words, it’s a system designed for “us” instead of for “me” — for collaboration instead of heads-down work.

At IDEO, a lot of the work we do is collaborative. Most of it is. And since most software doesn’t allow for more than one input at a time, they don’t fit much into the collaborative part of our work. But we can’t help but wonder what might happen if we started messing around with multi-touch.

On the technology side it’s still early in the game, but there’s already some pretty cool stuff out there:

Apple

Apple is definitely the technology leader in the multi-touch world. They’ve got a number of patents, many of which secure their lead in capacitive multi-touch technology. Capacitive multi-touch systems, like the iPhone, are fantastic. They can handle lots of inputs in a wide variety of environmental conditions without requiring any special equipment on the part of the user. The only problem is that right now there’s nothing commercially available larger than the iPhone.

One of these days the good folk in Cupertino may release larger screens based on their technologies, but until that day comes there’s very little we can make use of here…

Perceptive Pixel

Meanwhile, Jeff Han and the folks at Perceptive Pixel have been extending Han’s work on frustrated total internal reflection (FTIR) to produce large-scale multi-touch displays. Their most recent display is a vertical, wide-screen system that easily supports two or more people working together:

Han’s systems are great in many ways. The technology is fairly cheap to implement (all you need is a projector, a sheet of acrylic, some IR LEDs, and an IR-sensitive webcam), and as you can see it scales well to large sizes. The downside is that the sun is a great source of IR light, so it’s easy to confuse the system (hence the dark room in the video). You also need enough space behind your screen for a projector to throw its image.

Johnny Lee

Johnny Lee may be the coolest thing to come out of Carnegie Melon since John Forbes Nash. Among other impressive projects, he’s managed to build a multi-touch system out of any ordinary computer, a Wiimote, and a couple of IR-LEDs.

Part of what makes Johnny Lee’s stuff so cool is that it’s so cheap to pull off. All you need is $50 worth of additional equipment to turn a computer or projected image into a multi-touch environment. The only downside is that the system can’t handle more than two inputs at a time, which limits what you can do with it.

At IDEO we’ve been exploring the pros and cons of these and other multi-touch systems. This isn’t an exhaustive list, but most of the other stuff we’ve seen seems to be a variation one one or another of these approaches. How about you? Are there notable systems we’ve missed?

2 Comments:

  1. Phil Coyne

    23/08/2008 at 6:11 pm // Permalink

    Yes, there is a notable system you have missed, resistive. My company, SiMa Systems, has developed a multi-touch (MT) and multi-force (MF) resistive touch sensor that we feel meets or exceeds virtually all other MT sensors in terms of performance characteristics and cost.

    Our sensor achieves a level of resolution at 20 points/mm, the width of a human hair.With this level of resolution the sensor is able to accurately measure the ‘Z’ axis (force).With the accurate measure of force a new dynamic to MT emerges. The user is now able to navigate the display/pad then activate ‘without’ having to lift their finger(s) and retouch maintaining uncommon accuracy thereby enhancing the user experience. Because of this capability we have filed for a patent for a gesture recognition methodology we term “Dynamic Home Row”.

    Our sensor is appropriate for touch screens and touch pads. Since our touch pad is malleable it can be built into all manners of products such as furniture armrest, clothing, appliances etc. Our first product will be a mouse pad with our sensor built into the pad at consumer price points. With the performance of our sensor it will meet all the challenges of the most demanding applications such as CAD, Gaming etc.

    BTW, another MT technology that was also not mentioned was the “In-Pixel” sensor developed by Sharp.

  2. Gentry

    25/09/2008 at 12:06 pm // Permalink

    Phil, thanks for the heads-up. We hadn’t seen your sensor and will definitely check it out.

Leave your comments