Home-brew Multi-touch

Can't get your hands on a multi-touch device? Build one.

August 15, 2008
Reading Time
2 minutes

At IDEO we’re all about building to think. Learning from books and websites and product demos is cool, but we think the really good stuff comes when you get in there and start messing around for real.

In the case of multi-touch interfaces, that meant building a system we could start prototyping on. What we wanted was:

  • a multi-touch display large enough to facilitate use by several people at once
  • an API for flash that would let us quickly prototype multi-touch interfaces and applications

It took us about 5 weeks to get everything together. Kyle, one of our all-around gearheads, had already been building a drafting-table-style FTIR system in his garage which helped kick-start us.

We knew we wanted our system to be larger and vertical, though, so we went back to the drawing board on hardware. We had some old projection surfaces from a project we had done long ago in storage.

With that as our base we designed a system around a short-throw projector and a wide-angle IR cam. This allowed us to build a pretty large rear-projection display without the depth requirements of Han’s systems.

Meanwhile, Kyle and Nicholas (another all-around whiz) were hard at work building a multi-touch server and an API that could be used to quickly prototype interactions. They built the back-end in Processing. It’s a great platform for rapid prototypes, particularly where there’s hardware-software integration involved.

On the front end we chose to create an API for Flash because it lets us realize our ideas quickly and use a wide variety of media, from video to vector graphics. Nicholas’ Flash API is pretty cool: a system that allows anyone who knows ActionScript to quickly mock up multi-touch using reference to “fingers” (e.g. when two fingers are down on this image, scale it…).

Using our server and API, Nicholas was also able to very quickly throw together a handful of demos. Take a look:

Want to jump into the multi-touch arena by creating a system of your own? We’ve just published the code from our own prototype system at Google Code: it should be all the software you need to make your own FTIR system: a back-end app to run a multi-touch server, and an API to create your multi-touch flash applications.

You can find the source code for the FTIR server and the Flash API at http://code.google.com/p/ideo-multitouch/. Everything is licensed under Creative Commons.

This stuff is definitely not polished to commercial application standards: It’s use-at-your-own-risk, unsupported code. We’ve all got day jobs and can’t offer much by way of technical support, but if you find it useful we’re happy to have you make use of it.

And by all means if you make something you’re excited about with it, send it our way!


23 thoughts on “Home-brew Multi-touch

  1. Chris

    Excellent work and very nice to push the industry forward with sharing your source. I’ve been working on similar experiments with webcam interaction rather than multitouch. Similar methodologies in technologies that will be the future of Flash.

  2. David A. Mellis

    Very cool! I like the games; it’s nice to see some new ideas for uses of a multi-touch system. Plus, it’s great that you’re releasing the libraries as open-source. The code looks pretty intense; I’m sure a lot of people will find it useful.

  3. justin powell

    thanks for sharing all this great info – process, code, and samples. what an excellent way to use flash and processing for rapid prototyping.

    i hope more examples start popping up once word gets out. keep up the great work!

  4. Christian Moore

    Really great stuff guys… Looking forward to trying it out.

    Ask Nicholas if he’d ever want to help work on a AS3 Touch API :)

    Thanks for your contributions, where about are you guys located? Palo Alto?

  5. Iron

    I’m trying to use.
    All ok with the server, I install and run. But I don’t know what happned with the client, just doesn’t work! How can I run de “swf” to connect correctly with the server?

    The other problem: When I open the “*.fla” and export, the swf stop work.

  6. Kirit Tanna


    I am currently building a FTIR screen with help from the NUI group forums and I found your video very exciting more because your screen works in daylight. Can you share more details about the hardware and where it can be purchased… From what I’ve found –

    Wide angle IR cams –

    Short throw projector –

    Can you suggest which one is suitable? Thanks!

  7. Gentry Post author

    @Kirit: Like all FTIR systems, ours really struggles in ambient daylight. The video you saw was shot in a room with only florescent lighting.

  8. jwopitz


    I concur, I would like to see this in AS3. Seeing as Flex is becoming quite ubiquitous in the RIA market, this would be a valuable asset in kiosk/installation development. In fact, I have a personal non-OpenSource project that could make great use of this if it where in AS3.

    I could certainly volunteer some time to help you port it over if you like. Let me know the details.

  9. Pingback: Two (Relatively) New Blogs From IDEO | Thinking Interactive

  10. Pingback: Make More Money » Blog Archive » Homebrew Multi-touch display

  11. Pingback: Daily DIY Network - Science Projects Plans Guides » Blog Archive » Homebrew Multi-touch display

  12. Pingback: Homebrew Multi-touch display | DiyUs.com

  13. Pingback: Homebrew Multi-touch display at Virtual Instrument Test

  14. scintilla

    This looks awesome! Did you just use a rear projection screen as your FTIR surface? Or did you have an additional surface on top of your projection screen?

  15. Diego Uribe

    Great stuff IDEO’ers
    By any chance, is the brainstorming/ideation multitouch softaware displayed in the video part of the open code? Is it available to the public?



  16. Pingback: Playfool® - Darren Richardson » Blog Archive » Home-brew Multi-touch

  17. Pingback: Open Source Drivers for HomeBrew MultiTouch « Amul’s Digital Life?

  18. Pingback: Multi-touch for Adobe Flex: Introduction in Hardware and Software « I don’t like computers.

  19. Sebastian

    Great work you guys, I am in the preocess of building my onw multitouch table, i would like to know what material or “thing” did you use as a projection surface?


Leave a Reply

Your email address will not be published. Required fields are marked *


Reload Image