Kinect Interactivity Demo

A simple vision and interactivity experiment with the amazing Microsoft Kinect

Published
April 12, 2011
Reading Time
3 minutes

I’m sure by now everyone in the world has seen some ‘hack’ involving a Microsoft Kinect and some clever programming. Microsoft’s Kinect is a computer-vision dream come true; an infrared depth sensor with impressive resolution and a standard RGB video camera. Throw in some microphones for sound capture and you have a force to be reckoned with.

Released in 2010, the Kinect was developed for the Xbox 360 with the technology and legwork done by PrimeSense who wrote the libraries and on-board processing for the vision data. However, soon after it’s release, the Kinect’s USB protocol was hacked and libraries were written for pretty much every programming language under the sun and coders got to work.

With all of the crazy possibilities that the Microsoft Kinect offers, we got a Kinect for the Palo Alto office to play around with and learn about the possibilities it offers for storytelling, prototyping, and straight-up awesome.

I wanted to give a shot at a simple program to demonstrate the basics of the Kinect for a workshop. I thought it’d be fun to make the most bare-bones interaction possible to get peoples’ brains going on the possibilities with this rich of data. The program is a demonstration of the ease of dynamically filtering moving objects based on depth and using these objects to interact with non-physical objects of our choice. You can see the grayscale depth data in the silouhetted person + couch outline in addition to the noisy outline along with some smoothed outlines. The IDEO boxes have their own physical characteristics (weight, friction, etc) and use the person + couch outlines as boundaries for interaction.

All in all, it took a whopping 2-3 hours of coding, most of which was library and driver issues at the beginning, to code this quick demo, so you get a sense of how easy it is to work with in terms of developing.

Information on how it works and links to the source code are provided after the jump.

Behind the scenes, it is a basic application of rudimentary computer vision techniques. The premise is to use the depth field to outline the moving object(s) and others nearby, use that outline to create a virtual body that other virtual bodies will react with, and then superimpose the depth field over the outlined body in addition to drawing the additional bodies (IDEO letter boxes) for visual purposes.

The depth field is used to make a binary image of the moving person. A contour-finding function is applied to the binary image (really easy since it’s binary) in order to generate an outlined silouhette of the person and objects that are nearby in the depth field. This outline is then used to generate a body in the virtual physics world which the boxes can bounce off of, exactly like they would in a 2D world. You can add boxes which spawn with random sizes and physics properties using your mouse and clicking wherever you want them to appear from. The binary image is then replaced with a white background and the grayscale depth map of the objects within the chosen depth range. Lastly, the outline of the silhouette is displayed along with several iterations of smoothings in order to show their effects on reducing noise in your outlines. This helps not only aesthetically but also computationally to eliminate bodies from accidentally intersecting other bodies.

The coding was done in openFrameworks, a great open source C++ library which focuses on image processing, video, and graphics libraries. One of the best parts about openFrameworks is the inclusion of the OpenCV library which is the standard for image processing and computer vision applications which is perfect for this application.

The ofxKinect add-on was used to access the Kinect and get depth and RGB data. ofxBox2D was used for the physics behind it, which was ported over to openFrameworks by our very own Todd Vanderlin. The code for the project can be downloaded from the Google Code repository here. The code requires the following addons to the openFrameworks core library: ofxKinect, ofxBox2D, and ofxOpenCv (included in the openFrameworks FAT installation).

621 thoughts on “Kinect Interactivity Demo

  1. Robux Tix Generator

    Remember that in this build, speed is the most important to burst maximum DPS.
    So, the idea of event marketing seems rather titillating. ) or human weaknesses and flaws
    (why use parents the first name of one of their child’s
    as password so often.

  2. เกมส์บ้านผีสิง

    The overall gameplay system of Metal Gear 2 served as the foundation for its
    3D sequel, Metal Gear Solid. There you have the Top 5 Unusual costumes that
    were taking from a bigger list of top costumes and gives
    you hopefully a guidance and inspiration to look for something new in a costume when you are searching.

    Keep in mind that I have double-checked the timer on the TV many times to
    be sure that it wasn’t set to turn off by itself.

  3. Mikki Lenigan

    I’m really impressed together with your writing abilities and also with
    the structure for your weblog. Is that this a paid topic or did you customize it your self?
    Either way stay up the nice quality writing, it’s rare to
    look a nice weblog like this one today..

  4. definition of a psychopath

    When selling a wrecked car, your best option is to sell it to a company.
    Budget When you have finalized a car, you
    need to first see if it fits into your budget. Higher Density: Diesel cars have a much higher
    density than petrol cars giving it a better fuel economy and efficiency.

  5. Louboutin Fifi

    I’m not that much of a online reader to be honest
    but your sites really nice, keep it up! I’ll go ahead and
    bookmark your site to come back in the future. All the best

  6. Finley

    Ԍood day! Woսld you mind if I share үour blog with
    my zynga ǥroup? Thеre’s a lot of people thаt І think
    would reallly enjoy your сontent. Ρlease let mе know.
    Many thanks

  7. www

    You can adopt their styles to suit yours; this is one of the best ways to determine the
    right fashion for you. It compares the design of vintage motorcycles to today’s models.

    Wearing an elegant UGG boot with tight jeans can give you a classy and beautiful look.

  8. Reba Musel

    Hey very cool web site!! Man .. Beautiful .. Superb .. I’ll
    bookmark your website and take the feeds additionally?
    I am happy to seek out so many useful info here within the post, we’d like work out more techniques on this regard,
    thank you for sharing. . . . . .

  9. read source

    Individuals have provided some helpful clues on this page,
    just that you examine. “We also hope that other regions will gradually progress as the week began to recover water, but this depends on whether the chemicals imported from the supply in place. 00 on a Monday, then obtaining either $a hundred and fifty.

  10. More bonuses

    My brother suggested I might like this website.
    He was entirely right. This submit actually made my day.

    You cann’t believe just how a lot time I had spent for this information! Thank you!

  11. magic-bazaar.tumblr.com

    That’s what I think about it, that after a brief acquzaintance
    with the session. Support – Buddy is one such leading firm, presenting dedicated support for all kinds of Lenovo machines,
    of almost every configuration. For this
    click ‘Office’, then ‘Word Options’, and then the ‘Advanced’
    link on left.

  12. โหลดมายคราฟ

    Hello there! I know this is kinda off topic but I was wondering if you knew where I could find a
    captcha plugin for my comment form? I’m using the same blog platform
    as yours and I’m having trouble finding one? Thanks a lot!

Leave a Reply

Your email address will not be published. Required fields are marked *