Kinect: Microsoft's unexpected hit
Motion controller hackers put Kinect to exciting new uses
By Mike Elgan | Computerworld US | Published: 16:05, 21 June 2011
Microsoft introduced a peripheral device for gaming in November called Microsoft Kinect for Xbox 360. The product turned out to be a braindead simple-to-use, innovative and completely new user interface. It became the fastest selling consumer electronics product in history, and has now sold more than 10 million units.
Kinect's reception has been more akin to an Apple product like the iPod, iPhone or iPad than a Microsoft one. Usually Microsoft has to spend hundreds of millions of marketing dollars just to become a second tier player. Think of Bing or Windows Phone 7.
But not the Kinect. It just took off. Everyone loves it. The Kinect is Microsoft's iPad, the hot consumer appliance with no significant competition.
Then something really unusual happened. Kinect became the hottest platform in the industry for bold new inventions. Scientists, hobbyists and inventors started "hacking" the Kinect to build crazy new contraptions, all without Microsoft's knowledge or permission.
Microsoft is used to battling pirates, counterfeiters, hackers and users who want to use without paying. So it was even more surprising when the company embraced and encouraged the movement. Microsoft's approach to Kinect reminds me of Google's approach to Android. In both cases, the companies have no idea where others will take the platform and take a hands off approach to the direction of innovation.
Microsoft officially sanctioned such "hacks" this week by releasing a software developers kit (SDK) for Kinect. What's unusual about the release is that Microsoft says it's "a free beta release for noncommercial applications."
In other words, it's for hobbyists, tinkerers, hackers, scientists and inventors, rather than industry partners, software developers and OEMs. The company plans to release a commercial version later, which can be used for third party applications that will be sold to the public.
Microsoft's SDK is very exciting for one major reason: People have already done mindblowing things with Kinect even without the SDK. Kinect has been hacked to make serious breakthroughs in technology for marketing, medicine, business, computer science, entertainment and robotics.
A Russian company called ARDoor has created an instore "mirror" called the AR Door Kinect Fitting Room that superimposes clothes on shoppers. They can quickly cycle through different outfits to see how they'll look.
An academic researcher at the Technical University of Munich created something he calls the "Magic Mirror." Designed for teaching anatomy, the system creates the illusion of a mirror that shows the user's insides through a "hole" in the body.
A graduate student at the University of North Carolina created a 3D video conferencing system using four Kinect devices. The system uses head tracking to simulate reality. For example, with a conventional videoconferencing system, the other user is just flat on the screen. If you're looking at the user straight on, the view doesn't change when you move to the side.
But with the Kinect-based 3D system, if you move to one side, you see the side of his or her face rather than just the front.
MIT researchers built a working version of the user interface from the Steven Spielberg movie Minority Report, which involves computer control via very fine hand and finger gestures.
Researchers at Germany's Fraunhofer Institute created an elegant system for controlling a Windows PC with in-the-air hand gestures.
One "Kinect hacker" invented a motion-controlled sound and laser system that enables an on-stage performer to control an audio track and laser lights just by moving around. The laser beams can imitate the performer's motion or respond to gesture-based commands, and the sounds mirror the movement of the lights.
A Portugese scientist created a shopping cart system called the wi-GO for people in wheelchairs that follows the user around automatically.
A team of researchers at Sweden's Lulea University of Technology invented an autonomous wheelchair for the blind. The wheelchair contains a map of its surroundings, which it uses to navigate. But it can also avoid obstacles and people.
French researchers are working on a Kinect-based system for reading sign language. The user signs, and the system translates it into written or spoken language.
UC Berkeley scientists created an autonomous flying robot using Kinect. The system's software can detect the floor, which it uses to know its own altitude, as well as objects and most importantly researchers to avoid flying into them.
Researchers at Technical University Munich used Kinect to enable a robot to shop and cook. Their custom software identifies the locations of products on a shelf, as well as ingredients on a counter, or cooking tools like pots, pans, spoons and so on.
Engineers at the University of Pennsylvania even taught a robot to read signs and posters using Kinect. The Kinect hardware is used to identify the signs, and compensate for different reading angles, which enables better optical character recognition.
Not bad for a cheap gaming peripheral. Microsoft has an amazingly hot product on their hands, and something even better: A hot platform.
There's no question that in-air gesture-based interfaces will be huge in the future. Microsoft's lead is so vast, and so many people have already invested so much time and effort mastering the system, that it's unlikely anyone will catch them.
Microsoft is helping out with SDKs. But to truly exploit this once in a generation opportunity, Microsoft should be funding hot Kinect projects, awarding grants to Kinect-based academic researchers and giving away Kinect units to inventors like candy. And Microsoft should wait and see who builds what, then buy the companies with technology that can be integrated into either Xbox or Windows.
Nobody should have to tell the company's motion gaming division the obvious truth: Microsoft needs to make the right moves if it wants to win. And so far, so good.