Painting the World with Pixels

Pranav Mistry is the man behind Sixth Sense, a tool that connects the physical world with the world of data.

Unveiled at the Technology Entertainment Design (TED) meet in 2009, Sixth Sense projects visual information on to walls and physical objects; and it can track hand gestures. A user can take a photograph simply by framing the scene with his hands or can tell the time by drawing a circle on his wrist. A speaker at the inaugural TED meet in India, scheduled on 4-7 November at Mysore, Mistry tells us it’s time for tech to keep pace with us. And not the other way around.

Q How does Sixth Sense differ from a gadget that allows you to access the Internet on the move?

A It is a new way of interacting between the physical and the digital world. I have, for four to five years now, been trying to merge the two worlds with a difference. The digital world today is not connected to our everyday environment. Whether it’s a computer or a gadget, we are forced to stare at a screen and turn away from the world. The idea is to take the digital world and merge it with the physical world.

Sixth Sense allows you to access the digital world, whenever and wherever you want. So far, information was confined in books and screen pixels. Now you can get information out and paint the world with pixels. Say, you are in a supermarket looking at a woolen garment. And even as you look at it, you get information like where the wool has come from, whether the production process has been environmentally friendly, or how effective it is for a certain temperature. This information is available seamlessly, in your day-to-day world.

Q Why is it that so far humans have been forced to adapt to technology rather than the other way around?

A The reason why this has not happened so far has to do with how we look at and design technology. The question always is what do we do with it. Instead, we have to look at what we need, what is the output I require and go about designing technology that fits that requirement.

I often use an analogy to illustrate this. You can open the fridge, look at the ingredients there and toss them together for a meal. Or, you can start with what you want eat, walk to the store, buy the ingredients and cook what you actually want.

The real way ahead is through ubiquitous computing. We have to start with the user so that computing is available where it is needed. Technology should learn human behaviour, it is not for us to keep pace with technological changes but for technology to keep pace with us, learn how we behave.

Q How do you see such an approach being implemented?

A I want to continue working on integrating the digital world into the physical world. It is this world that humanity has lived in for thousands of years. A major shift took place with the industrial revolution some 300 years ago and now, we are in the midst of the digital revolution. People have taken this to an extreme, some to the extent of living a ‘second life’. I don’t understand this, it draws people away from the physical world. I would rather that the digital world opens up new possibilities in the physical world.

The issue is that it is not clear how we go about this. Let me give you an instance of how I think it should work. If my mother wants to message me, I don’t think it’s a good idea that she has to acquire a knowledge of working on a computer and using the Internet to do so. I should be able to provide an interface she is familiar with, so that she can harness this new technology to do what she wants. Ideally, it should be a paper and pen tablet that allows her to write the message in the way she is used to and it should reach me. This is the way to go.

Consider farmers in rural India. They should not be burdened by having to learn an entire new language such as English to access all the information they need about crops. This is where the digital world can enter their lives seamlessly, without their having to change or adapt their life to a new technology.

Q Now that Sixth Sense is a viable technology, how do you intend to take it forward?

A The media lab in MIT has a number of prominent sponsors such as Samsung, LG and Google. They all want to enhance the product in some way. A telecommunications company may think of enhancing the experience of mobile telephony or a software company may want to enhance the entire computing experience. But what really interests me is making Sixth Sense available to the public. I want to provide the basic code as an open source and let people make their own Sixth Sense. Rather than give it to one company, I want to let it evolve through people.

I got an email from an NGO in India that wanted to use it to provide accessibility to blind people. This is more relevant than any use I can think of. For the blind, it will be a fifth sense. It can translate sign language into text. It is wonderful to see such applications.

Q In much that you say you still seem focused on how technology will impact India. Does India influence the way you think about digital technology?

A I grew up in India. It is always with me and the first thing that I think of is what this new technology can make available to the masses. Which is why I am not interested in a $20,000 Sixth Sense interface, but in one that costs no more than $300 and can be used by all. The idea is not to change people’s habits by constraining them to sit in front of a screen to access information. They should be able to do so in their day-to-day lives on the go. Farmers may be illiterate in English but they are not illiterate in their language and technology that can play to their strengths is the technology I’m interested in providing.

Q Do you see a considerable overlap in your ideas and TED’s approach to information?

A TED has changed the lives of many people. It has spread the expertise at the very highest level to people everywhere. Anyone who can access the Internet can listen to the foremost experts in various fields talking about their subject with passion. Now TED is available in several languages and is not limited to those who know English. After all, what can be better than the fact that a person sitting anywhere in a cybercafe can draw inspiration from Bill Gates’ words.

About The Author

Hartosh Singh Bal

Hartosh Singh Bal turned from the difficulty of doing mathematics to the ease of writing on politics. Unlike mathematics all this requires is being less wrong than most others who dwell on the subject.