Kinsight

Kinsight, a project developed by two computer science researchers, is a depth-camera based system that keeps track of household objects as they move around the house. It relies on Microsoft’s Kinect sensors which are attached to a computer which runs the software that the team developed.

The researchers want to build a Smart search engine for homes by tracking the locations of objects. The search engine should be able to answer questions like where are my sunglasses, my TV-remote of my wallet. This is what the “real-world” scenario could be like.

Alternative solutions, like the use of RFID chips, already exist. But the researchers said that their system is much cheaper because of the high cost of RFID chips.

The system focuses on tracking human figures and then looking for objects that have changes position in their vicinity. The system works like this because objects can only change position when humans move them. And they can’t track all the owner’s objects in real-time because this would be too processor-intensive.

Commonsense notions are built into the Kinsight program although the Kinect sensor’s capabilities are limited. This means that the sensors know that a cup of coffee is most likely to be found at a kitchen sink or table and not inside a bath or bed. These object recognition algorithms can be used to identify an object by analyzing the likelihood of it being somewhere. Algorithms were also created by analyzing the data gathered. These algorithms help the computer learn appearances of objects and the context they are likely to be used in.

They tested the system by labeling 48 objects like knives, keys, remotes, toys and a ketchup bottle and identifying 80 possible locations in the house. Volunteers were then asked to remove the objects and place them somewhere else. The result was a disappointing, with lots of errors when the objects were small, far away, transparent or placed to closely together. So there is definitely room for improvement, but the researchers said that if they use more sensors per room and more sensitive depth-cameras, the problems should be addressed. Until then, the program can be used to tell you were the objects were last seen, and this can also be helpful.