Kinect-Based Micro-Behavior Sensing System for Learning the Smart Assistance with Human Subjects Inside Their Homes

The automatic sensing and understanding of the contexts are important to make the appropriate decision in Smart Systems. Especially, understanding and tracing human activities and behaviors are useful elements that enable the system to support or to imitate them. There are many studies that identify the activity or behavior with using the various sensors, but many of them could only identify rough activities, and some studies badly infringe on privacy. In this study, we propose a micro-behavior sensing system with Kinect sensor. To reduce the invasion of privacy with Camera device like Kinect sensor, our system identifies the micro-behavior by using features only extracted from the skeleton data. We deployed our system to actual Smart Home in Nara Institute of Science and Technology Japan. Then we conducted an experiment during 15 days to collect actual daily activities and confirmed the identification accuracy of the micro-behaviors related to the cooking activity, such as picking up a seasoning, cutting, mixing, washing etc. In the paper, 5 main micro-behaviors are considered. The achieved accuracy in the classification is 78%.