No Detail Found!

First Person Vision for Activity Prediction using Probabilistic Modeling.

Author(s)
Shaheena Noor Department of Computer Engineering, Hamdard University, Karachi,
VALI UDDIN Department of Computer Engineering, Hamdard University, Karachi,
Abstract
Identifying activities of daily living is an important area of research with applications in smart-homes and healthcare for elderly people. It is challenging due to reasons like human self-occlusion, complex natural environment and the human behavior when performing a complicated task. From psychological studies, we know that human gaze is closely linked with the thought process and we tend to “look” at the objects before acting on them. Hence, we have used the object information present in gaze images as the context and formed the basis for activity prediction. Our system is based on HMM (Hidden MarkovModels) and trained usingANN (Artificial Neural Network). We begin with extracting motion information from TPV (Third Person Vision) streams and object information from FPV (First Person Vision) cameras. The advantage of having FPV is that the object information forms the context of the scene. When context is included as input to the HMM for activity recognition, the precision increases. For testing, we used two standard datasets from TUM (Technische Universitaet Muenchen) and GTEA Gaze+ (Georgia Tech Egocentric Activities). In the first round, we trained ourANNs only with activity information and in the second round added the object information as well. We saw a significant increase in the precision (and accuracy) of predicted activities from 55.21% (respectively 85.25%) to 77.61% (respectively 93.5%). This confirmed our initial hypothesis that including the focus of attention of the actor in the form of object seen in FPV can help in predicting activities better.
Publication Details
Page(s) 545-558
Published Journal: Mehran University Research Journal of Engineering and Technology, Volume: 37, Issue: 4, Year: 2018

Request the full text from PASTIC

References
Kwapisz , J.R.,G.M. , and Moore,S.A., 2017.Activity Recognition Using Cell Phone Accelerometers”,2011 (Last Visit: 15 June 12 74 -82
S. , and Uddin, 2016.“Using ANN for Multi-View Activity Recognition in Indoor Environment”,International Conference on Frontiers of Information Technology 258 -263
DamaeviIius , R.,Vasiljevas , M.,AlkeviIius,Wofniak , M., 2016.Human Activity Recognition in AAL Environments Using Random Projections”,Computational and Mathematical Methods in Medicine 17 -
October,Krizhevsky , A.,Ilya , S.,Hinton,Pereira , F.,Burges , C.J.C.,Bottou , L.,Weinberger , K.Q., 2012.Imagenet Classification with Deep Convolutional Neural Networks”,Advances in Neural Information Processing Systems 25 1097 -1105
October,Fergus , R.,Bregler , C., 2010.Convolutional Learning of Spatio-Temporal Features”,Proceedings of 11th European Conference on Computer Vision 37 55 -7219
Laptev , I.,C. , and Stephens,M., 1988.“A Combined Corner and Edge Detector”,Proceedings of 4th Conference on Alvey Vision 147 -151
Dalal , N.,Triggs , B., 2005.Histograms of Oriented Gradients for Human Detection”,Computer Society 1 886 -893
Mahmudul , H.,Chowdhury , R.,Amit , K., 2014.Continuous Learning of Human Activity Models Using Deep Nets”,Proceedings of 13th European Conference on Computer Vision -
“PsychologyDictionary”,Sutton , C.,McCallum , A.,L. , and Taskar, 2006.“An Introduction to Conditional Random Fields for Relational Learning”,(Last Visit: 2 June -
2017.,/ (Last Visit: 2 May -
Noor, S., and Uddin, V., “Using ANN for Multi-View Activity Recognition in Indoor Environment”, International Conference on Frontiers of Information Technology, pp. 258-263, December, 2016.
DamaeviIius, R., Vasiljevas, M., AlkeviIius, J., and Wofniak, M., “Human Activity Recognition in AAL Environments Using Random Projections”, Computational and Mathematical Methods in Medicine, pp. 17, 2016
Zhang, M., and Sawchuk, A.A., “USC-HAD: A Daily Activity Dataset for Ubiquitous Activity Recognition Using Wearable Sensors”, Proceedings of ACM Conference on Ubiquitous Computing, pp. 1036-1043, New York, USA, 2012
Yang, X., and Tian, Y., “Super Normal Vector for Human Activity Recognition with Depth Cameras”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 39, No. 5, pp. 1028-1039, May, 2017
Cippitelli, E., Gasparrini, S., Gambi, E., and Spinsante, S., “A Human Activity Recognition System Using Skeleton Data from RGBD Sensors”, Computational Intelligence and Neuroscience, pp. 14, 2016.
Kumar, A., Singh, S.K., and Kala, R., “Human Activity Recognition in Real-Times Environments Using Skeleton Joints”, International Journal of Interactive Multimedia and Artificial Intelligence, Volume 3, No. 7, pp. 61-69, 2016
Mo, L., Li, F., Zhu, Y., and Huang, A., “Human Physical Activity Recognition Based on Computer Vision with Deep Learning Model”, Proceedings of IEEE International Conference on Instrumentation and Measurement Technology, pp. 1-6, May, 2016.
Ronao, C.A., and Cho, S.B., “Human Activity Recognition with Smart-Phone Sensors Using Deep Learning Neural Networks”, Expert Systems Applications, Volume 59, pp. 235-244, 2016.
Dengel, A., Sefen, B., Baumbach, S., and Abdennadher, S., “Human Activity Recognition Using Sensor Data of Smartphones and Smartwatches”, Proceedings of 8thInternational Conference on Agents and Artificial Intelligence, pp. 488-493, February 26-28, 2016.
Wang, L., “Recognition of Human Activities Using Continuous Autoencoders with Wearable Sensors”, Sensors, Volume 16, No. 2, 2016.
Ponce, H., Villaseor, M., and Pechun, L.M., “A Novel Wearable Sensor-Based Human Activity Recognition Approach Using Artificial Hydrocarbon Networks”, Sensors, Volume 16, pp. 1033, 2016
Yordanova, K., Krger, F., and Kirste, T., “Context Aware Approach for Activity Recognition Based on Precondition-Effect Rules”, Proceedings of IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 602-607, March, 2012
Zhu, Y., Nayak, N.M., and Chowdhury, A.K.R., “Context- Aware Modeling and Recognition of Activities in Video”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2491-2498,
Hasan, M., and Chowdhury, A.K.R., “Context Aware Active Learning of Activity Recognition Models”, Proceedings of IEEE International Conference on Computer Vision, pp. 4543-4551, 2015
Amaro, K.R., Minhas, H.N., Zehetleitner, M., Beetz, M., and Cheng, G., “Added Value of Gaze-Exploiting Semantic Representation to Allow Robots Inferring Human Behaviors”, ACM Transactions on Interactive Intelligent Systems, 2017
Das, D., Rashed, M.G., Kobayashi, Y., and Kuno, Y., “Supporting Human Robot Interaction Based on the Level of Visual Focus of Attention”, IEEE Transactions on Human-Machine Systems, Volume 45, No. 6, pp. 664-675, 2015.
Lowe, D.G., “Object Recognition from Local Scale- Invariant Features”, Proceedings of International Conference on Computer Vision, Volume 2, pp. 1150, 1999.
Lowe, D.G., “Distinctive Image Features from Scale Invariant Keypoints”, International Journal of Computer Vision, Volume 60, No. 2, pp. 91-110, 2004
Hinton, G.E., “Learning Multiple Layers of Representation”, Trends in Cognitive Sciences, Volume 11, No. 10, pp. 428-434, 2007
Krizhevsky, A., Ilya, S., and Hinton, G.E., “Imagenet Classification with Deep Convolutional Neural Networks”, Advances in Neural Information Processing Systems, Volume 25, Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q., (Editors), pp. 1097-1105, Curran Associates, Inc., 2012
0

Citations

0

Downloads

30

Views