RIKEN Information R&D and Strategy Headquarters Multimodal Data Recognition Research Team
Team Leader: Yasutomo Kawanishi (Ph.D.)
Research Summary

Our goal is to recognize surrounding environments and humans around a robot. We are focusing on signal processing and pattern recognition using sensory data in various modality.
We mainly tackle these topics.
- Human activity recognition, Facial expression recognition, and Intention estimation.
- Multimodal sensor fusion for more various and much detailed object recognition.
- Detection and description of unknown events/objects.
Main Research Fields
- Informatics
Related Research Fields
- Engineering
- Interdisciplinary Science & Engineering
- Perceptual Information Processing
- Intelligent Informatics
Keywords
- Human Attribute Recognition
- Environment Recognition
- Computer Vision
- Multimodal Data Fusion
Selected Publications
Papers with an asterisk(*) are based on research conducted outside of RIKEN.
- 1.Fujita, T., Kawanishi, Y.:
"Future Pose Prediction from 3D Human Skeleton Sequence with Surrounding Situation"
Sensors 2023, 23(2), 876 (2023). - 2.*Phueaksri, I. , Kastner, M. A., Kawanishi, Y., Komamizu, T., and Ide, I.:
"Towards Captioning an Image Collection from a Combined Scene Graph Representation Approach"
In Proceedings of the 29th International Conference on MultiMedia Modeling (2023). - 3.
John, V., and Kawanishi, Y.:
"Audio-Visual Sensor Fusion Framework using Person Attributes Robust to Missing Visual Modality for Person Recognition"
In Proceedings of the 29th International Conference on MultiMedia Modeling (2023). - 4.*Li, J., Kawanishi Y., Deguchi, D., and Murase, H.:
"A Preliminary Study on View Independent Panoptic Scene Change Detection"
In Proceedings of the 2023 International Workshop on Advanced Image Technology (2023). - 5.
John, V., and Kawanishi Y.:
"A Multimodal Sensor Fusion Framework Robust to Missing Modalities for Person Recognition"
In Proceedings of the ACM Multimedia Asia 2022 (2022). - 6.*Kawanishi, Y., Ide, I., Chu B., Matsuhira C., Kastner, M. A., Komamizu, T., and Deguchi, D.:
"Detection of Birds in a 3D Environment Referring to Audio-Visual Information"
In Proceedings of the 18th IEEE International Conference on Advanced Video and Signal-based Surveillance (2022). - 7.
John, V., and Kawanishi Y.:
"Audio and Video-Based Emotion Recognition Using Multimodal Transformers"
In Proceedings of the 26th International Conference on Pattern Recognition (2022). - 8.
Kawanishi, Y.:
"Label-Based Multiple Object Ensemble Tracking with Randomized Frame Dropping"
In Proceedings of the 26th International Conference on Pattern Recognition (2022). - 9.
Fujita, T., and Kawanishi, Y.:
"Toward Surroundings-aware Temporal Prediction of 3D Human Skeleton Sequence"
In Proceedings of the 26th ICPR Workshop: Towards a Complete Analysis of People: From Face and Body to Clothes (2022). - 10.
Sonogashira, M., Iiyama, M., and Kawanishi, Y.:
"Towards Open-Set Scene Graph Generation with Unknown Objects"
IEEE Access, Vol.10, pp.11574-11583 (2022)
Related Links
Lab Members
Principal investigator
- Yasutomo Kawanishi
- Team Leader
Core members
- Motoharu Sonogashira
- Research Scientist
- Vijay Cornelius Kirubakaran John
- Research Scientist
- Tomohiro Fujita
- Postdoctoral Researcher
Contact Information
3rd Floor, Advanced Telecommunications Research Institute International
2-2-2 Hikaridai Seika-cho, Sorakugun, Kyoto
619-0288 Japan (Kansai Science City)
Tel:+81-(0)774-95-1360
Email: yasutomo.kawanishi [at] riken.jp