RIKEN Information R&D and Strategy Headquarters Multimodal Data Recognition Research Team
Team Leader: Yasutomo Kawanishi (Ph.D.)
Research Summary

Our goal is to recognize surrounding environments and humans around a robot. We are focusing on signal processing and pattern recognition using sensory data in various modality.
We mainly tackle these topics.
- Human activity recognition, Facial expression recognition, and Intention estimation.
- Multimodal sensor fusion for more various and much detailed object recognition.
- Detection and description of unknown events/objects.
Main Research Fields
- Informatics
Related Research Fields
- Engineering
- Interdisciplinary Science & Engineering
- Perceptual Information Processing
- Intelligent Informatics
Keywords
- Human Attribute Recognition
- Environment Recognition
- Computer Vision
- Multimodal Data Fusion
Selected Publications
- 1.
Daiju Kanaoka, Hakaru Tamukoh, Motoharu Sonogashira, Yasutomo Kawanishi.:
"ManifoldNeRF: View-dependent Image Feature Supervision for Few-shot Neural Radiance Fields"
In Proceedings of the 34th British Machine Vision Conference, (2023) - 2.Shu Nakamura, Yasutomo Kawanishi, Shohei Nobuhara, Ko Nishino.:
"DeePoint: Visual Pointing Recognition and Direction Estimation"
In Proceedings of the 19th International Conference on Computer Vision, (2023) - 3.
Vijay John, Yasutomo Kawanishi.:
"Progressive Learning of a Multimodal Classifier Accounting for Different Modality Combinations"
Sensors 2023, 23(10), 4666 (2023) - 4.Masaya Mizuno, Tomohiro Fujita, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase.:
"Subjective Baggage-Weight Estimation from Gait ---Can you estimate how heavy the person feels?---"
In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), (2023) - 5.
Tomohiro Fujita, Yasutomo Kawanishi.:
"Future Pose Prediction from 3D Human Skeleton Sequence with Surrounding Situation"
Sensors 2023, 23(2), 876 (2023) - 6.Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide.:
"Towards Captioning an Image Collection from a Combined Scene Graph Representation Approach"
In Proceedings of the 29th International Conference on MultiMedia Modeling (2023) - 7.
Vijay John, Yasutomo Kawanishi.:
"Audio and Video-Based Emotion Recognition Using Multimodal Transformers"
In Proceedings of the 26th International Conference on Pattern Recognition (2022). - 8.
Yasutomo Kawanishi.:
"Label-Based Multiple Object Ensemble Tracking with Randomized Frame Dropping"
In Proceedings of the 26th International Conference on Pattern Recognition (2022). - 9.
Tomohiro Fujita, Yasutomo Kawanishi.:
"Toward Surroundings-aware Temporal Prediction of 3D Human Skeleton Sequence"
In Proceedings of the 26th ICPR Workshop: Towards a Complete Analysis of People: From Face and Body to Clothes (2022). - 10.
Motoharu Sonogashira, Masaaki Iiyama, Yasutomo Kawanishi.:
"Towards Open-Set Scene Graph Generation with Unknown Objects"
IEEE Access, Vol.10, pp.11574-11583 (2022)
Related Links
Lab Members
Principal investigator
- Yasutomo Kawanishi
- Team Leader
Core members
- Motoharu Sonogashira
- Research Scientist
- Vijay Cornelius Kirubakaran John
- Research Scientist
- Tomohiro Fujita
- Postdoctoral Researcher
Contact Information
3rd Floor, Advanced Telecommunications Research Institute International
2-2-2 Hikaridai Seika-cho, Sorakugun, Kyoto
619-0288 Japan (Kansai Science City)
Tel:+81-(0)774-95-1360
Email: yasutomo.kawanishi [at] riken.jp