1. Home
  2. Research
  3. Centers & Labs
  4. RIKEN Information R&D and Strategy Headquarters
  5. Guardian Robot Project

RIKEN Information R&D and Strategy Headquarters Multimodal Data Recognition Research Team

Team Leader: Yasutomo Kawanishi (Ph.D.)

Research Summary

Yasutomo Kawanishi (Ph.D.)

Our goal is to recognize surrounding environments and humans around a robot. We are focusing on signal processing and pattern recognition using sensory data in various modality.

We mainly tackle these topics.

  • Human activity recognition, Facial expression recognition, and Intention estimation.
  • Multimodal sensor fusion for more various and much detailed object recognition.
  • Detection and description of unknown events/objects.

Main Research Fields

  • Informatics

Related Research Fields

  • Engineering
  • Interdisciplinary Science & Engineering
  • Perceptual Information Processing
  • Intelligent Informatics


  • Human Attribute Recognition
  • Environment Recognition
  • Computer Vision
  • Multimodal Data Fusion

Selected Publications

  • 1. Daiju Kanaoka, Hakaru Tamukoh, Motoharu Sonogashira, Yasutomo Kawanishi.:
    "ManifoldNeRF: View-dependent Image Feature Supervision for Few-shot Neural Radiance Fields"
    In Proceedings of the 34th British Machine Vision Conference, (2023)
  • 2.Shu Nakamura, Yasutomo Kawanishi, Shohei Nobuhara, Ko Nishino.:
    "DeePoint: Visual Pointing Recognition and Direction Estimation"
    In Proceedings of the 19th International Conference on Computer Vision, (2023)
  • 3. Vijay John, Yasutomo Kawanishi.:
    "Progressive Learning of a Multimodal Classifier Accounting for Different Modality Combinations"
    Sensors 2023, 23(10), 4666 (2023)
  • 4.Masaya Mizuno, Tomohiro Fujita, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase.:
    "Subjective Baggage-Weight Estimation from Gait ---Can you estimate how heavy the person feels?---"
    In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), (2023)
  • 5. Tomohiro Fujita, Yasutomo Kawanishi.:
    "Future Pose Prediction from 3D Human Skeleton Sequence with Surrounding Situation"
    Sensors 2023, 23(2), 876 (2023)
  • 6.Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide.:
    "Towards Captioning an Image Collection from a Combined Scene Graph Representation Approach"
    In Proceedings of the 29th International Conference on MultiMedia Modeling (2023)
  • 7. Vijay John, Yasutomo Kawanishi.:
    "Audio and Video-Based Emotion Recognition Using Multimodal Transformers"
    In Proceedings of the 26th International Conference on Pattern Recognition (2022).
  • 8. Yasutomo Kawanishi.:
    "Label-Based Multiple Object Ensemble Tracking with Randomized Frame Dropping"
    In Proceedings of the 26th International Conference on Pattern Recognition (2022).
  • 9. Tomohiro Fujita, Yasutomo Kawanishi.:
    "Toward Surroundings-aware Temporal Prediction of 3D Human Skeleton Sequence"
    In Proceedings of the 26th ICPR Workshop: Towards a Complete Analysis of People: From Face and Body to Clothes (2022).
  • 10. Motoharu Sonogashira, Masaaki Iiyama, Yasutomo Kawanishi.:
    "Towards Open-Set Scene Graph Generation with Unknown Objects"
    IEEE Access, Vol.10, pp.11574-11583 (2022)

Related Links

Lab Members

Principal investigator

Yasutomo Kawanishi
Team Leader

Core members

Motoharu Sonogashira
Research Scientist
Vijay Cornelius Kirubakaran John
Research Scientist
Tomohiro Fujita
Postdoctoral Researcher

Contact Information

3rd Floor, Advanced Telecommunications Research Institute International
2-2-2 Hikaridai Seika-cho, Sorakugun, Kyoto
619-0288 Japan (Kansai Science City)
Email: yasutomo.kawanishi [at] riken.jp