1. Home
  2. Research
  3. Centers & Labs
  4. RIKEN Information R&D and Strategy Headquarters
  5. Guardian Robot Project

RIKEN Information R&D and Strategy Headquarters Multimodal Data Recognition Research Team

Team Leader: Yasutomo Kawanishi (Ph.D.)

Research Summary

Yasutomo Kawanishi (Ph.D.)

Our goal is to recognize surrounding environments and humans around a robot. We are focusing on signal processing and pattern recognition using sensory data in various modality.

We mainly tackle these topics.

  • Human activity recognition, Facial expression recognition, and Intention estimation.
  • Multimodal sensor fusion for more various and much detailed object recognition.
  • Detection and description of unknown events/objects.

Main Research Fields

  • Informatics

Related Research Fields

  • Engineering
  • Interdisciplinary Science & Engineering
  • Perceptual Information Processing
  • Intelligent Informatics


  • Human Attribute Recognition
  • Environment Recognition
  • Computer Vision
  • Multimodal Data Fusion

Selected Publications

  • 1. Masaya Mizuno, Tomohiro Fujita, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase
    "Subjective Baggage-Weight Estimation based on Human Walking Behavior"
    IEEE Access, Vol. 12, pp. 39390 - 39398, (2024) (open access)
  • 2. Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide.
    "Image-Collection Summarization Using Scene-Graph Generation With External Knowledge"
    IEEE Access, vol.12, pp. 17499 - 17512, (2024)
  • 3. Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide.
    "An Approach to Generate a Caption for an Image Collection Using Scene Graph Generation"
    IEEE Access, vol.11, pp. 128245 - 128260, (2023)
  • 4. Daiju Kanaoka, Hakaru Tamukoh, Motoharu Sonogashira, Yasutomo Kawanishi.
    "ManifoldNeRF: View-dependent Image Feature Supervision for Few-shot Neural Radiance Fields"
    In Proceedings of the 34th British Machine Vision Conference, (2023)
  • 5. Shu Nakamura, Yasutomo Kawanishi, Shohei Nobuhara, Ko Nishino.
    "DeePoint: Visual Pointing Recognition and Direction Estimation"
    In Proceedings of the 19th International Conference on Computer Vision, (2023)
  • 6. Vijay John, Yasutomo Kawanishi.
    "Progressive Learning of a Multimodal Classifier Accounting for Different Modality Combinations"
    Sensors 2023, 23(10), 4666 (2023)
  • 7. Masaya Mizuno, Tomohiro Fujita, Yasutomo Kawanishi, Daisuke Deguchi, Hiroshi Murase.
    "Subjective Baggage-Weight Estimation from Gait ---Can you estimate how heavy the person feels?---"
    In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), (2023)
  • 8. Tomohiro Fujita, Yasutomo Kawanishi.
    "Future Pose Prediction from 3D Human Skeleton Sequence with Surrounding Situation"
    Sensors 2023, 23(2), 876 (2023)
  • 9. Yasutomo Kawanishi.
    "Label-Based Multiple Object Ensemble Tracking with Randomized Frame Dropping"
    In Proceedings of the 26th International Conference on Pattern Recognition (2022).
  • 10. Motoharu Sonogashira, Masaaki Iiyama, Yasutomo Kawanishi.
    "Towards Open-Set Scene Graph Generation with Unknown Objects"
    IEEE Access, Vol.10, pp.11574-11583 (2022)

Related Links

Lab Members

Principal investigator

Yasutomo Kawanishi
Team Leader

Core members

Motoharu Sonogashira
Research Scientist
Vijay Cornelius Kirubakaran John
Research Scientist

Contact Information

3rd Floor, Advanced Telecommunications Research Institute International
2-2-2 Hikaridai Seika-cho, Sorakugun, Kyoto
619-0288 Japan (Kansai Science City)
Email: yasutomo.kawanishi [at] riken.jp