Takumi Nishiyasu

Takumi Nishiyasu

Cooperative Research Fellow at The University of Tokyo

About me

Takumi Nishiyasu is a Researcher at Mantra Inc. and a Cooperative Research Fellow at the Institute of Industrial Science, The University of Tokyo. He completed the doctoral course requirements at the University of Tokyo in March 2026 and is currently a Ph.D. candidate scheduled for his dissertation defense in July 2026. His research focuses on computer vision, gaze behavior analysis, and multimedia understanding.

Education

[2026.07 (Expected)] Ph.D. in Information Science and Technology, The University of Tokyo
[2026.03] Completed doctoral course requirements without dissertation
Supervisor: Prof. Yoichi Sato
Thesis: Multi-Layered Alignment for Understanding the Relationship between Task and Gaze

[2020.03] M.S. in Information Science and Technology, The University of Tokyo
Supervisor: Prof. Yoichi Sato
Thesis: Aesthetic-aware image cropping with convex quadrilaterals (in Japanese)

[2018.03] B.S. in Engineering, The University of Tokyo
Supervisor: Prof. Yoshihiro Kawahara
Thesis: Design of Thin Film Antenna for Soil Sensor with Big-Roll Method Leveraged Installation (in Japanese)

Research and Professional Experience

[2026.04 – Present] Cooperative Research Fellow, The University of Tokyo
Collaborative research on human gaze and computer vision within the Yoichi Sato Laboratory.

[2025.12 – Present] Researcher, Mantra Inc.
Developing “Mantra Engine” for automated manga translation.

[2024.04 – 2024.10] Visiting Ph.D. Researcher, PUI Group, University of Stuttgart
PI: Prof. Dr. Andreas Bulling.
Researched on gaze behavior modeling.

[2022.08 – 2023.12] Research Intern, CyberAgent Inc.
Mentor: Dr. Wataru Shimoda (Creative Research Team).
Researched on image cropping under design constraints.

[2022.10 – 2023.09] Research Intern, OMRON SINIC X Corp.
Mentors: Dr. Kazutoshi Tanaka and Dr. Shigeo Yoshida.
Researched on action generation for sports robots.

[2020.04 – 2021.03] Master Researcher, The University of Tokyo
Contributed to the Ego4D and GO-Finder projects and lab management.

[2018.08] Engineering Intern, VOYAGE GROUP Inc. (now CARTA HOLDINGS)
Engaged in web front-end development and system engineering.

Selected Publications

  • Takumi Nishiyasu and Yoichi Sato. “Gaze-to-Task Inference in Chart Reading: Best Practices for Integrating Human Attention with Multimodal LLMs.” Proc. ACM Comput. Graph. Interact. Tech. (PACMCGIT), Vol. 9, No. 2 (ETRA 2026), 2026.
  • Takumi Nishiyasu, Zhiming Hu, and Andreas Bulling. “Learning Alignments of Human Gaze and Fine-grained Task Descriptions.” Proc. ACM Comput. Graph. Interact. Tech. (PACMCGIT), Vol. 9, No. 2 (ETRA 2026), 2026.
  • Takumi Nishiyasu and Yoichi Sato. “Gaze Scanpath Transformer: Predicting Visual Search Target by Spatiotemporal Semantic Modeling of Gaze Scanpath.” Proc. CVPR Workshops (GAZE 2024), 2024.
  • Kristen Grauman, …, Takumi Nishiyasu, …, and Jitendra Malik. “Ego4D: Around the World in 3,000 Hours of Egocentric Video.” Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  • Takuma Yagi, Takumi Nishiyasu, Kunimasa Kawasaki, Moe Matsuki, and Yoichi Sato. “GO-Finder: Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery.” Proc. 26th International Conference on Intelligent User Interfaces (IUI), 2021.

Awards

  • Best Poster Award, GAZE 2024 (CVPR Workshop), 2024
  • Young Researcher Award (ヤングリサーチャ賞)/ DICOMO2018
  • Excellent Paper Award (優秀論文賞)/ DICOMO2018

Academic Services

Organizer

  • Web Chair, ACM Symposium on Eye Tracking Research & Applications (ETRA), 2025

Reviewer

  • Conferences: CVPR (2024, 2025), ETRA 2026, ETVIS 2025
  • Journals: IJCV, IMWUT (PACM IMWUT)