Loading [MathJax]/jax/output/CommonHTML/jax.js
Skip to yearly menu bar Skip to main content


Poster

Human Motion Instruction Tuning

Lei Li · Sen Jia · Jianhao Wang · Zhongyu Jiang · Feng Zhou · Ju Dai · Tianfang Zhang · Zongkai Wu · Jenq-Neng Hwang

ExHall D Poster #170
[ ]
Sat 14 Jun 3 p.m. PDT — 5 p.m. PDT

Abstract: This paper presents LLaMo (Large Language and Human Motion Assistant), a multimodal framework for human motion instruction tuning. In contrast to conventional instruction-tuning approaches that convert non-linguistic inputs, such as video or motion sequences, into language tokens, LLaMo retains motion in its native form for instruction tuning. This method preserves motion-specific details that are often diminished in tokenization, thereby improving the model’s ability to interpret complex human behaviors. By processing both video and motion data alongside textual inputs, LLaMo enables a flexible, human-centric analysis. Experimental evaluations across high-complexity domains, including human behaviors and professional activities, indicate that LLaMo effectively captures domain-specific knowledge, enhancing comprehension and prediction in motion-intensive scenarios. We hope LLaMo offers a foundation for future multimodal AI systems with broad applications, from sports analytics to behavioral prediction.

Live content is unavailable. Log in and register to view live content