Let Humanoids Hike! Integrative Skill Development on Complex Trails

Despite significant progress in humanoid robotics, research remains fragmented: low-level motor skill learning often disregards the influence of long-horizontal goals on current movement and lacks situational awareness. While, high-level navigation struggles to accommodate real-world constraints and adapt to the irregularity of local terrains, falling short in last-step feasibility. To bridge these gaps, we present LEGO-H, a universal learning framework that trains humanoid robots to become expert hikers on complex trails by developing and integrating skills across all levels, embracing physical embodiment through both visual perceptual awareness and body dynamics. At the heart of LEGO-H's designs is the harmonization of robots' visual perception, decision-making, and motor skill execution -- grounded in the new perspectives on the Hierarchical Reinforcement Learning (HRL) framework and the knowledge transfer process of privileged learning. Our key innovations include: (1) TC-ViTs, a Temporal Vision Transformer variant tailored into HRL, framing local navigation as a sequential hallucination task, softly guiding locomotion policy learning. This design seamlessly grafts locomotion and goal navigation into a unified, end-to-end policy learning framework. (2) Hierarchical Loss Metric for Policy Distillation. To ensure the varsities of motor skills, LEGO-H harnesses the power of privileged learning. However, humanoid robots are highly articulated, where rationality of the structural relationship between joint actions impacts real-world execution. We thus emphasize the importance of a hierarchical loss metric framework that reflects kinematic dependencies—an essential yet overlooked element in the literature—for oracle policy distillation under privileged learning. We employ Variational Autoencoder (VAE) latent representations and reconstructions as a task-agnostic hierarchical loss function set to optimize policy training. LEGO-H addresses challenges from both the physical constraints of humanoid robots and dynamic environments across various time scales, without relying on biased motion priors. Extensive experiments on diverse simulated hiking trials demonstrate LEGO-H’s robustness and versatility. We hope LEGO-H could serve as a baseline prototype for humanoid robots in this underexplored hiking domain.