Skip to yearly menu bar Skip to main content


Poster

PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation

Xinting Hu · Haoran Wang · Jan Lenssen · Bernt Schiele


Abstract:

We introduce PersonaHOI, a training- and tuning-free framework that fuses a general StableDiffusion model with a personalized face diffusion model to generate identity-consistent human-object interaction (HOI) images. While personalized face diffusion (PFD) models have advanced significantly, they often overfit facial features and fail to produce coherent full-body interactions with objects. To address this issue, PersonaHOI introduces an additional StableDiffusion (SD) branch to follow HOI-driven text descriptions in image generation. By incorporating proposed cross-attention constraints in the PFD branch, and spatial fusion strategies between SD and PFD branches at both the latent and residual level, PersonaHOI successfully blends personalized facial details with interactive non-facial regions, ensuring identity preservation and interaction coherence. Experiments, validated by a novel interaction alignment metric, demonstrate the superior realism and scalability of PersonaHOI, establishing a new standard for practical personalized face with HOI generation.

Live content is unavailable. Log in and register to view live content