The field of computer vision initially focused on human visual perception and has progressively expanded to encompass machine vision, with ongoing advancements in technology expected to drive further expansion. Consequently, image compressors must effectively respond not only to human visual perception but also to the current and future machine vision tasks. Towards this goal, this paper proposes a Test-Time Fine-Tuning (TTFT) approach for adapting Learned Image Compression (LIC) to multiple tasks. A large-scale LIC model, originally trained for human perception, is adapted to both closed-set and open-set machine tasks through TTFT using Singular Value Decomposition based Low Rank Adaptation (SVD-LoRA). The SVD-LoRA layers are applied to both the encoder and decoder of backbone LIC model, with a modified learning scheme employed for the decoder during TTFT to train only the singular values, preventing excessive bitstream overhead. This enables instance-specific optimization for the target task. Experimental results demonstrate that the proposed method effectively adapts the backbone compressor to diverse machine tasks, outperforming competing methods.