Interpretable and explainable geophysical parameter estimation from remote sensing data is essential for monitoring and forecasting the processes on the Earth's surface. However, explainable estimations are difficult to achieve with black box models, especially when the labeled datasets are small and do not cover many scenarios. Focusing on soil moisture estimation, we introduce a physical autoencoder for fully polarimetric SAR data by combining a neural encoder network with a differentiable physical model acting as a decoder. The architecture provides an interpretable physical latent space, indicates the reliability of the predicted parameters, and can be trained in self-supervised and hybrid ways. We validate the soil moisture predictions on data from two high-resolution airborne campaigns and provide a detailed comparison between purely supervised, purely physical, self-supervised, and hybrid models. Compared to a purely supervised approach, the hybrid model performs similarly on independent and identically distributed (IID) data. At the same time, the physical decoder strongly influences the hybrid model on unseen out-of-distribution (OOD) data. Furthermore, the hybrid model helps to locate areas where the physical model needs improvements. Combining machine learning and physics benefits both domains and enables new methods for geophysical parameter estimation. The source code is available at https://github.com/nbasargin/nb2025earthvision.