The advent of deep learning (DL) has significantly advanced artificial intelligence, driving notable progress in fields such as language translation, object recognition, and recommendation systems. Despite these successes, the computational complexity of advanced DL models continues to impede their practical deployment, particularly in clinical settings. Addressing this challenge, we introduce NExNet Seg, the Neuron Expansion Network for Medical Image Segmentation. Inspired by Progressively Expanded Neuron (PEN) structures and Manhattan Self-Attention (MaSA) mechanisms, NExNet Seg achieves exceptional accuracy with high parameter efficiency. It substantially reduces computational overhead, making it especially suitable for segmentation tasks in skin lesions and colorectal cancer using dermoscopic and endoscopic imagery. Empirical evaluations conducted on publicly available datasets—including PH2, ISIC (2016-2018), CVC Clinic, and Kvasir—demonstrate that NExNet Seg consistently outperforms current state-of-the-art methods in terms of accuracy, computational efficiency, and generalization capability. These results highlight its potential as an effective, scalable solution for clinical deployment in medical image segmentation. Code available at: https://github.com/MAIN-Lab/NExNet_Seg