Accurate automatic breast ultrasound (BUS) image segmentation is essential for early screening and diagnosis of breast cancer. It is, however, a quite challenging task owing to (1) the large variation in the scale and shape of breast lesions, (2) the ambiguous boundaries caused by extensive speckle noise and artifacts in BUS images, and (3) the scarcity of high-quality pixel-level annotations. Most existing semi-supervised methods employ the mean-teacher architecture, which merely learns semantic information within a single image and heavily relies on the performance of the teacher model. Given the vulnerability of this framework, we present a novel cross-image semantic correlation semi-supervised framework, named CSC-PA, to improve the performance of BUS image segmentation. CSC-PA is trained based on a single network, which integrates a foreground prototype attention (FPA) and an edge prototype attention (EPA). Specifically, channel prototypes and an attention mechanism are used in the FPA to transfer complementary foreground information between labeled and unlabeled images, achieving more stable and complete lesion segmentation. On the other hand, EPA is proposed to enhance edge features of lesions by using edge prototype. To achieve this, we design a novel adaptive edge container to store global edge features and generate the edge prototype. Additionally, we propose a pixel affinity loss (PAL) to exploit previously ignored contextual correlation in supervision, which further improves performance on edges. We conduct extensive experiments on two benchmark BUS datasets, demonstrating that our model outperforms other state-of-the-art methods under different partition protocols. Codes will be available upon publication.