Abstract: Recently, multi-class anomaly classification has garnered increasing attention.Previous methods directly cluster anomalies but often struggle due to the lack of anomaly-prior knowledge.Acquiring this knowledge faces two issues: the non-prominent and weak-semantics anomalies.In this paper,we propose AnomalyNCD,a multi-class anomaly classification network compatible with different anomaly detection methods.To address the non-prominence of anomalies,we design main element binarization (MEBin) to obtain anomaly-centered images,ensuring anomalies are learned while avoiding the impact of incorrect detections.Next, to learn anomalies with weak semantics,we design mask-guided representation learning,which focuses on isolated anomalies guided by masksand reduces confusion from erroneous inputs through re-corrected pseudo labels.Finally, to enable flexible classification at both region and image levels,we develop a region merging strategy that determines the overall image category based on the classified anomaly regions.Our method outperforms the state-of-the-art works on the MVTec AD and MTD datasets.Compared with the current methods,AnomalyNCD combined with zero-shot anomaly detection method achieves a 10.8\% $F_1$ gain,8.8\% NMI gain,and 9.5\% ARI gain on MVTec AD,and 12.8\% $F_1$ gain,5.7\% NMI gain,and 10.8\% ARI gain on MTD.The code will be publicly available.