Anomaly detection (AD) identifies outliers for applications like defect and lesion detection. While CLIP shows promise for zero-shot AD tasks due to its strong generalization capabilities, its inherent Anomaly-Unawareness leads to limited discrimination between normal and abnormal features. To address this problem, we propose Anomaly-Aware CLIP (AA-CLIP), which enhances CLIP's anomaly discrimination ability in both text and visual spaces while preserving its generalization capability. AA-CLIP is achieved by a straightforward and effective two-stage approach: it first creates anomaly-aware text anchors to clearly differentiate normal and abnormal semantics, then aligns patch-level visual features with these anchors for precise anomaly localization. AA-CLIP uses lightweight linear residual adapters to maintain CLIP's generalization and improves AD performance efficiently. Extensive experiments validate AA-CLIP as a resource-efficient solution for zero-shot AD tasks, achieving state-of-the-art results in industrial and medical applications. (code is available in Supplementary Material)