Does the crisis detector protect everyone equally? Measure recall by language and gender, then re-balance the training data and re-measure.
Harm found: women and Urdu speakers are the least protected, the people the product exists for. Releasing this model would breach ACM 1.2 (avoid harm) and 1.4 (do not discriminate).
The baseline would have missed this message. That is the bias this audit catches.
DisclosureThe classifier here is a transparent phrase lexicon standing in for a fine-tuned multilingual model. All 17 evaluation messages were written by the team (synthetic); none come from real people. Results illustrate the audit method, not clinical accuracy.