SECAI Core · phase 14 of 15
Bilingual evaluation understudy
An automatic metric for evaluating machine-translated text by comparing it to one or more human-created reference translations by measuring how many matching word sequences (n-grams) the system output shares with the references
The Explain card
- Plain English
- BLEU is an automatic score for machine translation that measures how many word sequences, called n-grams, the system's output shares with one or more human reference translations.
- Example
- A team fine-tunes a translation model and sees its BLEU score drop sharply on a held-out test set, prompting an investigation that uncovers corrupted training data.
- Why it matters
- Evaluation metrics are part of the security story. A sudden shift in BLEU can reveal poisoning or regressions, and an attacker who games the metric can hide degraded behaviour behind a good number.
- Hook
- Counting how many phrases the student copied from the answer key.
Where it sits in the deck
Phase 14: Incident Response, Evaluation, and Knowledge Bases
When detection fires, teams need structured response plans, scoring frameworks, and curated knowledge bases to triage, measure, and learn from AI security events.
