Safety and alignment — Constitutional AI, red-teaming, interpretability
**Constitutional AI (Bai et al. 2022):** Anthropic's approach to scalable alignment using a written *constitution* of principles (drawn from UN Declaration of Human Rights, Apple ToS, industry best practices, and research input). • **SL-CAI** — model critiques and revises its own outputs against the constitution → supervised fine-tuning data. • **RL-CAI** — AI feedback (another model picks the more constitutional response) trains the reward model → PPO/DPO.
Claude's helpfulness, harmlessness, honesty (HHH) properties emerge from constitutional training + RLHF + red-teaming.
**Red-teaming** — adversarial testing. Automated (HarmBench, GCG attacks, many-shot jailbreaks, PAIR) and human (paid domain experts probing CBRN, autonomy, cyber). Anthropic, OpenAI, Google, Microsoft publish system cards with red-team findings.
**Dangerous capability evaluations** (AISI, Apollo, METR, frontier lab internal): • Bio — CBRN uplift; CB-Eval, WMDP-Bio. • Cyber — pentest range, CTFs, autonomous replication. • Persuasion / manipulation — deceptive-planning, sandbagging. • Autonomy / self-exfiltration — METR long-horizon tasks.
**Responsible Scaling Policy (RSP) / Preparedness Framework (PF):** Anthropic's RSP defines AI Safety Levels (ASL-2 current, ASL-3/4 trigger thresholds). If evals show crossing, deployment pauses until mitigations meet the standard. OpenAI's Preparedness Framework and Google DeepMind's Frontier Safety Framework are similar commitments.
**Mechanistic interpretability:** sparse autoencoders (SAEs), circuit analysis (Anthropic, DeepMind, Apollo). Open questions: superposition, polysemantic neurons, deceptive alignment.
**Pre-deployment processes:** capability evals → safety evals → red-team → model card / system card → usage policy → responsible disclosure → staged rollout.