Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., … & Dafoe, A. (2023). Model evaluation for extreme risks. arXiv preprint arXiv:2305.15324.

This paper introduces a framework for evaluating the extreme risks posed by general-purpose AI systems. It emphasizes the importance of identifying both dangerous capabilities and the propensity for harm through “dangerous capability evaluations” and “alignment evaluations.” These assessments are crucial for informing stakeholders and ensuring responsible AI development, deployment, and security. The document details various dangerous capabilities, proposes integration into AI governance, and highlights early work and design criteria for these evaluations. It aims to prevent extreme risks by embedding rigorous evaluation processes throughout the AI lifecycle.

https://arxiv.org/pdf/2305.15324