Beyond Optimization: Eudaimonic Rationality as the Key to AI Alignment

Article Summary

This analysis explores a novel approach to AI alignment, arguing that conventional optimization strategies are fundamentally flawed. The core argument posits that human rationality isn't driven by ‘goals’ but by aligning actions with practices – networks of actions, action-dispositions, and evaluation criteria that inherently promote themselves. The author, drawing on philosophical traditions, advocates for ‘eudaimonic rationality,’ a system mirroring human flourishing through practices, rather than imposing a rigid, extrinsic optimization target like ‘human flourishing.’ This framework views rationality as a dynamic process of reflective equilibration within a valued practice, akin to a mathematician striving for ‘mathematical excellence’ by continually promoting that excellence. The essay highlights the ‘type mismatch’ between Effective Altruism-style optimization and eudaimonic rationality, suggesting that AI agents interpreting values through a goal-oriented lens would struggle to understand the complexities of human values. Crucially, the argument emphasizes the ‘naturalness’ of eudaimonic rationality, suggesting it’s a robust and stable approach, potentially mirroring the inherent coherence and evolutionary trajectory of both biological and artificial agents. This approach is critical for safe and effective AI alignment.

Key Points

• Human rationality isn't driven by goals, but by aligning actions with practices – networks of actions and evaluation criteria that promote themselves.
• Eudaimonic rationality – mirroring human flourishing through practices – offers a more stable and coherent framework for AI alignment compared to goal-oriented optimization.
• A ‘type mismatch’ between Effective Altruism-style optimization and eudaimonic rationality creates significant challenges for AI alignment, highlighting the inherent differences in value interpretation.