AI safety: a climb to Armageddon?

Philosophical Studies · 2025-03-06

Abstract This paper presents an argument that certain AI safety measures, rather than mitigating existential risk, may instead exacerbate it. Under certain key assumptions - the inevitability of AI failure, the expected correlation between an AI system's power at the point of failure and the severity of the resulting harm, and the tendency of safety measures to enable AI systems to become more powerful before failing - safety efforts have negative expected utility. The paper examines three response strategies: Optimism, Mitigation, and Holism. Each faces challenges stemming from intrinsic features of the AI safety landscape that we term Bottlenecking, the Perfection Barrier, and Equilibrium Fluctuation. The surprising robustness of the argument forces a reexamination of core assumptions around AI safety and points to several avenues for further research.

AI Safety: A Climb To Armageddon?

arXiv (Cornell University) · 2024-05-30

This paper presents an argument that certain AI safety measures, rather than mitigating existential risk, may instead exacerbate it. Under certain key assumptions - the inevitability of AI failure, the expected correlation between an AI system's power at the point of failure and the severity of the resulting harm, and the tendency of safety measures to enable AI systems to become more powerful before failing - safety efforts have negative expected utility. The paper examines three response strategies: Optimism, Mitigation, and Holism. Each faces challenges stemming from intrinsic features of the AI safety landscape that we term Bottlenecking, the Perfection Barrier, and Equilibrium Fluctuation. The surprising robustness of the argument forces a re-examination of core assumptions around AI safety and points to several avenues for further research.

J. Patout Burns, Augustine’s Preached Theology: Living as the Body of Christ

Augustinian Studies · 2024-01-01

Introspective Machines: Are LLMs Better at Self‐Reflection Than Humans?

Philosophical Perspectives · 2024-12-01 · 2 citations

ABSTRACT This article challenges conventional boundaries between human and artificial cognition by examining introspective capabilities in large language models (LLMs). Although humans have traditionally been considered unique in their ability to reflect on their own mental states, we argue that LLMs may not only possess genuine introspective abilities but potentially excel at them compared to humans. We discuss five objections to machine introspection: (1) the lack of direct routes to self‐knowledge in training data, (2) the conflict between static knowledge and dynamic mental states, (3) the distorting effects of reinforcement learning on self‐reports, (4) LLMs own denials of inner experience, and (5) arguments that LLMs simply mimic language without understanding. We think all these arguments fail and that there are deep parallels between human and machine introspection. Most provocatively, we propose that LLMs superior processing capabilities and pattern recognition may enable them to develop more sophisticated theories of mind than humans possess, potentially making them more reliable introspectors than their creators. If we are right, this has significant implications for artificial intelligence (AI) alignment, transparency, and our understanding of the nature of AI.

Making AI Intelligible: Philosophical Foundations

arXiv (Cornell University) · 2024-06-12 · 7 citations

Can humans and artificial intelligences share concepts and communicate? 'Making AI Intelligible' shows that philosophical work on the metaphysics of meaning can help answer these questions. Herman Cappelen and Josh Dever use the externalist tradition in philosophy to create models of how AIs and humans can understand each other. In doing so, they illustrate ways in which that philosophical tradition can be improved. The questions addressed in the book are not only theoretically interesting, but the answers have pressing practical implications. Many important decisions about human life are now influenced by AI. In giving that power to AI, we presuppose that AIs can track features of the world that we care about (for example, creditworthiness, recidivism, cancer, and combatants). If AIs can share our concepts, that will go some way towards justifying this reliance on AI. This ground-breaking study offers insight into how to take some first steps towards achieving Interpretable AI.

AI with Alien Content and Alien Metasemantics

Oxford University Press eBooks · 2024-05-22 · 3 citations

Abstract AlphaGo1 plays chess and Go in a creative and novel way. It is natural for us to attribute contents to it, such as that it doesn’t view being several pawns behind, if it has more board space, as bad. The framework introduced in Cappelen and Dever (2021) provides a way of thinking about the semantics and the metasemantics of AI content: does AlphaGo entertain contents like this, and if so, in virtue of what does a given state of the program mean that particular content? One salient question Cappelen and Dever didn’t consider was the possibility of alien content. Alien content is content that is not or cannot be expressed by human beings. It’s highly plausible that AlphaGo, or any other sophisticated AI system, expresses alien contents. That this is so, moreover, is plausibly a metasemantic fact: a fact that has to do with how AI comes to entertain content in the first place, one that will heed the vastly different etiology of AI and human content. This chapter explores the question of alien content in AI from a semantic and metasemantic perspective. It lays out the logical space of possible responses to the semantic and metasemantic questions alien content poses, considers whether and how we humans could communicate with entities who express alien content, and points out that getting clear about such questions might be important for more ‘applied’ issues in the philosophy of AI, such as existential risk and XAI.

Scoreboards Without Scorekeepers

2023-11-07 · 1 citations

Abstract Theories in the tradition of Lewis’s “Scorekeeping in a Language Game” conceive of linguistic expressions as providing instructions for updating a public conversational scoreboard. Metaphysically, that scoreboard is typically conceived psychologistically, as grounded in epistemically-coordinated mental states of conversational participants. But we can depsychologize scoreboards, taking them instead as mind-independent informational entities manipulated by linguistic moves. The psychologism is then relocated to a story about our collective tracking of the public scoreboard. Having scoreboards that are not effluences of our scorekeeping requires reorganization of much of our linguistic theory. As a test case, a reoriented perspective on presupposition and accommodation is given.

“Maimed and Naked Monks in the Bloodslaked Dust”: Augustine, Aquinas, and Cormac McCarthy on Just War

LIT Literature Interpretation Theory · 2023-01-02

Click to increase image sizeClick to decrease image size Disclosure StatementNo potential conflict of interest was reported by the author(s).

On the Uselessness of the Distinction between Ideal and Non-Ideal Theory (at least in the Philosophy of Language)

Routledge eBooks · 2021 · 46 citations

• Philosophy
• Epistemology
• Linguistics

Herman Cappelen and Josh Dever consider whether we can distinguish between ideal and Non-Ideal Philosophy of Language in the way that a philosopher like Charles Mills does for political philosophy. They argue that there is no deep distinction between the sort of philosophy collected in this volume and the more mainstream material.

Terminology: Aboutness, Representation, and Metasemantics

2021-04-22

Abstract This short chapter does two things. First, it shows that in fact workers in AI frequently talk as if AI systems express contents. We present the argument that the complex nature of the actions and communications of AI systems, even if they are very different from the complex behaviours of human beings, and the way they have ‘aboutness’, strongly suggest a contentful interpretation of those actions and communications. It then introduces some philosophical terminology that captures various aspects of language use, such as the ones in the title, to better make clear what one is saying—philosophically speaking—when one claims AI systems communicate, and to provide a vocabulary for the next few chapters.