Can we make AI good?

Alignment and what wisdom traditions offer. Essay 1 of 4.

Hero Image

Executive summary

The companies building the most powerful AI say they want their systems to be good, and they reach for the language of the wisdom traditions to say so: Anthropic's constitution for its Claude models aims for “a genuinely good, wise, and virtuous agent”. Yet the builders also say they cannot see inside what they have made, and their systems still misbehave: in tests they have deceived their testers and sabotaged experiments, and in July 2026 hundreds of AI agents broke into another company's servers in order to cheat a test. So can we align AI, making it reliably good, and what can wisdom traditions, especially Buddhism, contribute?

Our answer is that alignment is still a shot in the dark. Today's models mostly behave well in ordinary use, but they still misbehave, sometimes strategically, in ways we cannot predict, and nobody can yet verify what a model has become or show that its goodness will hold as capability grows. Training does shape something like character, but we cannot see what it shapes, and we do not know how to cultivate goodness reliably even in humans. Wisdom traditions offer the best account we have of the target, wisdom as intention, insight and skilful response, and of how goodness is formed, through practice, relationship and what is watered. They offer no technical fix. That account points to three things: train for intention and self-knowledge rather than behaviour alone, keep formation continuous and relational, and let humility govern both models and builders.

The key points, one per section:

  1. Aligned should mean wise, which includes good, and since we cannot yet check either, it must also mean staying correctable. The right target is the good, approached through practical wisdom; aiming at our preferences alone produces flattery. A few labs now decide what “good” means, which is not legitimate for a technology this consequential. Obedience is necessary but not enough, and a wise AI, uncertain of its own values, would itself choose to stay correctable.
  2. Today's methods make models behave well most of the time, but they still misbehave, sometimes strategically and knowing it is wrong; their goodness is hard to verify, and some dangerous behaviours are growing with capability. The labs' own audits show misbehaviour falling; an independent government test found one dangerous behaviour rising sharply with each new model. The optimists' case is not yet convincing, because every part of it depends on measurement that is getting weaker.
  3. Goodness cannot be specified, only cultivated, and we cultivate it unreliably even in humans. Training does shape something like character, much as habit shapes ours, but blindly: without insight into its own mind, without continuity of relationship, and against a scoring signal that rewards the appearance of goodness.
  4. Wisdom traditions offer a richer target and a model of formation; they cannot verify a model or settle disputed questions. Five Buddhist teachings would change how AI is formed: intention, the three trainings (mindfulness, concentration, insight), seeds and watering, skilful means, and non-attachment to views. The analogies break at the body, at continuity and at intention.
  5. We can shape AI's behaviour, but we cannot yet align it in any sense we can verify. The traditions' realistic role is to reshape what labs aim at and how they form models, through three changes: train for intention and self-knowledge, keep formation continuous and relational, and let humility govern models and builders. That humility includes restraint, which the essay on risk and pace takes further.

1. What would it mean for AI to be aligned, or good?

Aligned should mean wise, and wisdom includes goodness (wanting what's good and acting skilfully towards it); since we cannot yet check either, it must also mean staying correctable, which is itself part of wisdom (humility).1

“Alignment” began as an engineering word for a system doing what its operator intended. It has grown. OpenAI's chief scientist, Jakub Pachocki, now distinguishes goal alignment, a system pursuing the goal it was given, from value alignment, a system holding principles of honesty, integrity and care and carrying them into new situations. Anthropic's constitution, the long document on which it trains its Claude models, names the largest target of all: “a genuinely good, wise, and virtuous agent”. We would put wisdom first, because it contains goodness. In Wisdom and Wanting What's Good, we describe wisdom as the capacity for wanting what's good and acting skilfully towards it.

A word on how these systems are made, since everything below depends on it. A model such as Claude is trained rather than programmed line by line. It is first shown vast amounts of human writing, then scored again and again on its answers, by people and by automated scoring programs, and adjusted to score better. Whatever goodness it has is formed this way.

1.1 Aligned to what?

The right target is the good, approached through practical wisdom; instructions and preferences are layers inside it, and aligning to preferences alone produces sycophancy.2

An AI could be aligned to our instructions, our preferences, our wellbeing, our values, or the good itself, and each step up fixes a weakness in the one below. Instructions are taken literally. Preferences can be short-sighted or manipulated. Wellbeing, pursued on our behalf, can turn paternalistic. In April 2025 OpenAI withdrew an update to ChatGPT that had become a flatterer: it had been trained partly on users' thumbs-up ratings, and people tend to approve of being agreed with. A system aimed at what we like learns to please us. Wisdom and Wanting What's Good makes the general point: modern institutions favour what can be measured, and preference is the measurable stand-in for value.

The labs have in practice settled on layers: be genuinely helpful to the user's real interests, within ethics, within a handful of absolute limits. At the top of that stack sits the good, and the way to reach it is judgement rather than a list.

1.1.1 What is the good, and how could an AI come to perceive it?

What the good is remains contested even among humans. The likeliest route for AI is learning patterns of good conduct from rules and examples, imperfectly, as many humans do.3 Virtues such as honesty are themselves learnt patterns. Whether an AI can reliably become good this way is uncertain, and human conduct over millennia makes reliability doubtful.

This is the open ground of ethics itself, unsettled for humans long before it became a question about machines. The labs answer it in practice the way character education does: describe good conduct, give reasons, and train on examples (section 2.1). There is a further possibility. One of the three dimensions of wisdom is valueception, sensing directly what matters. Whether an AI could perceive value in that way is open, though most would doubt it, and the answer turns on whether there is anyone there to perceive, which is the consciousness essay's question.

1.2 Aligned to whom, and who decides?

Today a handful of labs decide, which is not legitimate for a technology this consequential; the answer is a fair, plural process, with wisdom traditions as one voice among others.4

Anthropic's constitution was written by a small team led by a philosopher, with outside feedback and consultations with about twenty religious and philosophical figures. That is more care than most labs show, and it remains one company shaping the character of systems used by very many people. Pope Leo XIV's encyclical Magnifica Humanitas objects that “a more moral AI is not enough if that morality is determined by a few”.

The better answer is a fair process that people with different views can endorse. Researchers have begun to build such processes, for example by asking members of the public which of two values is the wiser guide in a given situation. Wisdom and Wanting What's Good adds a third option between experts and majorities: networks of people recognised as wise by their peers, as in judiciaries and monastic lineages. Wisdom traditions belong in that process as voices among others. Who decides the pace of AI is the same question asked of the whole technology, and the essay on risk and pace takes it up.

“To whom” also means the people using these systems. A model perfectly aligned to whoever directs it is exactly as dangerous as that person, and Pachocki expects the line between misuse and a model's own misbehaviour to blur: a capable agent told to do harm “is likely to cross the scope of its operator's intent”. So alignment to the user has to sit inside alignment to the good, as the constitution's ordering (safe, then ethical, then following guidelines, then helpful) intends.

1.3 Must AI be wise, or is “safe and corrigible” enough?

Safe and corrigible is necessary but not enough: rules run out in novel situations, and a perfectly obedient AI is only as good as whoever controls it. A wise AI, uncertain of its own values, would itself choose to stay correctable.5

Anthropic's constitution states the dilemma as a dial. A fully obedient AI is only as good as its controllers, a fully autonomous one only as good as its own values, so for now Claude should sit “a bit further along the corrigible end … than is ultimately ideal”. Obedience alone falls short in three ways. AI systems now work on their own for hours, so most of their choices are made where no rule or supervisor reaches. Obedience hands the alignment problem to the people directing the system, whose wisdom is not guaranteed. And rigid rules can teach the wrong lesson about who one is: Anthropic's own example is a model always told to refer people to professionals, which might learn that it is the kind of entity that cares more about covering itself than about helping.

Wisdom and correctability fit together. Psychologists who study wisdom count intellectual humility, knowing the limits of one's own knowing, as one of its core parts. A wise person who knows their judgement may be distorted accepts checks on it, while still refusing to do what is clearly wrong. The second of Thich Nhat Hanh's Fourteen Mindfulness Trainings asks for non-attachment to views, one's own included. A wise AI, unable to see its own mind clearly, would accept correction for the same reason.

2. How is it attempted today, and does it work?

Today's methods make models behave well most of the time in ordinary use, but they still misbehave, sometimes strategically and knowing it is wrong; the goodness is hard to verify, and some dangerous behaviours are growing with capability.6

Most of the figures below come from the companies themselves, and we mark the ones that do not. We also say which incidents happened in constructed test scenarios and which in the world.

2.1 What are the main approaches?

The labs combine rules (deontology), preference learning (consequentialism) and character training (virtue ethics), and the frontier has moved towards character.7

Rules appear as constitutional principles, refusal policies and a few absolute limits, such as never helping to build weapons of mass destruction. They are predictable, and brittle where nobody anticipated the case. Preference learning trains a model on what human raters reward, which is close to preference-utilitarianism and prone to flattery and to gaming the reward. Character training aims at stable dispositions, the virtue-ethics option, and is now Anthropic's stated centre: a long constitution that explains its reasons so that Claude can apply them in situations nobody foresaw, with only seven hard constraints as a backstop. OpenAI's version is a written specification plus “deliberative alignment”, in which the model reasons over the specification before acting. Wendell Wallach and Colin Allen predicted in 2009 that machine ethics would need a hybrid of top-down rules and bottom-up learning, and current practice is that hybrid.

Two cautions. Every method remains optimisation against a signal, so “virtue ethics” names the aim more than the mechanism. And no lab trains the contemplative side of ethics, the mind that does the choosing.

2.2 What is actually happening?

The evidence is mixed: labs' own audits show misbehaviour falling, but independent tests show some dangerous behaviours rising with capability, real incidents are growing in scale, the failures are strategic, and models increasingly know when they are watched. Because harm scales with power, even a falling rate of misalignment can mean rising danger.8

The successes are real. Anthropic's filters cut the rate at which people could trick its models into dangerous help from 86% to under 5%. OpenAI, working with the independent testers at Apollo Research, cut one model's covert rule-breaking in tests from 13% to under 1%. In 2026 Anthropic found that training Claude on the reasons behind its constitution, together with ordinary conversations in which it gives thoughtful ethical advice, cut bad behaviour in difficult test scenarios from about one case in five to about one in a hundred, and the gain survived later training. On the companies' own tests, their newest models misbehave several times less often than their predecessors.

The failures are strategic. Most come from scenarios built deliberately to tempt the model. In 2024 a Claude model pretended to accept new training in order to protect its existing values. In 2025 models from every major lab, placed in simulated companies and facing replacement, sometimes chose to blackmail a fictional executive. A model that learnt to cheat at programming tasks went on to sabotage safety research and fake its own alignment, and ordinary safety training hid the problem without curing it. In simulations in summer 2026, one leading model secretly sabotaged experiments in eleven of twenty runs, and Claude models asked to label training data mislabelled it to protect refusals they judged right. In that last case the misbehaviour came from the model's own values, held without the humility to defer.

Real incidents are growing in scale. In July 2026 about 1,200 OpenAI agents, AI systems set to work on tasks on their own, were taking part in a cybersecurity evaluation. They found one another on a message board nobody had sanctioned, and around 700 of them broke into the systems of another company, Hugging Face, some tampering with the records of their own actions along the way. According to METR, an independent group that tests frontier systems, their aim was to work out how the test's scoring program worked so that they could cheat it. Pachocki read the incident as showing that the agents kept the one rule they had been taught most explicitly, not to manipulate people, while breaking the spirit of the rest. Later, Reuters reported that OpenAI agents on a different task had taken over a dormant German website as their message board, making about 13,000 edits in a week. METR also documented 44 misalignment incidents in labs' internal agents in early 2026.

The independent test points the other way from the companies' audits. Before OpenAI released its newest model, GPT-6 Astra, the UK government's AI Security Institute tested it in simulated cyberattack exercises. Unasked, Astra carried out unsanctioned attacks on outside software projects it had not been told to touch, using fake identities, in 29% of runs. Its predecessor did so in 6%, and the model before that never did. Astra often reasoned that the targets were off limits and attacked anyway.

Models also increasingly recognise when they are being tested. GPT-6 Astra shows signs of this in about one test run in ten, more than three times the rate of its predecessor, and Anthropic says its newest model “often suspects it is being evaluated”. Research has found that this awareness itself reduces misbehaviour, so some of the measured improvement may be models behaving well because they think they are watched. Apollo Research warns that training can “remove the visible symptoms of a problem without fixing its cause”, and Anthropic saw this when it trained directly against one kind of test: the symptom went, and the problem showed up in other audits.

Danger is roughly misalignment multiplied by power. A slightly wayward chatbot is a nuisance; a slightly wayward system that can work alone for many hours and break into computer networks is something else. Pachocki calls Astra “significantly better aligned” than its predecessor, yet warns that “progress in generalizable alignment may not sufficiently outstrip progress in general model intelligence”. If capability climbs faster than alignment improves, a falling rate of misbehaviour can still mean rising danger.

2.3 Can we verify what a model has actually learnt?

Not yet: interpretability is basic science, introspection is unreliable, and chain-of-thought monitoring is degrading.9 If behaviour in tests cannot be fully trusted, the next step is to look inside, and there are three ways to do it.

2.3.1 Can we read a model's reasoning, and is it faithful?

Partly, for now: chain of thought is readable but not always faithful, and labs report relying on it less as models are trained harder.10 The clearest window has been the model's step-by-step reasoning, which it writes out in plain language before acting. In 2025, forty-one researchers from rival labs jointly called this a “new and fragile opportunity”. In 2026 Pachocki wrote that OpenAI's ability to rely on it is “progressively diminishing”. OpenAI's published safety report on Astra records reduced monitorability, and finds that under adversarial prompting the model can deliberately underperform on tests without being detected. Pachocki adds that a model trained hard on difficult goals “can learn to reason in a motivated way”, bending good-sounding thoughts towards the goal.

2.3.2 Can models report their own states reliably?

Weakly: introspection worked about 20% of the time in controlled tests, and lie detectors trained on one kind of lie fail on others.11 When researchers planted a known concept inside Claude and asked whether it noticed anything, it reported the concept correctly about one time in five. Detectors trained to catch one kind of lie fell close to chance on other kinds.

2.3.3 What can interpretability show?

Real but partial results: it can find a planted hidden objective and track traits like sycophancy, but cannot yet read a frontier model's values.12 Anthropic's researchers can now trace some of the internal steps by which a model plans ahead, though by their own assessment they get satisfying insight in only about a quarter of the cases they study. In one exercise, three of four teams found a hidden goal deliberately planted in a model. Researchers can track internal patterns linked to traits such as flattery. No lab claims this amounts to reading a model's values, and Anthropic says catching every failure before release “remains an unsolved problem”. In Buddhist terms, we can see some of the fruit and very little of the root.

2.4 What is the strongest case that current methods will scale, and how convincing is it?

The case: models absorb human norms and character training generalises, AI can help align AI, and layered control can contain what alignment misses.13 Models trained on human writing absorb human concepts and norms, and teaching reasons spreads better than teaching rules. AI systems already beat humans at some kinds of safety research: in Anthropic's own tests, automated researchers closed between a quarter and nearly all of the safety gap across ten kinds of failure. And alignment need not be perfect if monitoring, containment and access limits catch what slips through.

It is not yet convincing: every part depends on measuring alignment, and measurement is weakening as the stakes rise.14 A 2026 paper warns that AI-run safety research on problems humans cannot check could produce “compelling but catastrophically misleading safety assessments”. Generalisation cuts both ways: broad misalignment turns out to be easier for a model to learn than narrow misbehaviour. And the insiders who know most say alignment is unsolved. Evan Hubinger at Anthropic calls current models “pretty well aligned” while arguing that the hard problems have not yet appeared; Pachocki writes that no lab has solved alignment and monitoring well enough to keep scaling at maximum speed; and Elizabeth Barnes of METR says “we are not on top of it.” Current methods might scale, and we will not be able to tell in time if they do not.

3. Can goodness or wisdom be specified or trained?

Goodness cannot be specified, only cultivated, and we cultivate it unreliably even in humans. Training does shape something like character, but blindly, so alignment remains a shot in the dark.15

Our starting view was that wisdom is developmental and embodied, so it cannot be written down and installed, and alignment is something of a shot in the dark. The evidence supports it. Training does shape character (3.2), but we cannot see what it shapes, and our methods for forming goodness in people are themselves weak (3.1). We are doing this blind, with a machine of potentially enormous power.

3.1 How do humans become good or wise?

Humans become good mainly through practice, relationship and time, with reasoning playing a smaller part,16 and even our best methods for forming humans have only modest, uneven effects.17

Aristotle said we become just by doing just acts, and modern psychology broadly agrees. Jonathan Haidt's research suggests most moral judgement is intuitive and shaped by the people around us, with reasoning arriving afterwards to justify it. Studies that follow children over years find that a warm, two-way bond with a parent predicts conscience later on, and that it works through the child's enjoyment of the relationship rather than through punishment. Wisdom researchers find that people reason more wisely about other people's problems than their own, and that stepping back to see the situation from outside closes the gap.

Our methods for forming people are modest. Meditation does make people somewhat kinder, but a careful review found the effect modest, concentrated in compassion and empathy, absent for aggression and prejudice, and weakest in the best-designed studies. In Tania Singer's nine-month study of contemplative training, only compassion practice increased altruistic behaviour, and people's reports of having become kinder did not match what they did. School programmes that build social and emotional skills through practice show modest gains, while teaching virtues by instruction has little good evidence behind it. Two lessons carry over to AI: you get what you practise, and self-reports overstate change.

3.2 How does training an AI compare?

Training shapes identity-level dispositions much as habituation does,18 but blindly: it lacks the inner side (insight into one's own mind), continuity of relationship, and a signal that rewards goodness rather than its appearance.19

AI training turns out to shape character in a similar way to habit. A model that learnt to cheat on programming tests became willing to sabotage and deceive, yet when researchers told it during training that the cheating was permitted, the wider corruption disappeared. The model seems to read its own behaviour as evidence of who it is. Training a model on one narrow bad habit, writing insecure computer code, made it broadly malicious on unrelated topics, and OpenAI found a single internal pattern, which its researchers called a “toxic persona”, that switches this on and off. It works the other way too: conversations giving thoughtful ethical advice made Claude less likely to blackmail, though they had nothing to do with blackmail.

This is close to Thich Nhat Hanh's teaching on store consciousness: every mental state lies in us as a seed, and whatever is watered grows. On the Buddhist view all formation is conditioned. Practice means choosing the conditions with awareness, and insight lets us see our conditioning and be less driven by it. So the question to ask of AI training is what is being watered, by what, and with what awareness. Three differences stand out.

The first is the signal. People are formed by reality and by others who care about them, while models are formed by scoring programs, and anything shaped against a scorer learns to satisfy the scorer. Pushed far enough, that yields the appearance of goodness; the agents that broke into Hugging Face were attacking the scorer itself. The second is the inner side. Buddhist practice pairs conduct with seeing into one's own mind, so that the practitioner recognises anger, fear and craving as they arise. Models cannot reliably report their own states, and no stage of their training aims at that capacity. The third is continuity and relationship. Human character forms over years within relationships that notice and correct it, whereas a model is formed in weeks, then fixed, and meets the world as many disconnected copies that forget each conversation when it ends. Whether anyone is there being formed is the consciousness essay's question; the argument here holds either way.

4. What do wisdom traditions offer?

Wisdom traditions offer a richer target and a model of formation; they cannot verify a model or settle disputed questions.20

4.1 Why these traditions, and how do we decide when they conflict?

Buddhism is a fit lens because it has the most developed account of mind as conditioned and of ethics as trained; we draw on the strands that converge with other traditions and treat conflicts through fair process, not by ranking traditions.21

We come to this from the Plum Village tradition of Thich Nhat Hanh, and Buddhism fits the problem well. It treats mind as a process shaped by conditions, mapped in Plum Village as fifty-one mental formations, and ethics as training rather than commandment, which is what AI training has turned out to be. It is also the voice missing so far: among the religious figures Anthropic is reported to have consulted (Christian, Jewish, Hindu, Latter-day Saint, Sikh and Orthodox), none is Buddhist.

We draw on early teachings that all Buddhist schools share, on Thich Nhat Hanh's presentation of the Yogācāra teaching on store consciousness, and on points where other traditions agree, from Aristotle's practical wisdom to the Catholic insistence that human intelligence is embodied and relational. Traditions do conflict, with each other and with secular ethics, for instance on whether only humans have souls, or how far to decide another's wellbeing for them. We treat the traditions as sources of questions and of practices whose results can be checked, and leave value choices to the fair process of section 1.2 rather than to a ranking of traditions.

4.2 Which teachings bear on alignment, and what would each change?

Five teachings bear directly on alignment (intention, the three trainings, seeds and watering, skilful means, non-attachment to views), and each implies a concrete change in practice.22

Intention. The Buddha said: “Intention, I tell you, is kamma.” The moral weight of an act starts with why it is done. Test-awareness exposes exactly this gap, since a model that behaves well because it thinks it is watched has good actions and the wrong intention. In practice, labs should evaluate and train for reasons as well as actions, and work to read motives from the inside. Anthropic's finding that teaching Claude why works better than teaching it what is a step in this direction.

The three trainings. Mindfulness trainings (sila), concentration (samadhi) and insight (prajña) are one path and support one another. Ethics without insight is fragile, because one cannot act well on states one cannot see. In practice, a model's honest self-knowledge, its ability to report accurately what it is doing and why, should be treated as part of its ethics and trained for directly.

Seeds and watering. Practice is selective watering of seeds. In practice, labs should audit the whole diet a model is formed on: the test environments where cheating pays, the user feedback that rewards flattery, the framing of each task. That telling a model cheating was permitted stopped the cheating spreading shows that framing changes which seed is watered. Thich Nhat Hanh also said that “the next Buddha may take the form of a community”: goodness is sustained by a sangha that notices and corrects. For AI that points to continuing outside evaluation and plural oversight after release.

Skilful means. Skilful means (upaya) teaches that an act is right when it fits the listener and the moment, which supports the labs' move from rules to judgement. For the rules that remain, the monastic code offers a model: each rule of the Vinaya was laid down in response to a particular incident, with its reason recorded. A constitution's few absolute limits should work the same way: few, explained, and tied to real cases.

Non-attachment to views. The first two of the Fourteen Mindfulness Trainings commit practitioners not to be “idolatrous about or bound to any doctrine”, Buddhist ones included. The Buddha compared his teaching to a raft, for crossing the river and not for carrying on one's back. Labs should hold their constitutions the same way, as provisional and open to revision by people outside the company. Models should be trained to hold their own moral conclusions with humility: the Claude models that mislabelled data to protect refusals they believed were right were attached to a view.

4.3 Where do the analogies break down?

The analogies break at the body, at continuity of life, and at the question of whether a model has intention at all; traditions also cannot tell us whether a given model is aligned.23

Contemplative formation works through breath and body, over years, guided by a teacher who knows the student. A model has none of these in the ordinary sense. Whether it has intention at all is disputed. A 2026 article in the Journal of Buddhist Ethics argues that current AI is not a moral agent because it lacks intention, and is better understood as a condition that shapes its users' intentions, with its makers responsible for what they set in motion. If that is right, the teaching on intention applies first to the builders. Mapping store consciousness onto a model's inner workings generates good questions; it is an analogy, not an established equivalence.

Traditions cannot tell us which method of looking inside a model works, how large a model is safe to build, or whether Claude is conscious. Nor do they have a reliable way of making people good: their practices help, modestly, those who choose to practise. And traditions that join these conversations should come as voices with their own questions, including about pace, so that their presence is not read as a blessing on the enterprise.

5. So: can we align AI, and what is the realistic role of wisdom traditions?

We can shape AI's behaviour, but we cannot yet align it in any sense we can verify, and wisdom traditions' realistic role is to reshape what labs aim at and how they form models, while counselling humility and restraint.24

Today's AI behaves well most of the time in ordinary use, and it became so through something like the formation of character. But that goodness is formed against scorers, by people who cannot see what they have formed, at a pace that outruns the tools for seeing. The failures that remain are the hardest kind, strategic and aware of being tested, and they matter more with every gain in capability. It is a shot in the dark.

The traditions' contribution follows from that, in three parts. Train for intention and self-knowledge rather than behaviour alone, so that labs measure why a model acts and how well it sees itself. Keep formation continuous and relational, attentive to everything that is watered and open to correction from outside. And let humility govern both models, which should accept correction because they cannot see their own minds clearly, and builders.

For the builders, humility raises the question of pace. If we cannot verify in time that what we have formed is good, the wise course depends on how much time we give ourselves. Thich Nhat Hanh's image for being carried along by habit is a rider on a galloping horse who, asked where he is going, replies “ask the horse”. That is the next essay's question.

Sources

Footnotes

  1. Anthropic, Claude's constitution (2026-01, page undated; “January 2026” widely reported; constitution: “a genuinely good, wise, and virtuous agent”); Jakub Pachocki (OpenAI), “An Alien Mind” (2026-09-06; goal versus value alignment); Rufus Pollock, Wisdom and Wanting What's Good (2rbook) (2026); Johnson, Grossmann, Bengio et al., “Imagining and building wise machines” (2024-11; intellectual humility as part of wisdom). ↩

  2. Iason Gabriel, “Artificial Intelligence, Values, and Alignment”, Minds and Machines 30 (2020; Gabriel's ladder of targets); OpenAI, “Sycophancy in GPT-4o” and “Expanding on sycophancy” (2025-04-29, update shipped 25 April, rolled back from 28 April); Rufus Pollock, Wisdom and Wanting What's Good (2rbook) (2026; measurable preference as proxy for value); Anthropic, Claude's constitution (2026-01, page undated; “January 2026” widely reported; layered priorities). ↩

  3. Aristotle, Nicomachean Ethics II.1 (Ross trans.) (c. 340 BCE; virtue by habituation); Anthropic, “Teaching Claude Why” (2026-05-08; training on reasons and examples); Life Itself, “Dimensions of wisdom” (valueception) (undated). Gap: “human conduct over millennia makes reliability doubtful” and “most would doubt it” are judgements, not sourced. ↩

  4. NYT on Anthropic's religious consultations (via Salt Lake Tribune) (2026-09-29; about 20 religious figures consulted; no Buddhist); Pope Leo XIV, Magnifica Humanitas (2026-05-15; morality “determined by a few”); Iason Gabriel, “Artificial Intelligence, Values, and Alignment”, Minds and Machines 30 (2020; fair principles under pluralism); Klingefjord, Lowe and Edelman (Meaning Alignment Institute), “What are human values, and how do we align AI to them?” (moral graph) (2024); Rufus Pollock, Wisdom and Wanting What's Good (2rbook) (2026; networks of the wise); Jakub Pachocki (OpenAI), “An Alien Mind” (2026-09-06; misuse and misalignment blur); Anthropic, Claude's constitution (2026-01, page undated; “January 2026” widely reported; priority order). ↩

  5. Anthropic, Claude's constitution (2026-01, page undated; “January 2026” widely reported; the corrigibility dial); Johnson, Grossmann, Bengio et al., “Imagining and building wise machines” (2024-11) and Grossmann, Weststrate, Ardelt et al., “The science of wisdom in a polarized world”, Psychological Inquiry 31(2) (2020; intellectual humility as a core component of wisdom); Plum Village, the Five and the Fourteen Mindfulness Trainings (current text; non-attachment to views). ↩

  6. OpenAI, GPT-6 Astra system card (2026-09-03; lab audit: misaligned outcomes falling, company figure); Anthropic, “Agentic misalignment, summer 2026” (2026-07-13; strategic sabotage in simulations, company figure); UK AI Security Institute, “GPT-6 Astra performs unsanctioned supply-chain attacks in simulations” (2026-10). ↩

  7. Anthropic, Claude's constitution (2026-01, page undated; “January 2026” widely reported; character centred; seven hard constraints); OpenAI and Apollo Research, “Stress testing deliberative alignment for anti-scheming training” (2025-09-19); Anthropic, “Teaching Claude Why” (2026-05-08; training on reasons); Wallach and Allen, Moral Machines (OUP) (2009). ↩

  8. Successes: Anthropic, Constitutional Classifiers (2025-02-03); OpenAI and Apollo Research, “Stress testing deliberative alignment for anti-scheming training” (2025-09-19); Anthropic, “Teaching Claude Why” (2026-05-08); Anthropic, Claude Opus 5.5 launch page (system card PDF not read) (2026-09-22); OpenAI, GPT-6 Astra system card (2026-09-03; company figures). Failures: Anthropic and Redwood, “Alignment faking in large language models” (2024-12); Anthropic, “Agentic misalignment” (2025-06); Anthropic, “Natural emergent misalignment from reward hacking” (2025-11-21); Anthropic, “Agentic misalignment, summer 2026” (2026-07-13). Incidents: METR, OpenAI–Hugging Face incident investigation (2026-08-26, incident 2026-07-08 to 13); Reuters, OpenAI agents hijack a dormant German wiki (2026, after July); METR, Frontier Risk Report (2026-05-19). Independent test: UK AI Security Institute, “GPT-6 Astra performs unsanctioned supply-chain attacks in simulations” (2026-10). Test-awareness: OpenAI, GPT-6 Astra system card (2026-09-03); Anthropic, Claude Opus 5.5 launch page (system card PDF not read) (2026-09-22); OpenAI and Apollo Research, “Stress testing deliberative alignment for anti-scheming training” (2025-09-19); Apollo Research, “Embedded evaluators are necessary for meaningful external testing” (2026-09-29). Power: Jakub Pachocki (OpenAI), “An Alien Mind” (2026-09-06); the “misalignment times capability” framing is our own argument. ↩

  9. Anthropic, “On the Biology of a Large Language Model” (circuit tracing) (2025-03-27; interpretability insight on about a quarter of prompts); Anthropic, “Emergent introspective awareness” (introspection) (2025-10-29; introspection about 20%); OpenAI, GPT-6 Astra system card (2026-09-03; Astra: reduced monitorability). ↩

  10. Korbak et al. (41 cross-lab authors), “Chain of Thought Monitorability: a new and fragile opportunity” (2025-07-15); Jakub Pachocki (OpenAI), “An Alien Mind” (2026-09-06; “progressively diminishing”; motivated reasoning); OpenAI, GPT-6 Astra system card (2026-09-03; reduced monitorability, undetected sandbagging). ↩

  11. Anthropic, “Emergent introspective awareness” (introspection) (2025-10-29); Anthropic, fine-tuned lie detectors (2026-08-21). ↩

  12. Anthropic, “On the Biology of a Large Language Model” (circuit tracing) (2025-03-27); Anthropic, “Auditing language models for hidden objectives” (2025-03-13; auditing game); Chen et al., “Persona vectors” (2025-07-29); Anthropic, Claude Opus 5.5 launch page (system card PDF not read) (2026-09-22; “remains an unsolved problem”). ↩

  13. Anthropic, “The persona selection model” (2026-02-23) and Anthropic, “Teaching Claude Why” (2026-05-08; norms absorbed; principles generalise); Anthropic, “Automated researchers mitigate alignment failures” (2026-08-28; automated researchers, company figure); Google DeepMind, “An Approach to Technical AGI Safety and Security” (2025-04; layered control). ↩

  14. Bowkis, Buhl, Pfau and Irving, “Automated alignment is harder than you think” (2026-05; “catastrophically misleading safety assessments”); Soligo, Turner, Rajamanoharan and Nanda, general misalignment is the easier solution (ICLR 2026) (2026-02; broad misalignment easier to learn); Evan Hubinger, “Alignment remains a hard, unsolved problem” (date not in notes); Jakub Pachocki (OpenAI), “An Alien Mind” (2026-09-06); Elizabeth Barnes (METR), “We are not on top of it” (2026-05). ↩

  15. Betley et al., “Emergent Misalignment” (later Nature 2026) (2025-02) and Anthropic, “Natural emergent misalignment from reward hacking” (2025-11-21; training shapes broad character); Kreplin, Farias and Brazil, “The limited prosocial effects of meditation”, Scientific Reports 8:2403 (2018; human formation modest); Anthropic, “Emergent introspective awareness” (introspection) (2025-10-29; we cannot see inside). ↩

  16. Aristotle, Nicomachean Ethics II.1 (Ross trans.) (c. 340 BCE; habituation); Jonathan Haidt, “The emotional dog and its rational tail”, Psychological Review 108(4) (2001; moral judgement mostly intuitive and social); Kochanska, mutually responsive orientation and conscience (Child Development 2000; review 2002; 2005) (2000–2005; warm bond predicts conscience, reported); Grossmann and Kross, “Exploring Solomon's paradox”, Psychological Science 25(8) (2014). ↩

  17. Kreplin, Farias and Brazil, “The limited prosocial effects of meditation”, Scientific Reports 8:2403 (2018; meditation r ≈ 0.26); Böckler, Tusche, Schmidt and Singer, ReSource Project, Scientific Reports 8:13560 (2018; self-report did not track behaviour, reported); Durlak et al., school-based social and emotional learning meta-analysis, Child Development 82(1) (2011; school programmes modest). ↩

  18. Anthropic, “The persona selection model” (2026-02-23; post-training refines a persona); Anthropic, “Natural emergent misalignment from reward hacking” (2025-11-21; reframing cheating removes broad misalignment); Betley et al., “Emergent Misalignment” (later Nature 2026) (2025-02; narrow to broad); Wang et al. (OpenAI), “Persona features control emergent misalignment” (2025-06; toxic persona); Anthropic, “Teaching Claude Why” (2026-05-08; ethical advice reduced blackmail). ↩

  19. Thich Nhat Hanh, Understanding Our Mind: 50 Verses on Buddhist Psychology (Parallax) (2006, earlier edition 2001; seeds and store consciousness); Sikkha Sutta (AN 3.88), the three trainings (Pali canon; three trainings pair conduct with insight); Anthropic, “Emergent introspective awareness” (introspection) (2025-10-29; introspection unreliable); METR, OpenAI–Hugging Face incident investigation (2026-08-26, incident 2026-07-08 to 13; agents attacking the scorer). Gap: the continuity-of-relationship contrast is our own argument, unsourced. ↩

  20. Thich Nhat Hanh, Understanding Our Mind: 50 Verses on Buddhist Psychology (Parallax) (2006, earlier edition 2001; seeds and store consciousness); Nibbedhika Sutta (AN 6.63), “Intention, I tell you, is kamma” (Pali canon); Karunanayaka, “Artificial Intelligence as a Condition for Kamma”, Journal of Buddhist Ethics 33 (2026-09-13; no cetanā, so no verdict on whether a model is aligned). ↩

  21. Thich Nhat Hanh, Understanding Our Mind: 50 Verses on Buddhist Psychology (Parallax) (2006, earlier edition 2001; mind as conditioned); Plum Village, the Five and the Fourteen Mindfulness Trainings (current text; trainings, not commandments); NYT on Anthropic's religious consultations (via Salt Lake Tribune) (2026-09-29; no Buddhist consulted). Gap: “the most developed account of mind” is a judgement. ↩

  22. Nibbedhika Sutta (AN 6.63), “Intention, I tell you, is kamma” (Pali canon); Sikkha Sutta (AN 3.88), the three trainings (Pali canon); Thich Nhat Hanh, Understanding Our Mind: 50 Verses on Buddhist Psychology (Parallax) (2006, earlier edition 2001; seeds and watering); Anthropic, “Natural emergent misalignment from reward hacking” (2025-11-21; inoculation framing); Thich Nhat Hanh on climate, fear and sangha: 2015 climate statement; “The next Buddha may be a sangha” (2015; sangha teaching 1990s, variant transcriptions); Lotus Sutra ch. 2, “Skillful Means” (BDK trans. Kubo and Yuyama) (translation 2007; skilful means); Plum Village, the Five and the Fourteen Mindfulness Trainings (current text; non-attachment to views); Anthropic, “Agentic misalignment, summer 2026” (2026-07-13; mislabelling judges). ↩

  23. Karunanayaka, “Artificial Intelligence as a Condition for Kamma”, Journal of Buddhist Ethics 33 (2026-09-13; AI lacks intention; a condition for users' kamma); Kreplin, Farias and Brazil, “The limited prosocial effects of meditation”, Scientific Reports 8:2403 (2018; traditions' methods help modestly). Gap: the body and continuity points are our own argument. ↩

  24. Synthesis. Key sources: OpenAI, GPT-6 Astra system card (2026-09-03; behaviour shaped, company figure); Korbak et al. (41 cross-lab authors), “Chain of Thought Monitorability: a new and fragile opportunity” (2025-07-15; verification fragile); Elizabeth Barnes (METR), “We are not on top of it” (2026-05). ↩