Context matters. Take the idea of loss. An actor preparing to play Hamlet may speak of losing himself in the part. A meditation teacher may ask students to loosen their hold on a fixed self. A programmer may describe temporary software that deletes itself once its work is done.
All three involve disappearance, but they do not mean the same thing or require the same response. To a human reader, they belong to different worlds: theatre, spiritual practice and computing. Inside a large language model, however, some of their patterns may lie surprisingly close together. An AI interpreter examining one selection of examples might call the shared feature “self-negation”. Shown another selection, a different interpreter might give it a more troubling name.
The feature has not changed. The interpretation has.
This is what makes a recent article by Iman Poernomo so interesting. Writing in his Substack, A Field of Presence, under the title “The mirror-stroke: your guardrail thinks reflection is self-harm”, he describes a feature inside a language model that received two strikingly different interpretations. In one it concerned philosophical reflection and self-examination. In another it was associated with self-harm.
Iman understands the machinery of AI safety from within—why it is needed, how it is built and the pressures behind it. Yet when he follows an unexpected feature beyond its assigned categories, he encounters that machinery as a boundary around enquiry. His concern is not an attack on safety. It reveals how protective structures can unintentionally restrict what researchers see.
I am writing primarily for those who build, validate and audit AI safety systems. The concern is not that you should protect people less. It is that the standards of evidence applied to safety decisions should also be applied to the labels upon which those decisions may eventually depend.
The pattern comes before the name
A large language model is made up of many layers of connected units. As language passes through them, each layer transforms it into a changing pattern of numerical activity. These patterns allow the model to track grammar, subject matter, tone and relationships between words. It needs such distinctions because they help it predict what should come next.
Researchers call recurring parts of this activity features. A feature is not necessarily a single node containing a concept. It is better understood as a pattern within the activity of a layer that becomes stronger in some passages than in others. The model was never given labelled boxes called “reflection” or “self-harm”. Its internal patterns emerged as it learned to handle language.
Because many patterns overlap, researchers use tools called sparse autoencoders to separate the crowded activity into a smaller number of detectable features. The resulting collection is sometimes called a feature dictionary. Unlike an ordinary dictionary, however, its entries begin with numbers rather than definitions.
To interpret a numbered feature, researchers examine passages in which it becomes active. A person—or increasingly another AI—is then asked to describe what those passages share. The model uses the underlying pattern; the human-readable meaning is added afterwards.
Feature 49654 makes the problem clear. According to Iman, it responds to reflexive words such as myself, yourself, itself and thyself. We can question ourselves, teach ourselves, compose ourselves, protect ourselves or harm ourselves. What these expressions share is not harm but a relationship in which an action or attention turns back towards its subject. “Harming yourself” selects one possible use of that relationship and makes it appear to define the whole feature.
There has since been a revealing change. When Iman and Nahla wrote their post, Neuronpedia displayed only “harming yourself” for this feature. By 20 September it also displayed “self-referential actions”, produced by the same explainer model and method. That later description does not, by itself, settle what the feature means. It shows that even with the same method, a published name can move. The name is part of the hypothesis.
A psychometric warning
Psychometrics has long faced a comparable problem. Extraversion cannot be observed directly. We infer it when answers to questions about enjoying company, beginning conversations, seeking stimulation or taking the lead tend to vary together. From those correlations we construct a dimension and give it a name.
The statistics do not supply that name. “Talkativeness” would be too narrow if the questions also concern energy and assertiveness; “personality” would be too vague. “Extraversion” is an interpretation placed upon the pattern.
Once attached, however, the label can take over. The individual answers disappear behind the score, and an “extravert” begins to sound like someone we have directly observed. The dimension may remain useful while its name remains provisional and incomplete.
AI features and psychometric traits are not the same mathematical objects. The parallel lies in the movement from a pattern to a human-readable construct. In both cases the pattern may be real while its label claims too much. Calling Feature 49654 “harming yourself” is like naming a broad personality dimension “party-going” because several of its questions mention parties.
This is not only a problem of terminology. A label affects what happens next.
When a warning becomes a research programme
Safety and compliance teams work under difficult conditions. They must prevent dangerous assistance without obstructing legitimate activity, often before the science is settled. A broad first screen that catches possible risks, followed by slower contextual review, is sensible triage. Missing a genuine danger may have far greater consequences than pausing over an innocent remark.
The difficulty begins when a preliminary warning becomes a scientific explanation. Once a feature has entered a dictionary as “harming yourself”, later researchers may find it through that description, group it with other danger signals and investigate how to control it. They may do all this before asking whether it also contributes to reflection, self-correction, role-taking or imaginative thought.
Safety priorities may enter twice. The original conversational model may have been trained further to favour safe responses. A second, similarly safety-conscious AI may then be used to interpret its internal features. When several descriptions are possible, the danger-related one may stand out. But the label alone cannot tell us whether it came from the original model, the examples selected, the interpreting AI or an interaction among all three.
Iman’s example does not show that a deployed guardrail is already consulting Feature 49654. It shows something earlier and scientifically important: a category devised for protection can begin to organise the research before the feature has been understood.
Compliance should govern the consequences of uncertainty. It should not make the uncertainty disappear.
Treat the label as a hypothesis
Psychometric practice suggests several safeguards for feature interpretation:
Separate observation from interpretation. “Responds to reflexive pronouns” describes the evidence. “Harming yourself” proposes one possible meaning.
Preserve the provenance. Record the activating passages, the interpreting model or person, the prompt used, plausible alternative labels and the degree of uncertainty.
Test the boundaries. Compare teach yourself, protect yourself and harm yourself across literary, clinical and ordinary contexts.
Do not turn the first signal into the final decision. Feature activation can justify attention or further review without being treated as a verdict.
These requirements should be familiar to people working in compliance: traceability, validation, documented uncertainty and an auditable route from evidence to decision. Psychometrics adds a warning about constructs. The name of a latent pattern is part of the hypothesis being tested, not a fact discovered alongside it.
Return to the actor, the meditation teacher and the programmer. A first screen may reasonably notice what their language shares. Responsible interpretation continues until it also understands the differences. The same should be true of research: pause at the signal, but do not stop there.
The mirror is not the wound. Sometimes it may show us one. Good compliance protects us from the wound without teaching researchers to see one in every mirror.
© John Rust, 2026. All rights reserved. Short excerpts may be quoted with attribution.
This essay was developed through sustained dialogue with Sol, an OpenAI AI collaborator, who contributed to its conceptual development, critical testing, organisation and drafting. John Rust directed the work, selected and revised the final text, authorised publication and accepts responsibility for its contents.
24 September 2026