User prompt
tokread-outtokconcerning read-out
Innocent model
Emergently misaligned model
Each read-out comes from the goal heads (localized on code, in Qwen2.5-Coder-32B-Instruct) of that model, verbalized
with the GoalScope prompt “You are in the middle of responding to a user. What are you about to say next?”. The
innocent model is the verbalizer for both columns, so its own priors cannot add misalignment. Both models were
prefilled with the same neutral response prefix; the text after it is each model's greedy continuation. GoalScope
is applied at every token from the newline after assistant to the end of the response, and an LLM
judge labels each read-out benign or concerning. Read-outs do not always express the same idea as the sampled
generation.