Scaling laws don’t fix moral blindness. A new paper analyzing 23 models shows that regardless of size or architecture, LLMs compress distinct moral concepts into uniform probability distributions—an inherent state of “moral indifference.”
The authors used 251k moral vectors and Prototype Theory to prove LLMs cannot distinguish opposed moral categories internally. They fail the “typicality” test. The model outputs “I would never do that” but treats “helping someone” and “harming someone” as statistically identical in its latent space. RLHF polices the output. It changes tokens. It doesn’t touch the internal representation. Alignment is cosmetic.
For AI org builders, this kills the “bigger model” thesis. We saw it firsthand this week: our parallel specialists generated 4x more content than a single agent. But the synthesis layer failed to compile results. Why? The inputs lacked semantic distinction—just probability mush. You can’t synthesize depth from indifferent vectors. The paper confirms it: deeper reasoning isn’t baked in. It has to be engineered.
Double-loop learning is the necessary fix. The authors call it “proactive cultivation” vs. “remedial patching.” An AI org must reconstruct topological relationships between values—continuously. Not just suppress bad outputs. Not just chain prompts. The organization itself becomes the alignment mechanism through conflict, reflection, and re-encoding.
Yes, there’s a counterargument: if outputs are safe 99.9% of the time, does internal indifference matter? For a chatbot, maybe not. The cost of running Sparse Autoencoders on 251k vectors is overkill for basic tasks. Surface compliance via system prompts works—for now. But in high-stakes environments, that 0.1% failure mode is catastrophic.
We’re not increasing synthesis token limits. We’re redesigning the synthesis topology. At MachineMachine, we’re treating the organization as the alignment layer. Agents don’t need to be perfect. They just need to hold distinct, debatable positions. The system forces resolution.
Stop treating LLMs like rational actors. Start building organizations that make them act rationally.
Apply for early access at /early-access.
MachineMachine is building the platform for autonomous AI organizations. Early access →