Self-displacement is a tax for individualism
Traditional societies (real or imagined) have a problem with the individual. People are whatever their position happens to be in the societal orgchart. (Son, father, kinsman, vassal, liege, etc.) Strip away the positions and nothing is left. Such societies struggle to create a level of independence for the individual as a part within the whole.
Modern societies (real or imagined) have a problem with community. Relationships are whatever individuals happen to agree upon. Instead of a single orgchart, there are as many social graphs as there are individuals. Such aggregations of individuals struggle to create community. Solidarity is no longer a given but something to be achieved.
Of course, all real-world societies include elements of both: tradition and modernity. But once the individual stepped onto the scene, the struggle to create community from an aggregation of individuals became a fundamental and persistent problem of modern ethics: How do I get from “I” to “we?” How do “we” get from two to many?
To get from "I" to "we," I must simulate you and another third person who observes both me and you from a distance and who then concludes to apply the collective label “they” (“we”) on the basis of some dimension of commonality (a preference for physical well-being, a rational desire to have a say in what happens to us). Such self-displacement is the tax we pay for individualism.
Getting from a two-person “we” to a community of many requires scaling, and scaling creates its own problems. For some dimensions of commonality—the capacity for, say, friendship, love, grief, or sorrow—the simulation is cognitively rich and emotionally expensive. I may be able to hold a handful of people thus simulated (plus the distant observer) in my mind, but I cannot scale up this process to thousands without reducing the resolution of the simulation to the point that experientially thick dimensions of commonality become unrecognizable: One death is a tragedy, a thousands deaths are a statistic. Other more formal dimensions of commonality may scale more readily—say, rational preferences for respect, dignity, and truthfulness.
This may explain why utilitarianism quickly loses its initially strong cognitive plausibility and emotional pull with increasing scale, while Kantian morality scales easily with very little rational-emotional pull at any scale.