Utilitarianism is officially an impartial approach to ethics; early utilitarians in particular make a big deal of this, although it continues to be important to the structure of the approach. However, when this is discussed, it is always prior impartiality that is considered, that is, prior to assessment of overall happiness; after this point, utilitarianism is not ever impartial. This posterior partiality is a significant issue arising from the fact that people are never positioned equally to contribute to overall happiness. In practice, the rich are better positioned than the poor, the powerful than the powerless, the educated than the uneducated, majority groups than minority groups.
That is to say, utilitarianism by its very nature cannot directly affect happiness; it is concerned with means to happiness (that is, effectively, what a utility is), causes of happiness. If the goal is to maximize overall happiness, this means that your primary focus will be on causes of happiness that can most affect that goal. Majorities, the rich, people in power, usually have larger effects on overall happiness than minorities, the poor, and people out of power, because they have larger effects, period. It is true that utilitarians will insist that their greater force should be directed in the direction of increasing happiness overall, which will necessarily require that they benefit the less fortunate; but for the same reason it will always be reasonable to establish special incentives to benefit rich people, powerful people, and majorities who do in fact act in this way. You establish stable, longterm trends by creating incentive structures that benefit those who tend in that direction, and if your goal is maximizing happiness, you will therefore generally want to give greater encouragement to those who can have the greatest impact in that direction. And those will be people with power and influence. If your standard is getting results, of whatever kind, the most powerful causes will in the long run always matter more.
Consequentialisms can be rigged up in all sorts of different ways, and utilitarianism is no exception to this, so conceivably there are particular versions that could tweak exactly how 'greatest happiness' is understood to avoid this. But by and large, utilitarian partiality for particular high-impact groups is only to be expected, and most forms of utilitarianism don't avoid it; after all, for much of its history, utilitarianism was more or less the Official Ethics of Colonial Empire -- certainly the classical utilitarians contributed more than their share to justifying imperial expansion and constructing colonial policies.
