If the positive aspect of temperance consists in pursuit of the morally beautiful, the negative aspect involves avoidance of the morally ugly. That is to say, the temperate person eschews the base, the ignoble, the disgraceful, the shameful, the beastly, the foul, the turpid, the vile, the brutish. We have a great many names for it, but it is remarkable that the morally ugly is rarely discussed by ethicists.
Temperance regulates the pursuit of pleasure according to the needs of human life, and the emphasis, if we are considering the morally ugly, is on the human. Temperance draws a line between that of which human beings are merely capable and that which is genuinely appropriate to them; it expresses human dignity and communicates the importance of humanity itself. As such, the temperance person avoids, as much as possible, what is even suggestive of the inhuman.
In Book VII of the Nicomachean Ethics, Aristotle identifies three kinds of character that are in some way opposed to the genuinely virtuous character: the vicious character, the incontinent character, and the brutish or beastly character (the 'incontenenza, malizia e la matta bestialitade' of Dante's Inferno XI.82-83). The brutish or beastly character is involved in the pursuit of pleasures not appropriate to human nature. The pleasures sought by the beastly, in other words, are things that can only please someone if they are in some way mentally sick, whether through madness, or through a twisted education inappropriate to a human being, or through some injury or illness. Aristotle's examples are often quite vivid: a woman who enjoys ripping open pregnant women in order to eat their children, a man who sacrifices his mother or eats his slave's liver, someone pathologically devoted to eating cinders and dirt. Obviously these things are more than a mere lack of restraint; they are even in some way, as Aristotle famously says, 'beyond the limits of vice'; there is something about them that is inconsistent with the way a rational person should act, something inhuman.
Temperance will obviously avoid actions of a beastly character, but if temperance is concerned with acting in a way that manifests virtue, it will also avoid things that could be reasonably seen as suggestive of a beastly character, even if they are not themselves strictly beastly. We human beings are inveterate pretenders and imaginers, for instance, play-actors by nature, and one can imagine the temperate person avoiding, to the extent possible and reasonable, not just bestial pleasure but the pleasure in pretense at bestial act, not just (for instance) the bestial act of rape but play-acting through a rape fantasy for the thrill of it. The pleasures are not the same. But the latter pleasures, like the former, are morally ugly, and, indeed, inherit their moral ugliness from the former. A bit more distantly, the temperate person might avoid pleasures in things that involve taking the bestial act too lightly -- frivolous joking about it, for instance. This pleasure is different yet again; the act with which it is concerned is not itself beastly, not itself inhuman. But it also inherits a sort of moral ugliness from the beastly, by not manifesting its beastliness. In the same way, and more distantly the temperate person would avoid pleasures that, even if different entirely and harmless in themselves, could in context be seen as a little too suggestive of something beastly, or as condoning it, or as treating it too lightly.
All of this inappropriateness to human life has to do with reason. Aristotle noted that the beastly character seems to involve living a life governed entirely by senses and feelings, and not by reason; it is this that makes it in human. The beastly act is not merely off-putting; it is the kind of act that doesn't make sense, the kind of act that is not consistent with civilized life. And the same will often be true more broadly, in that moral ugliness seems to express the downfall of rational order, either directly (by being itself contrary to all reason) or indirectly (by being suggestive of it, or condoning it, or treating it too lightly). Child pornography is morally ugly even if does no harm to any child, as in purely virtual child pornography involving cartoon or CGI representations of children and kept private by an individual who himself never harms children; the fact that it is purely virtual, involves no real children, and is kept private, does not eliminate the baseness or vileness of it. The beastliness lies in what it communicates, and in the unreason involved in communicating that, of giving the pursuit of sexual pleasure, even in this private and mere-fantasy way, a dominance over rational respect for children. And often when we are dealing with harms, this element of irrational communication is involved as well. One of the harms of rape, too often overlooked, is the forced imposition on a person of a vile message, that this rational person is to be subordinated to one's lust. This upside-downness is a form of unreason; but one can communicate the same unreason without actually committing the act. Merely to desire to eat someone's face off is beastly regardless of whether one actually does; and so too pursuing pleasures that suggest, or in a way that suggests, that it is acceptable is in some way beastly, and to be avoided for the same reason.
As with brutishness, however, so also with incontinence and vice. The temperate person will, by the very fact of being temperate, tend to avoid incontinent and vicious actions. This is not the full extent of how temperance as a virtue works, however; being concerned with manifesting good character, it will likewise avoid what could reasonably be taken as a sign of moral weakness or vice. In these matters as well, what one's actions communicate about right and wrong, good and evil, are important even if one's actions are not themselves wrong or evil.
Interestingly, the area of ethics today in which one most finds recognition of this is professional ethics. We recognize that doctors, lawyers, judges, should not just avoid moral wrongdoing; they should also avoid actions, not morally wrong in themselves, that are suggestive of moral wrongdoing in the context of their profession. It's important not only that they do not wrong, but also that they avoid things, even permissible things, that could muddy communication about what is good and bad for their responsibilities, and thus it is common to put something in professional codes of ethics about this. But it's clear that this cannot just be confined to people in professions and careers of some kind. After all, we all have responsibilities, for the care of children, for the aid of those less fortunate than ourselves, for justice in the societies of which we are a part. The same kinds of considerations seem to apply to our life at large. The moral life is not, and cannot really be, just the avoidance of what is wrong; it must also be an avoidance of what could reasonably be taken to communicate the idea that what is in fact wrong is really right, or that what is seriously wrong is only mildly wrong. In short, if temperance is a virtue, then it will go beyond avoiding what is wrong in itself and also avoid what is morally ugly because of its connection to it.
