Calah Alexander of "Barefoot and Pregnant" has an excellent series of posts on modesty in clothing:
Nuking the Modesty War, Part I: Control Yourself, Not Women
Nuking the Modesty War, Part II: Stop Treating Men Like Pigs
Nuking the Modesty War, Part III: Stop Thingifying People
I've always thought it one of the many signs of the corruption of the modern age that when we think about immodesty in clothing, we start babbling about lust; when our ancestors thought about immodesty in clothing, they worried about vainglory and greed. When Chaucer's Parson, for instance, goes off on an absurd rant about men and women's fashions, what is riling him up is not indecency in our sense but the fact that rich men and women are using their clothes not in a useful or practical way but as a way of treating themselves as superior to ordinary people. While Chaucer is partly making fun of the Parson, he is able to do so largely because it's an exaggerated version of a type of rant everyone would have been familiar with in his day, because immodest dress was associated in everyone's mind with vainglory.
We see this in Aquinas as well, despite the fact that Aquinas has a quite conservative view of the matter. Aquinas notes (ST 2-2.169) that modesty in dress is not a matter of what you wear but of the way you wear it; you are modest in dress precisely to that extent that you wear your clothes moderately. Then he goes on to identify ways in which you could fail to wear your clothes moderately:
(1) By deliberate violation of customary law. In any society there are things that are just taken by people in general to be completely unacceptable to wear under particular circumstances; and if you deliberately disregard this, this is one way of being immodest.
(2.1.1) By excessive attachment to clothes due to the pleasure taken in them that arises from vainglory. This is precisely the case I already mentioned.
(2.1.2) By excessive attachment to sensible things due to the pleasure taken in them that arises from comfort.
(2.1.3) By excessive attention to one's own attire in one's practical reasoning, arising from imprudence.
(2.2.1) By inadequate attention to clothes due to the pleasure of disregarding them that arises from self-indulgence.
(2.2.2) By inadequate attention to attire due to the pleasure of disregarding it that arises from vainglory.
(2.3) By the use of clothes with the deliberate intention of provoking lust. Note, incidentally, that it is the deliberate attempt to provoke the vice of lust, not sexual desire; Aquinas is quite explicit that there are circumstances in which it is entirely reasonable to dress sexually, as long as you still show some sense in doing so.
And note, in particular, that the issue of lust only enters into the picture when we are talking about the deliberate attempt to provoke it. In any case in which someone is not deliberately trying to provoke lust, immodesty only arises from vainglory, or self-indulgence, or imprudence.
The basic principles of modesty of attire are quite simple, actually, and are touched on (organized in a different way) in Calah's discussion:
 Modesty and immodesty are matters of character, and thus what one is deliberately doing in wearing them, not matters of what one wears. If we are using 'immodesty' in some sense that does not have to do with personal character, we are not using it to talk about a vice, and thus it is of at most secondary moral importance.
 Modesty as a virtue must have two extremes, involving excessive regard for clothes and deficient regard for clothes. If we are talking about some kind of 'modesty' that does not have two extremes, we are not talking about a virtue, and thus it is of at most secondary moral importance.
 The seriousness of different kinds of immodesty must be related to the seriousness of the vices they express; and thus the most serious kind of immodesty occurs when people are immodest as a way of expressing the most serious vice. This is vainglory. The temptation that really matters most when it comes to immodesty is the temptation to use clothes as a way of trying to make the situation all about you and how great you are. This is something that can be done with any kind of clothing.
 Another person's lust cannot pollute your character; the only way it is relevant to the question at all is if you are deliberately trying to provoke it. Anyone who says otherwise is like those people who think they have a right to shut down anyone's opinion just by insisting that other people could be offended by it; it is one thing to say that people should not deliberately try to be offensive, and another thing entirely that people should let their lives be controlled by people whose capacity to take offense knows no bound. That is forcing everyone to kowtow to vice, and it is necessarily unjust.
 Like every other virtue associated with temperance, modesty in attire consists chiefly in love of balance, in regarding levelheadedness as beautiful and acting accordingly, and secondarily in being ashamed to act in a way that is inappropriate to the kind of rational animal we are. It's all about reason and good sense, taking the circumstances you face and living in a way appropriate to them. And that means all the circumstances you face. Yes, other people's opinions are part of those circumstances, but so is the cost of clothing, so is what people usually wear, and more important than all of these is whether it is practical or useful for doing good things. Demanding that other people dress in ways that make it harder for them to do good things is irrational and vicious -- it's a sin against both prudence and justice -- and, ultimately, the primary authority on whether a certain type of clothing is necessary or most useful for the kind of thing being done is the person wearing it. To be sure, they may choose more or less wisely, more or less safely, but it is morally vicious to demand that people only use the safest means. (Such a demand is known as tutiorism or rigorism, and it violates prudence and justice. You are perfectly free to advise what you think is a morally safer course; it is morally wrong, however, to demand that people only follow what you think is the morally safer course, because there may be riskier courses that are nonetheless perfectly fine if done right, and you have no right to attack people for acting prudently merely because they could have acted more prudently.)
So if you
(1) are not deliberately trying to shock and scandalize people by the inappropriateness of your attire;
(2) are not deliberately using your clothing to treat yourself as superior to other people;
(3) don't make clothes a huge part of your life;
(4) try to wear clothes that make sense for what you are doing; and
(5) are not using clothes as a means for making other people do bad things
then, chances are, your clothing is perfectly modest, even if you happen to be wearing nothing but a short grass skirt or form-fitting spandex. Modesty is not a virtue concerned with the value of clothes; it is, like every other virtue associated with temperance, concerned with the value of people, yourself and others, as rational creatures capable of acting with good sense and proportion. And, of course, prudence and justice require that we tailor our assessments of other people's clothing accordingly.