Why can we not say something like this: In finite being, certain things are predicable of wholes that are not predicated of parts, yet in the case of infinite being there is no such disjunction between whole and parts. With finite being parts are in a relation of potency to the whole; in infinite being, each part is fully actualized in itself and in the whole. What can’t we say that the parts are eternally inseparable from the whole, and that the parts are actualized eternally in a whole that is actualized in its parts? What prevents us from talking about the equal ultimacy of parts and whole in infinite being? We’re going to introduce a finite/infinite distinction anyway; why can’t it run between “infinite composite/finite composite” rather than between “infinite simple/finite simple”?
(1) If there is nothing predicable of the whole that is not predicable of the parts, at all, then there is no way to distinguish wholes and parts, and therefore the claim that this is whole and that is part is unfounded.
(2) To be a part is by definition to be potential to some whole. Whether it is actualized is irrelevant; we aren't talking about change here but about whether all the parts considered are together potential-whole, even if that potential is actualized. If they have no potential to compose the whole, then either there are no parts, or there is no whole, or there is no composition. If there is no whole, though, there are no parts, and if there is no composition, well, noncompositeness is literally just another name for simplicity.
(3) Having parts is a limitation that is one of the things we recognized as constituting something as finite rather than infinite being. Parthood is an ordering relation; having parts is a kind of dependency; nothing about those parts always or necessarily existing changes the fact that the relation is one of dependency.
(4) Leithart is ultimately trying to suggest that these matters are matters of metaphysics. There are indeed metaphysical arguments on the point. But first and foremost one has to justify why one is talking about 'parts' and 'wholes' in the first place. Is it just a metaphor? Nothing about the doctrine of divine simplicity prevents that, as long as you recognize it is just a metaphor; although we would still need to know why we are using the metaphor. If it's not a metaphor, we need some reason to explain why we are calling them parts and wholes to begin with. Leithart suggests that the argument for divine simplicity makes the assumption that "there must be a close analogy between the part-whole relation of finite being and a similar relation in infinite being". But it doesn't make this assumption; the assumption, to the extent it is made, is made by the compositionist: he is proposing that there is a close enough analogy between the part-whole relation of finite being and a similar relation in infinite being that the latter can be called a part-whole relation. The defender of divine simplicity is denying this: there can be no "similar relation" in infinite being at all. There is no need for an assumption of a "close analogy" if you're denying that there is any similarity justifying the use of the same terms in both cases; you do need a "close analogy" -- at least, one close enough -- if you are going to say that both cases are genuinely cases of the part-whole relation.
There is no room for just making up a new construal of parts and wholes; they have to be actually parts and wholes, which means that they have to have certain basic features that makes it coherent and rational to call them 'parts' and 'wholes'. And Leithart cannot point to any account of parts and wholes in which (a) they are what a reasonable person would normally call parts and wholes but (b) they don't involve any kind of asymmetric ordering or dependency, formal or otherwise. None exists. Anyone who is using 'part' or 'whole' in a way that doesn't involve (b) is using them in a nonstandard way.
(5) But even if none of this were true, the question at hand is the doctrine of divine simplicity. Anyone who is going to reject this, or even question it, has to be considering what the doctrine itself is actually denying -- and it is not denying just any arbitrary meaning that can be attached to the words 'part' and 'whole', by Leithart or by anyone else. It is denying that God is composite in the way it is characterizing composition. Nothing else is even relevant.