It's important to keep in mind what Duhem insists vehemently upon, particularly since people don't always do so when talking about Duhem: he is only talking about theory in physics. Duhem, of course, was a thermodynamicist, and in his scientific heyday at the tail-end of the nineteenth century was considered one of France's best minds in physics; his philosophy of science grows, in great measure, out of his work in physics and his teaching of physics and his study of the history of physics. So none of his argument is intended to apply to any field but physics; Duhem recognizes that not everything he says about physics will turn out to be true about other fields. This is in part because a lot of what he says about physics depends crucially on the fact that physics is a highly mathematical discipline in a way that most other fields are not.
Carefully restricting his discussion to physics, Duhem opens with the question: What is the object or aim of physical theory? He proposes two possible answers to this question. The first possible answer, and the one that seems to leap to mind most readily, is that physical theories are there to explain the phenomena. The second possible answer, the one that Duhem accepts, is that physical theories are there to summarize and classify the phenomena. At stake is whether the notions involved in the propositions identify fundamental features of things that underly phenomena, or represent general characteristics of the phenomena themselves. In order to accept the former, however, we must accept the following affirmation: Under the sensible phenomena there is a reality distinct from those phenomena. Further, even with this affirmation we don't get a view of physical theories in which they explain phenomena; we must add to it an answer to the question, "What is the nature of the elements that constituted material reality?" Now it's important to point out that neither of these claims derive from physical theory or experiment themselves; if physical theory is explanatory, both of these claims must precede any physical theory that can be constructed, and neither of them admit of non-theoretical experimental justification. From this Duhem draws the obvious conclusion: If physical theories explain, physics is dependent on metaphysics, and the physics will differ as the metaphysics differs. To this Duhem responds:
Now, to make physical theories depend on metaphysics is surely not the way to let them enjoy the privilege of universal consent. In fact, no philosopher, no matter how confident he may be in the value of the methods used in dealing with metaphysical problems can dispute the following empirical truth: Consider in reveiw all the domains of man's intellectual activity; none of the systems of thought arising in different eras or the contemporary systems born of different schools will appear more profoundly distinct, more sharply separated, more violently opposed to one another, than those in the field of metaphysics.
While Duhem is a positivist, he is not a logical positivist; he has considerable respect for metaphysics, and as we will see, even with regard to science itself he thinks metaphysics has an important role to play. But if the interpretation of our physical theories depends on our metaphysical views, we seem to be faced with a serious problem for physics, because however important metaphysics is, the straightforward fact is that there's no general agreement about it.
Moreover, no metaphysics of itself suffices to build a physical theory, and this, too, causes problem for the thesis that physical theories explain. If physical theories depend on a prior metaphysics for their explanatory value, then they only have explanatory value to the extent that can be fit into that prior metaphysical framework. However, there will always be a looseness of fit between metaphysics and physics, no physics changes over time and metaphysics never gives sufficiently precise instructions for the construction of particular physical theories (and usually consists mostly of negations, in any case). In that case, however, there is always something mysterious at the root of these alleged explanations.
So Duhem argues that the true object or aim of physical theory must be summarization and classification of the phenomena. If we take this route, metaphysics does have a role to play (as we will see), but it's a role that succeeds rather than precedes the physical theory itself. Duhem's own definition of a physical theory is this: "It is a system of mathematical propositions, deduced from a small number of principles, which aim to represent as simply, as completely, and as exactly as possible a set of experimental laws." It is constructed through four successive operations.
1. Out of all the properties we are faced with, we select those we regard as simple properties, i.e., the properties that are such that all the other properties will consists of groupings of these simple properties, and through methods of measurement assign them mathematical symbols and operations.
2. We connect these mathematical symbols and operations together to form a small set of propositions that will serve as basic principles.
3. These principles are combined according to rules of mathematical analysis. In this combination, it is important to note that there is no assumption that a physical transformation corresponds to every mathematical transformation. The only thing we are bothered with at this stage is the mathematics of it.
4. Consequences drawn from these mathematical transformations are translated into judgments about the physical properties of bodies using the same methods of measurement used before; these judgments are then compared with experimental laws, i.e., the general conclusions we have drawn about the phenomena from our experiments. If there is a match, the theory is good so far; if there isn't, that's a sign the theory needs work.
These operations partly drive, and partly are driven by, a tendency toward simplicity or economy: we tend to accept the simplest physical theory that summarizes our experimental phenomena well. This is one way in which physics progresses. The other way it progresses is more interesting for our purposes, because it has to do with the fact that these operations make the physical theory a classification of phenomena. As we follow out the ramifications of our physical theory, different propositions start grouping themselves as related to this or that principle. So, for example, it arranges the spectrum of the prism alongside the spectrum of the rainbow, but places the colors of Newton's rings closer to the fringes discovered by Young and Fresnel; still other colorations are closer to diffraction. They aren't unrelated, but they aren't each equally related to everything else, and it is theory that tells us how to group them. These groupings make our knowledge about the world easier to use and apply; they protect us from obvious errors; and, equally importantly, they are beautiful. More importantly for our purposes, this high degree of perfection gives us the feeling that the classifications of physics are approaching a natural classification, in which the abstract relations of the theory match the real relations in the world. As Duhem says,
The neat way in which each experimental law finds its place in the classification created by the physicist and the brilliant clarity imparted to this group of laws so perfectly ordered persuade us in an overwhelming manner that such a classification is not purely artificial, that such an order does not result from a purely arbitrary grouping imposed on laws by an ingenious organizer. Without being able to explain our conviction, but also without being able to get rid of it, we see in the exact ordering of this system the mark by which a natural classification is recognized. Without claiming to explain the reality hiding under the phenomena whose laws we group, we feel that the groupings established by our theory correspond to real affinities among things themselves.
Something that particular shows this conviction in operation is the physicist's emphasis on prediction -- he doesn't just summarize using the theory, he extrapolates from it. In Duhem's picturesque phrase, he tells the theory, "Be a prophet for us." If the theory is artificial, it fits the phenomena so far; but if the theory is natural, it should fit all the phenomena (to the extent, at least, that it is natural). So in a prediction we are wagering that if the prediction is fulfilled the classification is natural. It is only a wager, but it is a good one; and it is a wager based on the view that our physical theory is at least approaching a natural classification.
It is important to note that this conviction is not itself a physical theory; nor does it admit of any direct experimental justification. The claim, "This theory approximates a natural classification to a high degree" is not a physical claim but a metaphysical. The physicist cannot justify it in physical terms. The closest he can come to doing so is by pointing to predictions; but these are only suggestive of natural classification. There is no way for the physicist to prove in physical terms that his theory parallels real affinities. (Duhem develops this at somewhat greater length than I have in my summary, since he deals with, for instance, the way physicists use instruments to extend sensation, and other issues relevant to this point.) But the conviction is almost impossible to avoid. Duhem calls it an act of faith; it is rational, but it is not based on any demonstration or rigorous argument (from the perspective of physics itself). But this conviction that physics is approaching a natural classification is, in and of itself, sufficient to justify physical work; even though physics does not explain, it does genuinely approach reality, because the ideal form of a physical theory is a natural classification. Such is Duhem's view of the relation between natural classification and physical theory.
All quotations are taken from Pierre Duhem, The Aim and Structure of Physical Theory, Wiener, tr. Athenaeum (New York: 1962), Part I. For further conclusions Duhem draws from this use of natural classification, read his classic essay, "Physics of a Believer."