In all cases, Knowledge implies a combination of Thoughts and Things. Without this combination, it would not be Knowledge. Without Thoughts, there could be no connexion; without Things, there could be no reality. Thoughts and Things are so intimately combined in our Knowledge, that we do not look upon them as distinct. One single act of the mind involves them both; and their contrast disappears in their union. (PIS Part 1, Book 1, Chapter 2, Section 1)
This antithesis of Thought and Thing serves as the foundation of the antithesis of Theory and Fact, although the two are not strictly identical. To explain this it is useful to go to yet another antithesis that depends on the antithesis of Thought and Thing: that of Ideas and Sensations:
There are other modes of expression also, which involve the same Fundamental Antithesis, more or less modified. Of these, the pair of words which in their relations appear to separate the members of the antithesis most distinctly are Ideas and Sensations. We see and hear and touch external things, and thus perceive them by our senses, but in perceiving them, we connect the impressions of sense acording to relations of space, time, number, likeness, cause, &c. Now some at least of these kinds of connexion, as space, time, number, may be contemplated distinct from the things to which they are applied; and so contemplated, I term them Ideas. And the other element, the impressions upon our senses which they connect, are called Sensations. (126.96.36.199)
He then later explains:
That which is a Fact under one aspect, is a Theory under another. The most recondite Theories when firmly established are Facts: the simplest Facts involve something of the nature of Theory. Theory and Fact correspond, in a certain degree, with Ideas and Sensations, as to the nature of their opposition. But the Facts are Facts, so far as the Ideas have been combined with the Sensations and absorbed in them: the Theories are Theories, so far as the Ideas are kept distinct from the Sensations, and so far as it is considered still a question whether those can be made to agree with these. (188.8.131.52).
(The edition from which I am quoting is published by John W. Parker in 1847). Whewell tends to be influenced by Kant, and there are some aspects of this influence that I don't particularly like. But one thing I find interesting is his insistence that the mind cannot be regarded merely as a passive spectator of the world; it must act on the sensations it receives to make them intelligible. It is this active aspect, by which the mind actively makes sensations intelligible, that he intends to indicate by the word 'Idea'; Ideas are not objects of mind but "Laws of Thought" or "different forms fo teh impulse of the mind to generalize" or "Forms of Intuition". As he says at one point, we do not usually see Ideas; we see through them. One way he describes this is as the interpretation of nature; another way is as the information of (or formation of) sensations.