Chances are, when you were told in your elementary school days what prime numbers are, you were told (as I was) that the number 1 is prime. The usual definition given is that a prime number has no positive divisors other than 1 and itself, and this is certainly true of 1. But in fact this has never been universally accepted among mathematicians, and seems only to have been widely accepted (as far as I can tell) for a brief period from the late nineteenth to the middle of the twentieth century.
The ancient Greeks did not regard 1 as a prime number; they didn't regard 1 as a number, so they would have regarded the idea of 1 being a prime number as absurd. (1, they thought, was the source or principle of number -- it is not a number because it is what you use to make numbers out of. If 1 is not a number, then you can explain how you get numbers; otherwise, it's a bit of a mystery.) On the authority of the Greek mathematicians (and above all Euclid himself), pretty much nobody in the ancient or medieval West would have even thought of the primality of 1 -- someone proposing it would have sounded like they were making up new definitions for no good reason at all.
It's only about the sixteenth or seventeenth century that significant mathematicians occasionally start treating 1 as a prime number, and this slowly increases over time. The reason is primarily due to the idea that all positive numbers should be either prime or composite. (Those who are interested can look over some of the basic evidence in Caldwell, Reddick, and Xiong, "The History of the Primality of One: A Selection of Sources", which is available free online; there is commentary by Caldwell and Xiong in another paper, also freely available.)
The Fundamental Theorem of Arithmetic, however, has been the determining issue. The basic idea of the Fundamental Theorem of Arithmetic is that all whole numbers greater than 1 have a unique prime factorization -- for instance, the unique prime factorization for 25 is 5x5, that of 12 is 3x2x2, and so forth. This idea, extremely important since Gauss, has played an increasingly large role in the mathematician's conception of what whole numbers are. If we take 1 to be a prime number, though, there are no unique prime factorizations. You can still get the Fundamental Theorem of Arithmetic, without that much difficulty, just by adding additional 'except for 1'. But if you do that, then you have to add the exception every single time you use it -- and the importance of the Fundamental Theorem of Arithmetic is such that this might be a lot. So the very practice of mathematics has pushed against the idea that 1 is prime; taking it to be prime is extra work that doesn't get you anything important, but only makes it harder to reason smoothly and communicate simply -- only a little bit, to be sure, but it's a little bit that can add up very quickly. The rough and basic is described in the following Numberphile video:
What interests me about all of this is that it's a straightforward case of classification in mathematics. There's a lot of classificatory work in mathematics, but it tends to get overlooked when we are talking about how classification in general works. The question of whether 1 should be classified as prime, however, strikes me as a good example of what Whewell has in mind in discussing the primary elements of classification.
Whewell takes the basic regulative principle of all classification to be that it enables the assertion of true and general propositions, or, as he also puts it, general propositions (of science) should be possible. This is the end or goal of classification, and it is according to Whewell the measure of progress in classificatory sciences. Whewell is usually thinking of botany, zoology, and his own field of mineralogy, but it is a perfectly general point, and recognized as such by Whewell himself. In Aphorism VIII Concerning the Language of Science, Whewell gives an entirely general version of it: "Terms must be constructed and appropriated so as to be fitted to enunciate simply and clearly true general propositions". He calls this "the fundamental principle and supreme rule of all scientific terminology", and he goes on to say that it "applies equally to the mathematical, chemical, and classificatory sciences." Classification is the foundation of terminology, and both classification and terminology subserve the ends of rational inquiry.
Interestingly, Whewell recognizes that this general principle could lead to situations in which different contexts might require us to go in different directions. His example is that of whether whales should be counted as fish, and his answer is that it just depends in context on which happens to be more useful to expressing true general propositions, and it could be that in biology we should say No, while in law, however rational or rigorous, we should say Yes, because the relations that matter for true general propositions in law are not necessarily those that matter for true general propositions in biology.