Friday, June 07, 2013

Lady Mary Shepherd on the Inverted Image Problem

Vision was a significant philosophical topic in the early modern period, one that was often held to be related to a wide variety of other philosophical topics. It is thus not as surprising as might be thought that, when Lady Mary Shepherd (1777-1847) identifies her three most important philosophical arguments in a letter to Robert Blakey [Note 1], she treats her account of single and erect vision as one of them, and as closely related to the other two, causation and final causes, and as related even to as metaphysical question as the existence of God. As she puts it:

[The three topics] confute modern Atheism, founded, as it is, upon fallacious inferences, from Locke, Newton, Hume, and Berkeley. For unless there be a cause, there exists no first, essential, or necessary cause. Unless final causes are physical efficients, they could not operate, unless upon every theory of the mind. The fact of single and double vision cannot be explained consistently with any theory, and as being deducible from the general laws of causation. Such a theory is null, for two reasons; therefore, I encourage myself to hope for the future success and prevalence of my own notions [Note 2].

In short, the fact that we see singly and in a non-inverted way is a test case not just for a theory of vision itself, but also for the theory of mind (whose status as a the receptive cause of sensation is a major part of Shepherd's account of the external world) and, even more broadly, for the theory of causation, since any account of these facts would have to be causal.

Shepherd discusses single and erect vision in two essays. The first is Essay XIV of the Essays on the Perception of an External Universe, "On the Reason Why Objects Appear Single Even Though Painted on Two Retinas, and Why They Appear Erect Although the Images Be Inverted on Them" (henceforth Essay XIV). The second was published in The Philosophical Magazine in 1828, a year after the publication of the Essays [Note 3], and is called "On the Causes of Single and Erect Vision" (henceforth CSEV). The essays approach the topic somewhat differently, but both provide essentially the same account. Understanding them, however, requires recognizing the state of the problem by the time it reached Shepherd.

For the purposes of most discussions in early modern philosophy, we can attribute the beginning of the inverted image problem to Kepler, who in Ad Vitellionem paralipomena (1604) proposed an optical account of image formation in the eye in which light formed a 'painting' on the retina in much the same way that a camera obscura forms an image on the wall. This account, however, had the optical rays crossing between the lens and the retina, and it was recognized by everyone that this had the immediate, and puzzling, implication that the image on the retina was upside-down despite the fact that we obviously don't see the world upside-down. A number of people, including Pierre Gassendi, argued against this account. The Keplerian account became generally accepted, however, with the publication of Descartes's Dioptrics (1637) [Note 4], whose Fifth Discourse describes a famous experiment. Descartes took an eyeball (he recommends that of a deceased man, or, failing that, of an ox) and scraped out the sclera at its back, so that the back of the eyeball was transparent. He was then able to observe directly the inverted image on the back of the eye. The Cartesian explanation of this was that the mind traces back (so to speak) the rays and thus recognizes that the lower part of the image actually comes from above and the upper part from below. Descartes uses a famous analogy to a blind man with two crossed sticks; when he touches something with the sticks, he will surely recognize that what he touches with the stick in the lower hand is actually towards the top, while what he touches with the stick in the upper hand is actually towards the bottom.

The inverted image problem was later discussed by Berkeley in An Essay Towards a New Theory of Vision (1709), who sums it up well in section 88:
Among the discoveries of the last age, it is reputed none of the least that the manner of vision hath been more clearly explained than ever it had been before. There is at this day no one ignorant that the pictures of external objects are painted on the RETINA, or fund of the eye: that we can see nothing which is not so painted: and that, according as the picture is more distinct or confused, so also is the perception we have of the object: but then in this explication of vision there occurs one mighty difficulty. The objects are painted in an inverted order on the bottom of the eye: the upper part of any object being painted on the lower part of the eye, and the lower part of the object on the upper part of the eye: and so also as to right and left. Since therefore the pictures are thus inverted, it is demanded how it comes to pass that we see the objects erect and in their natural posture?

Two and a half decades later, in The Theory of Vision; or Visual Language, Shewing the Immediate Presence and Providence of a Deity, Vindicated and Explained (1733), he will say (section 52), "The solution of this knot about inverted images seems the principal point in the whole Optic Theory, the most difficult perhaps to comprehend, but the most deserving of our attention, and, when rightly understood, the surest way to lead the mind into a thorough knowledge of the true nature of Vision." Berkeley is strongly dissatisfied with the Cartesian answer to the problem. How is this tracing of rays to be done? Any child can see what is right-side-up, but when we ask the Cartesian how they do this, we get the optical theory of the eye, with its abstract geometrical inferences. Children surely do not think through the implications of optical principles in order to see the world as not inverted. Berkeley attempts to provide a better solution using his key distinction between visible and tangible ideas in our perception of situation. We do not actually see the image on the retina at all. The idea of light hitting the retina is a tactile idea, an idea of physical contact, not a visual idea. On Berkeley's new theory of vision, our visual ideas of the world are in general signs of tactile ideas, so that we can often find things in our visible experience that have a direct correspondence in our tangible experience, to such an extent that we often confuse the two despite the fact that they are not the same at all. Thus when we talk about the inverted image on the retina, the 'inversion' is not a literal inversion of anything visible. All we are saying is that, when we compare our visual experience with the tactile idea of the physical contact of light on the retina and how we move our eyes up and down (the source of our experience of things as up or down in Berkeley's account), we find that the tactile pattern is inverted from what we are in the habit of taking the visual experiences to indicate.

With Thomas Reid, in An Inquiry into the Human Mind on the Principles of Common Sense (1764), we get a summing up of the entire tradition, and of the inverted image problem as Shepherd understood it. Like Berkeley, Reid rejects the Cartesian solution because it requires the solution to be a rational deduction from premises of which the greater part of the human population seem to be unaware. Ordinary people do not do optical geometry in order to see the world the way they do, and thus optical geometry does not actually provide any explanation of the apparent discrepancy at all. He also rejects Berkeley's solution, however. The Berkeleyan solution to the problem requires that our ability to determine whether things are inverted or not is wholly derived from experience. We simply develop the habit over time of thinking of some things as up and down, based on things like the motion of our eyes; it's entirely a matter of acquired expectation through consistent association. One of Reid's problems with Berkeley's solution is that it makes the distinction between visible and tangible ideas too sharp, treating them as having no natural similarity to each other, which Reid denies; and he suggests that Berkeley's acceptance of the purely associative answer is at least partly influenced by his acceptance of idealism. If you think that the external world consists entirely of ideas in the mind, the position that we directly perceive things as having an orientation and situation, which would also have to be purely mental, might well seem unattractive.

Faced, then, with a choice between the Cartesian solution based on rational inference and the Berkeleyan solution based wholly on associations acquired from experience, Reid jumps through the horns of the dilemma and proposes a third kind of position: our seeing of the world as erect rather than inverted is due neither to rational inference or empirical association but an original principle of the constitution of our minds, or, in other words, it's all simply a matter of the way we are set up from the beginning. Against Berkeley, Reid argues that the inverted pictures on the retina are indeed part of the process of vision, a means of seeing the world. However, the greater part of this process is still a mystery. It seems that the picture on the retina affects the optic nerve somehow, which affects the brain somehow, which makes us see the world somehow; but we are deceiving ourselves if we think we have any clear conception of how this is done. The image on the retina does not travel up the optic nerve into the brain, and even if it did, there is nothing about literally having an image in our brains that would explain how we see anything. It is not, however, particularly necessary to have an explanation of all this in order to address the inverted image problem. With most things we do not need to know more than the fact that one thing follows another by a regular connection; that is, our usual mode of explanation is by tracing things back to laws of nature, which means simply that we recognize that two things are invariably and constantly connected.

The real question at the heart of the problem, therefore, is just this: What is the law of nature according to which the image on the retina is constantly and invariably connected to my seeing things a certain way? Reid argues that the relevant law of nature is this: Every visible point of the object is seen in the direction of a straight line from the picture on the retina through the center of the eye. This is the regularity that links the inverted image with our visual experience, and this law of nature describes a structure of the mind itself, part of our design-plan, so to speak. Thus the solution to the inverted image problem lies not in reasoning, nor in experience, but in the constitution or structure of our minds.

This gives us the lay of the land. When Shepherd considers the inverted image problem, she is considering a problem that prior philosophers had attempted to resolves with three different explanatory principles: reason, experience, and mental constitution. How then does she resolve the problem?

She identifies the five basic assumptions of her solution in CSEV:

(1) "Vision is a consciousness in the mind, and its next proximate cause must be a power equal to its production, and which unites it to the material world."

(2) "Vision of one colour only can never yield the vision of figure, because the proximate cause of the vision of figure is a line of demarcation formed by the sensation of a junction of two colours."

(3) "The physical impulse producing such consciousness of colouring, is an equal proportional variety upon the retina of an eye; one eye alone being first supposed, as it is sufficient to yield the idea of figure."

(4) "An object cannot be in two places at the same time."

(5) "An object cannot exist and put forward its action where it is not."

The first of these assumptions identifies the general character of the effect and establishes the kind of inquiry in which we are engaged in trying to solve the problem. It is a causal inquiry, and the result for any phenomenon needs to be a cause of consciousness of the phenomenon, adequate for causing the phenomenon, that connects conscious vision to the material world. The second gives the proximate cause of seeing a figure or shape. To see a figure or shape requires seeing a boundary, and this requires seeing at least two distinct colors. The third is an empirical discovery based on the study of the eye. The fourth and fifth assumptions serve as filtering out genuine from spurious candidates for solutions. All of them except (3), which is presupposed by the problem, are regarded by Shepherd as necessary principles, although not necessarily equally obvious (she spends a considerable portion of Essay XIV explaining (2), for instance.)

On the basis of these assumptions the related problem of single vision, that is, why we see one thing rather than two, given that our vision of this one thing is based on two distinct images on two distinct retinas, is almost trivially easy: since we can only see two visual objects as two by observing some line of demarcation between them, we could only distinguish the images on our retinas if we saw them both with a line of demarcation between them. As she notes, if this is true, we only regard single vision as a puzzle because we are imagining the space between our eyes as a demarcation. But unless we are doing strange manipulations with our eyes, we do not actually see the space between them, because the eyes receive no physical impulse of light from that space. Thus the two images are for all practical purposes superposed, and, not being distinguished visually by a visual line of demarcation, the mind's capacity for vision is naturally affected by them as indistinguishable. We can do various manipulations that change this (like pressing our eyes in different directions), but it will only do so by introducing something visual that serves as a line of demarcation. In short, Shepherd's resolution of the problem depends on recognizing that answering the question of why we see one thing despite two images in reality depends on asking the question, "What immediate cause would make us distinguish things as two?" And the only immediate visual cause that can make us distinguish things as two is some kind of visible demarcation between them. Thus the physical impulses in the eyes and the images on the retinas can only cause us to see two if they are such as would create a visible demarcation. But each image on the retina is (in most cases) equivalent to the other; thus neither image includes a visible demarcation between itself and the other image. Thus, since nothing else in the situation would seem capable of introducing a visible line of demarcation, there actually is no reason to think that the two images would cause us to see anything other than one thing. That something like this is right is confirmed by further experimental facts about how the images correspond to each other and to our vision.

Single vision is the easy case, but the inverted image problem is handled in the same way. Whether a figure is inverted or not is a matter of relative position of color. I see a flag pole inverted in a reflection for instance, because it is relatively positioned in a way that contrasts with the erect flag pole it is reflecting. But, recognizing this, we already see the beginnings of the solution to the problem:

Now the real fact is, the painting of objects, though they be inverted, does not alter the painting of their relative positions; the whole colouring of all within the sphere of vision, maintains precisely the same position of things towards each other: but it is the appearance of an opposite appearance of things, i.e. an opposition of the relative colouring of things, which only can yield the idea of inversion of images:--Thus a candle would appear to be topsy turvey upon a table, if the flame appeared to touch the table, and the bottom of the candlestick pointed upwards towards the ceiling; but if the bottom of the candlestick maintains its relative position to the table, and the flame the same relative position to the heavens, and the table the same to the earth, and the earth the same to the table; then the whole,--from the earth to the heavens, being painted in an inverted position on the retina, cannot possibly occasion any sense of inversion of images;--because the sense of the soul must be to perceive the whole relative position of objects, precisely in the relation of parts they have to touch and motion.(Essay XIV, page 414)

Like most of Shepherd's super-sentences, that one has to be worked through carefully; but it is well worth it. The inverted image case is very much like the single vision case. To think it a puzzle, we have to be imagining both the image on the retina and the thing in the world of which it is an image. To recognize visually that anything is inverted, we have to recognize that it is opposite in orientation to something, which we can only do visually if we see the thing it is opposite to. As she puts it in CSEV, "The idea of inversion is the result of the comparison of the line of demarcation of one object with that of another of a similar kind placed in a contrary direction to it." You can't see a relation if you don't see the relata. When philosophers puzzle about why we do not see the world as inverted, they are really supposing that vision simultaneously sees two things, the image and the objects they invert. Otherwise, how, could we see the one as inverted in comparison to the other? The inverted image is somewhat more complicated than the single vision case because we also get information about orientation from touch and motion (Shepherd occasionally calls our capacity to move through space our "sixth sense"). Thus a full answer would require discussing the relation between sight and touch (one of the most important topics in early modern discussions of vision). But we can already see that any discussion along these lines would merely refine the basic point: we do not see the image and the object in our eye, and we do not see the object causing the image in our eye. Objects are out in the world; they can neither be two places at once nor act where they are not.

Thus when we compare Shepherd's remarkably elegant solution of the problem to other proposed solutions, we find it is yet another option on the table, and does not reduce to the other three. It does not require that erect vision be a result of rational inference, nor does it make erect vision a matter of custom and experience, nor does it attribute it to the original constitution of our natures. It is more closely related to the rational explanation than to the other two, since both the the rational explanation and Shepherd's causal explanation make erect vision a necessary consequence, as opposed to the contingency attributed to it by the custom and original constitution explanations. But Shepherd's solution eliminates the problem by arguing that there is no causal reason why the image would look inverted, and that the problem only arises by counterfactual imagination. It is in this way an excellent example in miniature of Shepherd's general analytic style, since she does similar things on a larger scale with Hume's account of causation and Berkeley's idealism.

We have had nearly two hundred years of additional study for the problem, so you might be wondering what the results of that are. The answer is that, as with many of the problems in the theory of vision first discussed by early modern philosophers, it is not a completely closed question; fi nothing else, they knew how to pick the hard problems. For a very long time there was a considerable tendency to accept the custom explanation. The reason for this is that at the very end of the nineteenth century, George Stratton designed a kind of experiment to study the question, in which he wore inverting spectacles for a considerable period of time. His conclusion, which he published, was that after a sufficient period of time he saw the image in the inverting spectacles as right-side up. This was apparently confirmed by some other experiments. However, different modifications of the experiment through time have not been quite so definite. In 1999 Linden et al. published a study (PDF) in which they came to the conclusion that people wearing the inverting glasses eventually adjusted to the change, but that they never actually saw the image as right-side up. Thus the problem is still in play (Note 5).


(1) The letter to Blakey, dated May 26, 1843, is the latest extant comment by Lady Mary Shepherd on the subject. It was published by Blakey in his Memoirs.

(2) "Single and double vision" seems to be used here simply as a summary title for the entire essay in Essays on the Perception of an External Universe, rather than as a way of singling out the single vision argument rather than the inverted vision argument.

(3) This essay is missing from the Thoemmes Press edition of Shepherd's collected works, probably because it seems never to have been published in book form. The essay was widely distributed through the journals of the day, however.

(4) The Dioptrics is another example showing the importance of the topic of vision to early modern philosophy, since it was one of the three essays for which the Discourse on Method was written as an introduction, and thus explicitly put forward by Descartes as an account of early successes of Cartesian method.

(5) And still discussed heatedly by philosophers. To take just one example, István Aranyosi in The Peripheral Mind has a good discussion of how the problem is related to issues in the dispute over whether the mind is representational or "enactivist".


  1. Reid’s
    account sounds very interesting; a book to look for.

    Indeed, the retina doesn’t ‘see’,
    there are discreet receptors getting stimuli, etc., and we don’t ‘perceive’ the
    image formed on the retina, seeing is a complex mental function.

    But why would we see different, how
    could we? We acknowledge spatial relations to the parts of our body.

  2. ronmurp4:14 PM

    We can 'see' the world through impulses on the tongue fed by a camera. There need be no simple relation between retinal image geometry and mental experience.

    Think of the George Stratton experiment in terms of language. We read English and it makes sense. A new language at first is meaningless, but as we acquire it we improve, to the point of being fluent without active intention. But unless you learn both languages from an early age only you native language will be natural. This distinction, between the 'natural seeing' through the native language as opposed to the 'adjusting to the change' would explain Linden's observed limitation on adaptation. The brain is plastic, particularly when young, but no so much once a particular natural or native use has been acquired.

    Philosophical notions of vision seem to retain ideas of projections. But the mental experience is far richer than the physical mechanism of eye operation allows. There seems to be much live 'reconstruction' (it's hard to avoid geometric metaphor) and filling in of detail. And the visual process is distributed in the brain, and related to other senses, like balance.

    When thinking about vision we tend to apply our subjective experience as if that's indicative of what's going on when it need not be. Our subjective mental experience of vision is just one more mental illusion.

    We observe a rotating mask and the concave side appears convex. We call this a visual illusion; but it's visual only in terms of the sense with which we acquire the data. There is nothing illusory in the data that reaches the eye, or in the neural signals that enter the brain. The illusion is a mental one. It is the brain's higher level constructive processes that mistake the concave for convex, because convex faces are the greater expectation.

    Mental 'vision' need be nothing like a geometric representation. All that is required is that there should be some consistent correspondence between reflective light from surfaces and the internal neural events, and that's good enough. The adaptive capacity of the brain needs only see consistently for the data to become 'normal' vision.

    Another analogy: some musicians can read sheet music and 'hear' the music in their heads. The grossly encoded data *becomes* the audible experience. Neuronal impulses *become* the visual experience. Changing the data, inverting for example, is merely changing the encoding of the data. The brain learns the new language and re-acquires the capacity experience the data as normal vision - or as near normal as these limited expriments allow.

  3. branemrys4:38 PM

    Philosophical notions of vision seem to retain ideas of projections.

    But none of the views of vision discussed in the post regards vision as constituted in any way by projections -- this is most obvious with Berkeley, in whose theory of vision it makes no sense at all to think of vision as a projection, but it is true even of Descartes, who puts a much greater emphasis on optics (because he is very interested in the physical mechanism of the eye), because Descartes's dualism prevents him from taking the actual experience of vision to be reducible to the physical mechanisms he discusses.


Please understand that this weblog runs on a third-party comment system, not on Blogger's comment system. If you have come by way of a mobile device and can see this message, you may have landed on the Blogger comment page, or the third party commenting system has not yet completely loaded; your comments will only be shown on this page and not on the page most people will see, and it is much more likely that your comment will be missed.