She was brilliant but she didn't have a grain of sense. It seemed to Mrs. Hopewell that every year she grew less like other people and more like herself--bloated, rude, and squint-eyed. And she said such strange things! To her own mother she had said--without warming, without excuse, standing up in the middle of a meal with her face purple and her mouth half full--"Woman! do you ever look inside? Do you ever look inside and see what you are not? God!" she had cried sinking down again and staring at her plate, "Malebranche was right: we are not our own light. We are not our own light!" Mrs. Hopewell had no idea to this day what brought that on. She had only made the remark, hoping Joy would take it in, that a smile never hurt anyone.
The girl had taken in a Ph.D. in philosophy and this left Mrs. Hopewell at a complete loss. You could say, "My daughter is a nurse," or "My daughter is a schoolteacher," or even, "My daughter is a chemical engineer." You could not say, "My daughter is a philosopher." That was something that had ended with the Greeks and Romans. All day Joy sat on her neck in a deep chair, reading. Sometimes she went for walks but she didn't like dogs or cats or birds or flowers or nature or nice young men. She looked at nice young men as if she could smell their stupidity.
Malebranche does indeed insist that we are not our own light. In saying this Malebranche is actually quoting Augustine, and it turns out to be perhaps his favorite quotation. It comes up repeatedly in the long dispute with Arnauld, because from Malebranche's perspective (although, of course, not from Arnauld's), Arnauld is denying Augustine's dictum by denying Malebranche's claim that we see all things in God.
I don't know how much O'Connor knew about Malebranche, but this scene is beautifully done. There is a lot of irony in Joy's (or, to use the name she prefers, Hulga's) reference to Malebranche here. Malebranche, of course, has a thoroughly theistic metaphysics and epistemology; when he is quoting this line from Augustine (who is making a point about our metaphysical dependence on God in our ability to know), he is making a thoroughly theistic point. But Hulga, of course, is an atheist and nihilist. Nonetheless, her description here, if taken in a literal way she didn't intend, describes Malebranche's point to a T. "Do you ever look inside and see what you are not? God!" The last bit is clearly just a cry of exasperation; but the whole thing describes a good portion of Malebranche's argument, which is that when we look in side we see that we are not God, we are not a light to ourselves; it is in His light that we see light. But of course, she is making the opposite point, that we are nothing, that there is nothing to believe in, the sort of thing O'Connor has her reading in the book that strikes simple Mrs. Hopewell as an "evil incantation:
One day Mrs. Hopewell had picked up one of the books the girl had just put down,and opening it at random, she read, "Science, on the other hand, has to assert ist soberness and seriousness afresh and declare that it is concerned solely with what-is. Nothing--how can it be for science anything but a horror and a phantasm? If science is right,t hen one thing stands firm: science wishes to know nothing of nothing. Such is after all the strictly scientific approach to Nothing. We know it by wishing to know nothing of Nothing."
As she describes herself later on, "I'm one of those people who see through to nothing."
Malebranche has an interesting passage in which he talks about seeing nothing (Search 3.2.8, Lennon-Olscamp 241):
The clear, intimate, and necessary presence of God (i.e., the being without individual restriction, the infinite being, being in general) to the mind of man acts upon it with greater force than the presence of all finite objects. The mind cannot entirely rid itself of this general idea of being, because it cannot subsist outside God....One might well not think about oneself for some time, but it seems to me one cannot subsist a moment without thinking of being, and at the very time that one takes himself to be thinking of nothing, one is necessarily filled with the vague and general idea of being.
As Malebranche says later, "nothingness is neither perceptible nor intelligible" (LO 321); what we call 'nothing' is really just nothing in particular. But nothing in particular is not being-less; it is being in general, without consideration of how it relates to any particulars. And on Malebranche's metaphysics, being in general, as an object of thought, is divine (unrestricted being). So for him, someone who is "seeing through to nothing" is really just seeing being in general in the divine being, without recognizing it as such. In her allusion to Malebranche, Hulga has (without apparently realizing it at all) entangled herself in a rather massive philosophical irony, one that (I hate to use the word, but it'll do) subverts her own nihilism by juxtaposing it with its complementary opposite.