Re-reading Abbott's Flatland led me to look into which of Abbott's other works are available online. As a rule the other works of this polymathic schoolmaster tend to be hard to find, but thanks to the Internet Archive, you can find several.
Onesimus is an interesting little didactic novel in which the slave Onesimus comes into contact with the intellectual currents of the first century, including paganism, Epicureanism, Stoicism, and Christianity. Abbott was a classicist, so he does an excellent job of getting the Epicureanism and Stoicism more or less right (although he acknowledges that to do so he has to engage in a certain amount of anachronism, to bridge gaps in our knowledge) while at the same time keeping them accessible. The work can be seen as an argument for Christianity in the form of a historical novel. One of the things that makes it interesting is that Abbott was what has variously been called a Modernist or a Liberal: i.e., he accepts that Christ is human and divine, and that the basic outlines of Christianity are divine revelation, but denies that miracles have ever occurred. He is, in other words, someone who regards Christianity as 'morally true'; he argues, for instance, that the miracle stories encapsulate important moral truths, but that they are cases in which valuable metaphors for moral life have been mistaken for physical truths. He prefers to call this 'spiritual Christianity' or 'natural Christianity'. Onesimus is a fairly enjoyable introduction to his views on this subject.
The reasoning behind this form of Christian thought is developed rather ingeniously in his work The Kernel and the Husk, which, while little-read, is one of the great classics of religious epistemology, an interesting counterpoint to Newman's An Essay in Aid of a Grammar of Assent, which takes a very different sort of line. Abbott develops similar issues in the less interesting but more systematic The Spirit on the Waters, where he argues for a form of spiritual evolution. (It's interesting that he uses Flatland in both works in order to illustrate his claims that omnipotence, omniscience, and omnipresence are not distinctive features of the divine.)
Mentioning Newman and Abbott as a sort of antithesis is fitting. In his day Abbott was perhaps the foremost Anglican expert on what might be called the Problem of Newman. The Problem of Newman, roughly speaking, is this. Here we have an Anglican, Newman, who is well-known to be an anxious seeker for truth, who is often considered to be one of the Church of England's top-tier minds even by those who disagreed with his views, and who made a very close and profound study of the history of Christianity, who feels drawn -- compelled -- to leave the Church of England and become Catholic. It's hard for us to grasp how serious an issue this was for Anglicans at the time, but many Anglicans followed Newman's path, and the three qualifications Newman possessed were enough to give many Anglicans pause who had no interest at all in becoming Catholics. Abbott, who had flirted with some of the ideas of the Oxford movement when younger, and who studied Newman extensively, wrote several works addressing the Newman Problem from various perspectives. His Philomythus is an attack on Newman's book on Ecclesiastical Miracles; it's a mix of insightful discussion, rant, penetrating criticism, and missing the point. There is a tendency here to read Newman very uncharitably, I think; but it has to be admitted that some of his criticisms do strike close to home. He has an interesting, and very sarcastic, satirizing of Newman's reasoning toward the end of the work.
Another of his works on the Newman Problem is The Anglican Career of Cardinal Newman (Volume 2 here), a sort of critical -- very critical, in parts -- biography of Newman in his Anglican period. One of the features of this work, as with Philomythus, that is difficult to get used to is that the criticism is largely psychologizing: it's an attack not so much on Newman's arguments as on his state of mind, and involves considerable speculation about what features of his psychology led him astray. It's certainly flawed in that sense, although it has to be recognized that in some sense the Problem of Newman is precisely concerned with Newman's state of mind: it's the question of how an Anglican mind, truth-seeking, ingenious, and well-informed, could become Catholic, and for the Anglican it's a problem precisely because the Anglican must suppose that Newman went wrong somewhere. Thus these works on Newman are Abbott's attempt to pin down what went wrong. The hazards of such an undertaking, however, are obvious; and one wonders what a biographer would make of Abbott if they were given the room for close critical scrutiny and psychological speculation that Abbott allows himself with regard to Newman.
One of Abbott's major influences was Francis Bacon. He wrote a book on Bacon's life and works that makes for interesting reading. Bacon's life is a very complicated one and Abbott does an excellent job both in clarifying it and in avoiding oversimplifications. Again, there's probably too much interest in lesser details of Bacon's psychology to be quite healthy, but as Abbott is not as thoroughly and hotly critical of Bacon as he is of Newman, it's less noticeable in this work. And to some extent, perhaps, it's a nineteenth-century British thing; witness the heated disputes over Newton's character.