[O]ur rulers can have no authority over such natural rights, only as we have submitted to them. The rights of conscience we never submitted, we could not submit. We are answerable for them to our God. The legitimate powers of government extend to such acts only as are injurious to others. But it does me no injury for my neighbor to say there are twenty gods or no god. It neither picks my pocket nor breaks my leg.
But this doesn't imply that we have a natural right to do wrong. Rather, it implies something very different: that we have natural rights of conscience -- in this particular case, worshipping according to conscience -- and these are inalienable; because they are inalienable, government can't interfere with the exercise of such rights at all unless such exercise is injurious to other people. In other words, it does not support the claim that we have a natural right to do wrong, but that because we have natural rights there is a necessary limit on the jurisdiction of government to limit certain kinds of action, regardless of whether they are wrong or not; some acts are such that we are answerable only to God. That's not the same kind of argument.
So it seems to me that Rowe's argument requires a controvertible assumption about how natural rights are distinguished. One way of thinking of natural rights is to consider them to be general ends or objects that must be protected. Thus, for instance, I have a natural right of liberty in matters of speech; and by typing this I am exercising that right. However, one could one might always try to individuate natural rights much more finely than this; in which case you get strange results. For instance, since to exercise my natural right of liberty in matters of speech in this particular way that I am doing, I have to have a natural right to the precise means. Thus, one would say, I have a natural right to press the N key, and a natural right to press the A key, and a natural right to press the T key, and so forth, and, moreover, I have a natural right to write 'natural' and a natural right to write 'right', and so on through all the letters and words of all possible languages. But this is very peculiar, because of course I don't have a 'natural right to press an M key'; the existence of M keys at all is both contingent and incidental to my own existence. Such 'natural rights' would trivialize the very notion of a natural right. What we do have natural rights for are more general types of action for which pressing such-and-such keys is in a particular set of circumstances a necessary means. This does not imply that the pressing of keys generally is a natural right.
So the objects of rights cannot be regarded as arbitrarily fine. In which case, we can't say immediately that anyone has a natural right to worship false gods, without an argument that this worship is either not contingent or not incidental to my existence; only that (for instance) people have a natural right to something more general, and the protection of this requires the protection of people exercising that right regardless of whether they do so rightly or wrongly. That is, a natural right can lead to our having a civil right to do something wrong, in the sense that because of the natural right we must at times be protected from government interference even in doing wrong. But civil rights and natural rights should not be confused. It's possible that Jefferson holds a stronger view; but the above passage does not show it; and given other things that Jefferson holds (like the two points above), any stronger view would seem to require us to say that he's being inconsistent.
(As an incidental but related matter, Rowe claims here that Calvin says we have no right to revolt. This is a reasonable interpretation, if 'right to revolt' is taken to be a general right. However, it must not be confused with the claim that we never have a right to revolt in the sense that we never have a right to do something that might as a necessary means involve revolting; Calvin is quite clear in the Institutes, for instance, that magistrates may be overturned if there is a clear call from a higher authority. Calvin denies, in effect, that we have any general right to revolt; however, we do have not merely the right but the duty to do things that will in particular cases involve revolting against a particular authority under the auspices of a higher authority. Further, we have to be careful in interpreting Calvin, because it is clear in every place he discusses the matter that his chief worries are people who want to get rid of civil government entirely, i.e., certain Anabaptists, and many of the strongest-sounding things he says against revolt are clearly aimed at that particular group. Further, in the Institutes he also makes the distinction that will be used at length by Calvinist supporters of particular revolutions, between private subjects, who are to obey authorities, and magistrates, who are to protect the people; Calvin is quite clear that magistrates who are appointed to protect the people from tyranny have every right to oppose tyranny over those people, even by a higher authority. This is important because Rowe often argues that Calvin's view is inconsistent with the principles of the American revolution. However, the American revolution was not a popular uprising like a peasant revolt but a revolt of magistrates against higher magistrates; and that means that Calvin's principles can underwrite it if in so doing they were exercising a legitimate authority to protect the people.)
