Another reason was that the Founding Fathers were smart enough to distinguish between terrorists and everybody else on the whole damn planet, as demonstrated in the Treaty of Tripoli in 1797. It was in this agreement that the U.S. declared: "The government of the United States of America is not, in any sense, founded on the Christian Religion, as it has in itself no character of enmity against the laws, religion or tranquility of Mussulmen [Moslems]."
The problem with this was that the Treaty of Tripoli wasn't signed with peaceable Muslim kingdoms but with the Barbary states (officially part of the Ottoman Empire but for all practical purposes functioning as independent satellite states), whose favored economic activity consisted of seizing ships and either ransoming its crew or selling them into slavery. Prior to the American Revolution we were protected by the most powerful navy in the world, but after it we needed to protect ourselves. Spain helpfully got us out of our first scrape once the pirates realized we were ripe for looting, but it was clear that it couldn't go on. We took the only route seriously available to such a weak naval power as the U.S. was at that time: we arranged to pay the princes of the Barbary nations protection money. That was the whole point of the Treaty of Tripoli; it was a treaty with pirates in which we agreed to pay them if they would stop hijacking ships and taking Americans hostage. Every word in the treaty has to be understood in this context; Article 11, which is quoted above, is nothing other than an affirmation that because we had no established church we would not get Christian scruples and try to wiggle out of payment, but continue to be good little cash cows.
Such diplomacy is very tricky; the problem with paying pirates protection money is that they have a tendency to want more free money. Fortunately, Jefferson and the Founding Fathers were no idiots; they began building up the U.S. Navy (the Department of the Navy was founded in 1798, not so long after the Treaty of Tripoli) in preparation for the inevitable time when it would come to blows. The Pasha of Tripoli got greedy, began demanding ever higher payments, until Jefferson, who had always had his doubts about the long-term effectiveness of tribute, got tired of it and refused. The Pasha of Tripoli then broke the treaty, leading to the First Barbary War. (Algiers and Tunis, the other Barbary states, did not follow Tripoli into war.) Congress didn't declare war, but they did authorize Jefferson to do what was required to respond to the Pasha's acts of war. One of the famous events of that War, of course, was the extraordinary daring feat of the recently formed U.S. Marines, led by Lieutenant Stephen Decatur, in storming and recapturing the U.S.S. Philadelphia. The Pasha agreed to another treaty in which we ransomed back our prisoners, but did not have to pay further tribute to Tripoli. The Second Barbary War began in 1815 with Algiers, in another variation on the theme of the previous war.
So, while it's likely true that most of the Founding Fathers "were smart enough to distinguish between terrorists and everybody else", it's probably not a good idea to try to support that claim with a treaty whose whole purpose was to try to bribe and flatter pirates into not attacking us; particularly given that pirates were pretty much the closest think in pre-modern times to what we would call terrorists nowadays (which is why they traditionally have been treated very harshly by international law).
