This whole controversy is by no means earth-shattering to me. Nothing wrong with building a mosque, in a public-square sense, and I don’t think if it gets built it will mean America has lost its backbone or whatever. But a couple of thoughts.
Cultural sensitivity
Campus Crusade for Christ isn’t called that everywhere. In parts of Europe it’s called Agape Europe, or Student Life. In Canada it’s called Campus for Christ.
Why is that? It’s because CCC wisely understands that the word “Crusade” carries negative connotations in some part of the world. They seek to be culturally sensitive by avoiding it if it’s likely to give offense.
The issue with a mosque and Islamic cultural center in what would have been the shadow of the Twin Towers isn’t one of legality. Of course, if the center’s financial backers can buy the land and get the permit, they have as much right to build there as anyone else. The question is one of wisdom and sensitivity.
As Charles Krauthammer has pointed out, if the mosque’s backers really wanted to build bridges of understanding between Muslims and other Americans, they would know that this is not the way to do it. Having the right to do something does not make it the right thing to do.
The First Amendment
As I said before, the legality of the mosque is not in doubt. The First Amendment guarantees the free expression of, among other things, religion. If this group obeys the law, there’s no legal reason they can’t build.
But it won’t do to accuse those who oppose the mosque of being anti-First Amendment or un-American. (Remember, oh, 2 years ago or so, when it was considered really bad to imply that someone was less than patriotic?) The First Amendment does indeed grant the right for religious groups to build houses of worship. But it also gives others the right to protest and try to change things if they don’t want a mosque– or a strip club, or a car lot, or a Wal-Mart– in a certain spot.
Wiser Time 
I’ve been wondering about this from another point of view. Here in America it’s convenient to believe that the sole purpose of Islam is to destroy anything that is not Islam, i.e. white, Evangelical, conservative corporate, etc. Obviously not all that study Islam are terrorists in the same fashion that not all Christians murder Abortion providers as was strangely in fashion a decade or so ago.
So what if this is an attempt to show a non-militant Islam? What if this is their attempt to bring some degree of peace to a place where others associated with Islam have brought so much tragedy?
They obviously have all legal rights to build and as a religious group they will attempt to proselytize but at the same time I don’t think it’s wise to assume Muslim’s are solely “out to get us.” I think it’s easy for all people to assume people that fall into the “other” or “outsider” category, anyone different from themselves, is one-dimensional and that dimension be negative.