Having played hostess to a number of different non-Muslim groups visiting our mosque, this issue of taking off one's shoes in the mosque has manifested itself in some interesting ways. I'm afraid some people even might have the wrong idea about why shoes are removed inside the mosque.
Before the expansion had opened, but while the old musallah had been closed, we were forced to hold our da'wah presentations in the gym. Most activities took place in the gym--including the prayers. But several of our visitors were "regulars," from school groups whose instructors regularly sent their classes to the mosque for presentations. They were well-aware that shoes needed to be removed inside the prayer hall, and informed their students ahead of time.
For these presentations in the gym, we had to explain to everyone that they could leave their shoes on. Frankly, if I were to walk across a gym floor, I'd rather leave my shoes on. But this is a much easier problem to deal with than the reverse, which is an audience previously unaware they would be removing their shoes--for sometimes this can make people uncomfortable.
Because it's so common for shoes to be removed inside a mosque, and to see shoe racks near the doors, I thought it would be worthwhile to explain (if I can, inshaaAllah) the purpose for this behavior.
Some people, if they were asked why Muslims remove their shoes in a mosque, would respond by vaguely saying "out of respect." Respect for what, or for whom? Do Muslims need to pray with their shoes off? Some people might say yes, others might say no. The answer is: not unless their shoes have impurities on them. So it isn't necessarily for the prayer, but is it to respect the space? I have heard some people say that the masjid is "sacred ground," like when Moses (s) was told to remove his shoes in front of the burning bush because he was on sacred ground.
But refer to a hadith reported in Sahih Muslim where the Messenger of Allah (s) in explaining the differences between other messengers and himself (s), says "the earth has been made sacred and pure and mosque for me, so whenever the time of prayer comes for any one of you he should pray whenever he is." If the earth has been made sacred, pure, and a mosque, then how can we say that inside the walls of a mosque are sacred and we must take our shoes off?
In fact, we have traditions about the Prophet Muhammad (s) praying with his shoes on. Anas bin Malik was asked the question, if Muhammad prayed with shoes on, and he answered yes (reported in Bukhari and Muslim.) But the condition about the shoes being free of impurities comes from another hadith in Sunan Abu Dawood, on the authority of Abu Sa'id Al-Khudri. In this hadith, the Prophet Muhammad (s) was leading his companions in prayer, and during the prayer he removed them and placed them to his left. His companions copied the behavior, removing their shoes, and at the end of the prayer the Messenger of Allah (s) asked them why they did so, and of course they replied that they saw him remove his shoes. At this point, the Prophet Muhammad (s) explained that the Angel Gabriel had come to him during the prayer and told him that there was some filth on them, which is why he (s) removed them. So he told his Companions at that point to check their shoes before coming to a mosque and if they were dirty, to wipe them off and then pray in them.
Given that story, one might expect for Muslims to pray in their shoes in the mosque. And in fact, Muslims were instructed (according to another hadith in Sunan Abu Dawood) by the Prophet Muhammad (s) to pray with their shoes on to differentiate themselves from the Jews.
So returning to the question--why remove our shoes? And the answer has to do with the different nature of our mosques today, and our shoes. Because walking on the carpet with shoes will soil it, we should remove them. And that is the fatwa on this particular issue. Additionally, the carpets are considered waqf, meaning they should be preserved and maintained in good condition. Dirt on the carpets is likely to upset the people who will be praying on them, which is why we should take our shoes off. Praying with shoes.
So, simply because shoes carry dirt, and dirt will soil the carpets on which people prostrate, it is not appropriate to walk on the carpets with shoes. It is perfectly fine, however, to pray with shoes when praying outside the mosque--like at home, at work, in the park, etc.--so long as you ensure they are free from impurities.
At the mosque, on the other hand, if you are taking off your shoes to walk on the carpet, please be sure to neatly stow them on the appropriate racks. :-)