The Biblical religion was never about tolerance, it was about imposing rules and making demands. Jesus did a bit about turning the other cheek, and not stoning adulterers, but the overall message was live life as commanded or go to Hell.
Including based on thought crime. In some ways thought crime was the worst of it, because disrespecting the incorporeal God was the worst thing you could do. Gather sticks on the Sabbath and you're dead. Maybe Jesus wouldn't have stoned him, but he would've sent him to Hell, infinitely worse.
I don't think the concept of tolerating the intolerant has been around for very long, and maybe it's been more in use as a mocking term than as an actual practice. What I go by is that all people should be as free as they can possibly be, and that there should be as much total freedom as possible, which means the limit is when one person's freedom is increased at the cost of another person having less freedom. So it's akin to the right to swing your fist concept, but it can go well beyond that. I think we can increase the level of freedom in society by having some level of legal duty. Or to put it this way; I think that mandatory taxation used to pay for public services is superior to expecting people to give up their money without being required to. But that does have to be based in democracy, so the challenge is in convincing people that it is the best way.
Imagine, for example, that you had good reason to believe that a monster was on its way to attack your town, slaughtering everyone in its path. You and your fellow townsfolk would be wise to arm yourselves and set up a defensive perimeter. If the danger were clear and present, the monster visible on the horizon, you would rightly see anyone who didn’t participate without a good reason as a no-good freeloader.
Some failures to participate are more dangerous than others. If any noise might attract the monster’s attention, then dancing and reveling of any sort must be forbidden; you put not only yourself at risk, but everybody around you. If it’s a horror-movie monster, attracted by premarital sex, then this might be restricted as well. And what if some kinds of people pose a danger to the town by their very existence? Is it worth the town’s life to let them stay? A town in enough danger might make a moral choice to exile, or even sacrifice, some of its members.
People see fantastical monsters all the time, this is why we need both science and a justice system that can work to nullify the many faults in human perception, and get as close to the truth as possible.
Claiming that there's a monster is far from enough, and the "you're either with us or you're against us"-attitude only speaks to your failure to prove your case objectively. "You're not on my side, so you must be in the wrong".