To complain is always nonacceptance of what is. It invariably carries an unconscious negative charge. When you complain, you make yourself into a victim. When you speak out, you are in your power. So change the situation by taking action or by speaking out if necessary or possible; leave the situation or accept it. All else is madness.
If we are not being maligned, scandalizing those who still sleep in the church, then we are missing the incarnational call to love the world's most vulnerable, the world's exiled, the world's most wounded. Our concern should be what the poor and prostituted think of us, what the God of the universe thinks of us, not what those around us think of us.