For what it's worth, my opinion on things like this is do keep smiling.
Normally, certainly for my customers, it's more about how i say it than what i say.
If I'm ploughing through the rain and a customer moans, chirping back something like:
'don't worry, it's the same price' or
'it always makes it feel a bit easier when a customer makes me a cup of tea...white with one, thanks'
All but the most miserable of customers will realise that you're cleaning as a business, and as such, need to make money, whatever the weather.
If people still continue to moan, suck it up until you can afford to lose them. Once you can afford to pick & choose (to a degree) who you clean for, chuck 'em