I've got a few 'if it's going to rain....' type customers and also a few 'not this month.....' customers also. I see them as bonus work in addition to my regular work as they're near or next to work that gets done regardless. If they want their windows done, I do them but if not, it don't matter.
If those customers were somewhere where I had to drive any distance more than a few hundred yards or were in a place that's awkward to get to, or somewhere where it's difficult parking, etc, then I'd dump them.
For me, I don't have hard and fast rules about these types of customers unless it interferes with my regular work.
I picked up a real corker of a job two months ago and as a result, I've said goodbye to three customers. One is someone who I have to always chase for payment, one is incredibly fussy and the other I just don't like her attitude.