Definately dont take it too seriously ! You will always make up for lost customers, and usually with better ones like everyone says.
I'm only into my 5th month and i've only lost two customers to date, one moved and the other was a pensioner who took offence to a price increase. I was angry at the time as he got on a high horse (bless him). I put his price up from £5 to £6 and he had a mad rant about how i had given myself a 50% increase ! - I can only think that he was under the impression that his house was £4 (it was quite small) and putting it up to £6 was well out of order.
Now i have a lot of time for pensioners but this guy was just plain rude and offensive, after the initial onslaught i was fuming at the abuse, i am always friendly and polite to my customers but i now realise that i was silly.
I should have just turned around and walked off mid abuse. Later that same day i picked up three new customers in the same road, all £6 for 15 mins work and all nice people.
I decided then, that any grief from customers and they will be dropped. We don't have to put up with it, its been said many times but there really is plenty of work out there.