his a cocky rich bloke who gets me to ring him & when he said says a time to come round his not in. fed up with him
As far as I'm concerned they can be cocky, rich, and a number of other things. But if they mess my money around too often and I can afford to dump them, then they're dumped. I don't dump customers for messing me around in the past but I do if I feel they are likely to mess me around in the future. It sounds like you believe that he's likely to mess you about in the future so maybe you need to dump the job?
I don't find it easy but I always try to keep emotion out of such decisions. That way, I make a better business decision. Please note that I don't always succeed in this regard.