I wouldn't worry, Richard. We were threatened early on - big style - face-to-face; two of 'em, and we got the police involved. If you've this guys number, contact the police and let them know; it's important that you do, just in case something else happens; which I doubt it will.
I would say just put it behind you, worrying about it doesn't help, and carry on with what you're doing.
You're obviously doing something right for this gimp to be worried himself.
You will find the vast majority of window cleaners to be nice, friendly types, and the toss pots who aren't generally don't last long. That's been my experience. In my area, despite the odd spat, all the window cleaners get on with each other, and I remember as 'the new guy' getting help and encouragement from other window cleaners.