This is a difficult one.
I think you know that you should have informed your custies of your situation. Naturally they will phone up someone else.
If they are phoning the other guy then it is fair to assume he will think you have given up, or not doing a proper job etc. Of course he's going to take them back.
Get round to all your custies and let them know your situation.
If you think the other guy has acted badly then text him your situation. (At least you know he's read it).
Good luck.