Matt,
I like your way of thinking, I thought the same way when I first started out and sometimes wish I'd stuck do just doing what I could on my own and be happy with it.
Instead people kept on offering me more and more work and I took on more and more staff, currently 19, of which 6 are window cleaners.
Staff is a big headache, or it can be at times, there is always something that needs sorted, they'll only tell you something is broken the day after you needed it for a job, never when it happens, or when it's starting to give up the ghost.
Ultimately responsibility needs to be delegated to those you can trust and empower them to drive the business forward. We have probably 5 staff who literally control what's done and when by the rest of their team, this allows me to move away from being hands on at every job and look at where the next opportunity is.
It can backfire if you get your judgement wrong, and it did on us earlier this year, that has been resolved now

Customers can be a problem, but generally if you provide a good service and good communications at a realistic price then they will be happy and not bother you.
Credit control, we run with an on going outstanding debt of several £,000, you just need to continually chase up the ones who fail to pay on time, again we got stung for nearly £2K last year from a commercial job that went to the wall very suddenly, but we have learnt from it.