Andy,
I started my own business last year and have been with an agency all the time, I may only do the occasional driving job and perhaps some at weekends but I feel that during start up if you can do a bit and not affect the work that you need to do in your own business then why not. My agency does not seem to have a problem I have not worked for them for about 6 weeks yet they phoned Friday evening asking me to do a last minute job, what better work if you can do it. Saves taking money out of your own pocket?
Arran