As easyclean says, it's just the oxidised rubber from the seals discolouring your bristles, I have this happen to me occasionally and so far I have not noticed any problems with the oxidised rubber geting onto the glass, and I always rinse with the brush on the glass.
And as you say, after a while the bristles of your brush gradually become cleaner.
If it bothers you though, dip your brush in a bucket of diluted bleach and wash, I've done that countless times...at the end of the day when I get home mind!
Or just leave the brush to soak in it overnight.
Ian