I know what you mean. But in the end it doesn't matter what people think really as long as most of your customers are OK.
Don't worry about the naysayers just keep plodding and you'll get a decent bunch.
Had a customer cancel cos I had charged what she felt was too much for the inside of her shop. Because, you'll like this, 'We haven't had a payrise this year'
FYI not my problem, if you want to work for a pittance love that's your problem