Haven't had this problem, but sounds like the battery isn't being charged, faulty or disconnected.
The charger will be supplying enough charge to run to PDA, but the USB won't.
I would look at returning it to its previous owner and getting your money back.
Do a search on the internet to see how difficult it is to replace the battery.
I choose to replace my Zire31 as the battery wasn't holding a charge and got an E2 in perfect condition for £9.50 plus post and packaging. The Palm cost less than a replacement battery for my Zire31 which needed soldering in.