I am sorry to hear about what happened to your hamster at the vet! Go to a vet known as an "exotics" vet...or an "avian specialist" is often an exotics vet. My mouse has a malformed bite, and as you know, rodents' teeth never stop growing. They need stuff to gnaw on to keep them trimmed. In the case of your hamster, if his teeth were growing over too much it would make them locked together or even protrude into the opposing gumline. My vet uses Isoflurane (sp?) to put him under anesthesia, and then trims his teeth. There is NO WAY he would be still to try this procedure without anesthesia!! You can try the RMCA.org for vet referrals by particular state. The website is for pet rats and mice, but alot of information is certainly transferable for your hamster! BTW, my vet has used a formulation for "pain" called Medicam. He makes it up special according to the mouses weight. Please find another QUALIFIED vet! My heart goes out to you!
On another note...and I hope not to sadden you, but if your baby is so ill, ( and yes, older) and you decide to put him to sleep, MAKE SURE the vet puts him under the isoflurane anesthesia, before he inserts (administers) the needle with the medication in it that "puts him to sleep" ! If an injection is given just into his abdomen or by the heart stick method, that is NOT the standard. It is considered the most humane way to put them under anesthesia first, other wise it is painful! I recently had to have my female mouse, Miss Muffett, put to sleep in this manner with the anesthesia, and it is the approved way of doing that procedure on small mammals. Please take care, and know you are a wonderful mommy to your little hamster!