Yes, they can be. It's always wise to consult a physician to rule out all the possible physiological causes before deciding you have a mental disorder. There's even a class of psychiatrist that decided that, hey, we're doctors, and started doing all kinds of tests to rule out physiological problems. They call themselves functional physicians, and estimate that 10-15% at least of all supposed mental disorders are physiological in nature. Things that can cause panic attacks are food allergies (MSG is big at doing this, found in most Chinese food), food intolerance (most common with dairy and wheat and of course Ciliac disease), fibromyalgia, thyroid disorders, adrenal gland problems, hormonal imbalances, blood sugar imbalances, hidden viruses, etc. A lot of these things are very hard to find, and it's much easier for a doctor to just chock it up to mental problems, give you a drug that tamps down the symptoms, and move on. But also know that panic attacks almost always come out of nowhere doing things we've done a thousand times before with no problem. Just like a bad knee, you don't have it until you have it.
Anx is different for everyone, so there are no rules. Have you tried therapy to try to deal with whatever causes the anx?