First off, stress can cause the food allergies, and allergies can cause the anxiety and stress, it is a vicious circle. One reason you may be feeling like your throat is swelling is due to the fear of having an allergic reaction. You see, when we have anxiety, all of our muscles, including those in our throat, constrict, this can cause the "lump' in the throat feeling.
I think you will most assuredly know the difference between an allergic reaction causing your throat to constrict or if it is an anxiety attack, your body just "knows" and you will know what it is, first, the anxiety will not shut off your throat or breathing, the allergy will.
Would be interesting if you can find out what's causing your allergic reaction. If you find out what's the reason you can tell us. The symptoms that you have are very commen in many allergies. Maybe it's not an allergy but an intolerance. There are many possibilities. If you can't find out whats the reason, you should go and see an allergist. Probably he's gonna make a blood test.
last but not least, if you get the diagnosis of having histamine intolerance, (that's the only one that I know in detail) I can recommend Daosin ;)
best of luck!