If it is 5 miles away and it is effecting you i have to tell that you are allergic or sensitive to some particular substance/fumes/dust etc. Try to find out if someone is affected in your neighbourhood and you can rise an alarm to the authorities .
Next time see if your neurological problems are starting with the fumes reaching you.
Try anti histamines, if possible go inside a room with exhaust or AC and try to find out for sure if it is related to this.
You can take a break for couple of days and visit your parents place and find out if these symptoms are related to the fumes or not.
Because diagnosis is important and after that you can be sure and go to a Physician for treatment if it is not fumes related or you have to take the drastic step, what care giver has adviced you . Take care!
You ask an interesting, and by no means a foolish question.
The fumes from burning plastics and misellaneous toxic substances can indeed cause neurological damage. It depends upon genetic susceptibility, the nature of the toxic substances, and duration of exposure.
I'm surprises that in this day and ecologically conscious age local authorities continue to permit this.
Whether or not this is connected to your symptoms is a "so many angels on the head of a pin" question.
I would suggest moving, if at all feasible.