It seems unlikely that it is heart disease, given your age and the fact that the ecg was normal. Stress and adreneline can certainly trigger missed beats. Bobad is right though - my own ectopics have caused me to have massive anxiety and panic attacks, seriously affecting my quality of life. Have you had an echocardiogram? This would show if your heart was structurally normal or not - if it is normal, then (from what I've read) ectopic beats are harmless, though in many cases very distressing. If you are in distress about these ectopics, see your cardiologist again, just to put your mind at rest.
Oh, and stop smoking - very, very, very bad for your heart - which you know, don't you?!
Since you have had the ectopics since very young, nothing sounds real suspicious to me. Ectopics that start suddenly or permanently worsen are more likely to be caused by coronary artery disease. The absence of angina is another clue that your heart is probably OK.
Having said that, you are 35 now, and as you know, you're in a higher risk category. I think you need to see your cardiologist and get a stress test. If nothing else, it will set your mind at ease, and perhaps reduce the occurrence of future panic attacks. Panic attacks are not trivial, and can wreck your life just as surely as any serious disease.